General Diffusion

GeneralDiffusionisbuildingfrontiermodelsforheterogeneouscomputephysics.

WeWearearepursuingpursuingaacentralcentralquestion:question:cancanmachinesmachineslearnlearntotounderstandunderstandcomputationcomputationthethewaywaythetheworld'sworld'sbestbesthardwarehardwareengineersengineersdo?do?

TheThenextnextgenerationgenerationofofsupercomputesupercomputeisisheterogeneous:heterogeneous:CPU,CPU,GPU,GPU,ASIC,ASIC,andandFPGAFPGAoperatingoperatingsimultaneouslysimultaneouslyatatgigawattgigawattscale.scale.TodayTodaythatthathardwarehardwareisisorchestratedorchestratedbybyarmiesarmiesofofhumanhumanspecialists.specialists.TheTheefficiencyefficiencylostlostininthethegap,gap,thetheMismatchMismatchTax,Tax,isismeasuredmeasuredininbillionsbillionsannuallyannuallyandandgrowsgrowswithwitheveryeverynewnewchipchiparchitecture.architecture.

RecentRecentbreakthroughsbreakthroughsininmodelmodelarchitecturearchitecturehavehavedemonstrateddemonstratedthatthatdynamic,dynamic,context-awarecontext-awareroutingroutingacrossacrosscomputationalcomputationalstatesstatesproducesproducesdramaticallydramaticallysuperiorsuperiorintelligenceintelligenceoutcomesoutcomescomparedcomparedtotostaticstaticpipelines.pipelines.GeneralGeneralDiffusionDiffusionisisestablishingestablishingthatthatthethesamesameprincipleprincipleholdsholdsatatthethephysicalphysicalhardwarehardwarelayer.layer.

TheTheimplication:implication:superintelligencesuperintelligencewillwillnotnotemergeemergefromfrombiggerbiggermodelsmodelsororfasterfasterchipschipsalone.alone.ItItwillwillemergeemergewhenwhenthethecomputecomputethosethosemodelsmodelsrunrunononisisfinallyfinallyasasintelligentintelligentasasthethemodelsmodelsthemselves.themselves.ThatThatisisthetheproblemproblemGeneralGeneralDiffusionDiffusionisissolving.solving.

ComputationComputationacrossacrossphysicallyphysicallyheterogeneousheterogeneoushardwarehardwareobeysobeyslearnablelearnablelaws.laws.WeWeareareestablishingestablishingthosethoselaws,laws,advancingadvancingthetheco-evolutionco-evolutionofofheterogeneousheterogeneouscomputationcomputationandandintelligence.intelligence.