AI Evolves to Achieve Artificial Innovation
The debate is over: AI can innovate, invent, and create new things. AlphaEvolve is the latest and greatest AI innovation engine. More AI inventions to come.
The Debate is Over, AI Can Innovate
A recent New York Times article “Why We’re Unlikely to Get Artificial General Intelligence Anytime Soon” from Cade Metz presented a skeptical view on timelines to get to AGI:
… many technologists have grown increasingly bold in predicting how soon A.G.I. will arrive. Some are even saying that once they deliver A.G.I., a more powerful creation called “superintelligence” will follow.
As these eternally confident voices predict the near future, their speculations are getting ahead of reality. And though their companies are pushing the technology forward at a remarkable rate, an army of more sober voices are quick to dispel any claim that machines will soon match human intellect.
Aidan Clark (from OpenAI) summed the article on X as “people think AGI is near, but no one can point to a scientific reason it will happen soon.” Problem: Actual AI progress has outpaced claims of what AI can or cannot do.
Claims that AI cannot innovate and invent new things led Nate Jones to rebut this on YouTube, pointing out claims that AI cannot innovate are false because it already has. AI systems are already delivering solutions humans have never conceived before.
Google DeepMind has been a provider of several of these innovative inventive AI:
DeepMind’s AlphaDev in 2023 discovered brand-new faster sorting algorithms.
DeepMind’s GNoME in 2023 discovered thousands of new materials are reshaping biology and materials science by predicting structures at super-human scale and speed.
In 2020, MIT used deep learning to discover a new antibiotic compound, Halicin, showing how AI uncover new antibiotics. The AI discovery process leans in on power of automated search and predictive computer models:
The computer model, which can screen more than a hundred million chemical compounds in a matter of days, is designed to pick out potential antibiotics that kill bacteria using different mechanisms than those of existing drugs.
DeepMind’s AlphaFold has gone from solving the protein-folding problem to creating a 200 million protein atlas. Their latest AlphaFold release is AlphaFold 3, able to “predict the structure and interaction of all life’s molecules.” AI is revolutionizing biological science.
DeepMind’s Alpha Geometry is an innovative geometry problem-solver that can solve Math Olympiad-level geometry problems.
Beyond DeepMind, notable uses of AI for invention include:
Microsoft's MatterGen and MatterSim employ generative AI to propose novel molecular structures with desired properties and then computationally validate their stability and viability.
NASA designed innovative titanium mounts with AI design software.
Johns Hopkins researchers are using AI and machine-learning to find materials that can withstand extreme heat as components of a hypersonic vehicle.
There are many more examples, but as Nate Jones mentions, most AI innovations are more prosaic business innovations, such as UPS using AI to optimize delivery routes, reducing miles driven by millions and saving millions in fuel and delivery costs yearly. AI tools improve productivity of knowledge workers, which accelerates innovation in business and science.
Nate puts it this way:
“Arguing about whether AI can create at all is yesterday’s fight—the data has already decided it.”
The Importance of Alpha Evolve
DeepMind’s release of Alpha Evolve last week puts the exclamation points on DeepMind’s record of AI that can generate innovative solutions. In our latest AI weekly news review, we reported on Alpha Evolve and mentioned some the ‘earth-shattering results’ they achieved:
When presented with 50 open problems across mathematics, geometry, combinatorics, and other fields, Alpha Evolve rediscovered the best known solutions in 75% of cases and improved upon existing solutions in 20%.
AlphaEvolve shattered a 56-year-old record by improving on Strassen’s algorithm for matrix multiplication.
It also advanced mathematics knowledge with an improved lower-bound solution for the 11 dimension kissing number problem.
AlphaEvolve has paid for itself by improving Google’s data center compute orchestration, finding a better heuristic that clawed back 0.7% of Google's compute capacity, saving millions.
DeepMind’s prior inventive AI models were ‘narrow AI’ problem solvers in that the AI solved a specific problem domain, albeit some important ones like materials (GNoME) and biology (AlphaFold). In contrast, AlphaEvolve addresses the much more general domain of algorithms. As an improvement on DeepMind’s prior work (StepFun) in algorithm discovery, AlphaEvolve is the most powerful and most general AI for innovation.
Algorithms are recipes for doing tasks, solving computations, and completing work processes. Much of what we do, from a cooking, to business decisions, to mathematical proofs, can be characterized as an algorithm. By generating and optimizing algorithms, AlphaEvolve is a general-purpose problem solver across a range of fields.
Alpha Evolve - Evolutionary Algorithm
It’s not just that AlphaEvolve is general because algorithms are general, but the Alpha Evolve evolutionary process itself is a general approach to discovery and optimization problem-solving. Pairing generative AI to evolutionary algorithms is a powerful and general paradigm.
Optimization problems are pervasive and important in business (production planning), design (semiconductor chip layout, engine materials design), and science (chemical reaction analysis).
While simple or linear problems can be solved with convex optimization, more complex optimization problems, where a local minimum might not be the global optimum, are best solved using evolutionary or genetic algorithms.
Such algorithms are based on a conceptually simple iterative loop: Generate a new solution, based on refinements of good prior solutions; evaluate its quality or fitness based on desired metrics; use that evaluation to keep the best solutions or solutions in the population (genetic algorithms keep a population of solutions). This general technique just needs measurable metrics to guide the optimization.
AlphaEvolve employs an LLM-based mutation and selection mechanism. It starts with an initial pool of candidate algorithms, which can be existing solutions or human-written starting points. Directed by a prompt sampler, the LLM (AlphaEvolve uses Gemini 2.0 Flash and Pro to do this) then generates modifications (the “mutations”) to these algorithms, in the form of code changes. These modifications aren't random; the LLM uses its understanding of code and problem-solving to suggest potentially beneficial changes.
Following the mutation phase, AlphaEvolve evaluates the modified algorithms. This evaluation uses LLMs to evaluate, and it can involve direct code execution to test performance, correctness, and efficiency against defined metrics. This is far more complex than calculated metrics in typical optimization algorithms. The paper mentions taking up to 100 compute-hours to evaluate a potential solution through use of parallelized execution.
Based on this feedback, a selection process (also guided by an LLM) determines which algorithms “survive” and proceed to the next generation for further refinement. As with other evolutionary algorithms, AlphaEvolve maintains a population of solutions with their evaluation results attached. This iterative loop of mutation, evaluation, and selection allows AlphaEvolve to explore the solution space and converge on highly optimized algorithms.
Conclusion - AlphaEvolve as an AI Innovation Engine
We started by pointing out that AI is already able to innovate, sharing examples of AI innovation going back several years. However, many examples of prior AI model innovations were limited and in narrow domains. DeepMind’s Alpha Evolve opens up the capability for more general-purpose AI-based invention.
AlphaEvolve’s ability to autonomously discover novel and more efficient algorithms is both powerful and general. Both the domain AlphaEvolve addresses, optimizing algorithms, and the evolutionary algorithm approach it takes are general-purpose. A well-defined optimization target is all you need.
Coincidentally, reinforcement learning (RL) has the same characteristic. Reasoning AI models are advancing in science, math, and coding because the reward function used in RL is grounded in the factual correctness of a proposed solution. Ask for a proof of a math theorem and the proof can be verified; code can be executed and tested. Expect rapid advances in AI capabilities in areas where measurable metrics can be taken to guide training and optimization.
Advancing the fundamental design of algorithms will accelerate scientific discovery and technological innovation, and this includes accelerating AI progress itself. AlphaEvolve improved Google’s infrastructure and sped up Gemini model training by 1% by discovering “a heuristic that yields an average 23% kernel speedup across all kernels over the existing expert-designed heuristic.”
AI can invent, AI has developed some inventions, and in the near future, AI will do a lot more inventing. AlphaEvolve is just the first in a class of AI Artificial Innovation engines yet to come.