Why is AI progress uneven? a16z and Replit founders: Verifiability is the key difference.

Renowned venture capital firm a16z founder Marc Andreessen and Replit founder Amjad Masad had a discussion on 10/25, focusing on the current state of AI development. Andreessen pointed out that the progress of AI in programming, mathematics, and simulating physical phenomena is astonishing, but there are stagnations in fields such as healthcare and law. Masad responded that the key is not the difficulty, but the verifiability. In other words, tasks that can be objectively judged as right or wrong progress the fastest, while more ambiguous areas are harder to breakthrough.

AI has two speeds: the verifiable runs faster than anyone else.

Masad first reviewed the model training method. Early language models simply looked at the text and then guessed the next character. Although they could speak, they didn't truly reason. It wasn't until the introduction of reinforcement learning (Reinforcement Learning, RL) that AI began to learn in verifiable environments. He explained:

“For problems like debugging, having unit tests, or producing clear results, AI can directly know whether it is right or wrong.”

This instant feedback training method allows AI to quickly enhance its problem-solving abilities.

Why can't the medical and legal fields keep up?

Andreessen asked, “What about fields like healthcare and law? Why does it seem like progress is so slow, and there are even stagnation phenomena?” Masad replied:

“Because of them, there is no single answer.”

For example, Masad, medical diagnoses may have multiple causes and variations in progression, and legal judgments may differ due to differences in judges and cases, which means they are too flexible and lack a clear standard answer. This makes it difficult for the model to self-validate and unable to establish a reinforcement loop.

Although reinforcement learning from human feedback (RLHF) can assist the model, it still belongs to subjective evaluation, unlike mathematics or programming which are direct and clear.

( Note: Reinforcement Learning from Human Feedback ( RLHF) is a type of machine learning ( ML) technique that uses human feedback to refine ML models to enhance self-learning efficiency. )

The real key is not “difficulty” but “verifiability.”

Andreessen sorted out his observations and said, “So the point is not whether the question is difficult or not, but whether you can confirm the correct answer?” Masad says:

“Yes, the explosion of AI in programming and mathematics is not because it is simpler, but because it can verify results.”

For example, when writing code, as long as the program can compile and the tests pass, AI can immediately receive feedback of the “correct answer,” allowing the model to automatically practice thousands of times every day and evolve quickly.

What other fields have verifiability?

The two people listed several fields where AI is progressing the fastest:

Mathematics and Physics: There are clear equations and simulation results.

Chemistry and Biology: Processes such as protein folding and gene sequencing can be validated through experiments or simulations.

Robot: The success or failure of the mission can be directly quantified.

These all belong to the realm of “objective verification,” thus becoming the steepest learning curve for AI at present.

AI for programming will soar to great heights first, while medicine and law are still on the way.

Masad said in summary:

“AI that writes code will soar first, followed closely by mathematics, physics, and chemistry. However, in more abstract fields like medicine and law, it will take time to catch up.”

Andreessen also echoed:

“This is a natural phenomenon. AI first erupts in areas that can be quantified, and will lag behind in fields where human judgment is ambiguous.”

Why is AI progress uneven? a16z, Replit founders: verifiability is the key difference. Originally appeared on Chain News ABMedia.