Physics First, Verification Always

A working position on where AI actually belongs in real scientific work — with one small experiment as a case study, not the whole argument

This isn't a takedown of any tool, company, or person. It's a position, developed while deciding where AI/ML genuinely earns a place in a real materials-discovery pipeline, rather than where it's merely convenient to insert one.

The position, briefly

AI and ML models are useful as candidate-narrowing tools — ranking, filtering, prioritizing — never as a replacement for physical reasoning, and never trusted as ground truth without an independent check. A prediction is a hypothesis to verify, not an answer to act on.

This isn't a rejection of AI/ML — several tools used directly in the alloy-lab project are themselves ML- or heuristic-based. The position is about how such tools are used, not whether.

A hierarchy, not a binary

A colleague, Sergei Kalinin, put a related idea well in a recent post: machine learning isn't automatically the best way to analyze data — when a good physical model exists, the physical model usually wins, because it connects observation to cause and tells you not just what happened but why. ML earns its place specifically where the physical model is incomplete, too expensive to evaluate, or where the experiment itself is moving faster than a person can steer it by hand.

That suggests a hierarchy rather than a single rule: use physics where physics is known; use ML to help learn representations where physics is incomplete; use ML for real-time decisions only where the experiment has genuinely outrun human speed — with physics and a human always still in the loop. It's a good complement to the position below: not "AI or not," but "AI for which part, checked how."

Why this position, specifically

Composition-only and structure-only models are interpolation machines, and materials discovery is fundamentally an extrapolation problem. A model ranks plausible candidates well within the space it was trained on, but has no principled way to know when a new composition crosses a phase boundary into territory it's never seen — and a small stoichiometric change can do exactly that.

Simulation-derived training data inherits the simulation's own systematic errors too. A model trained on DFT-derived properties learns DFT's known failure modes alongside whatever real signal exists — high accuracy against a held-out split of the same simulated data can mean the model learned the simulation's biases faithfully, not that it learned physics. The only real test is independent, physical verification: synthesis, characterization, comparison against a certified reference.

And a prediction without a stated validity envelope isn't falsifiable, which means it isn't actionable. "This model predicts stable compositions" means very little without also stating: stable by which measure, verified against what, within which region of composition space. Claims framed as universally applicable, with no boundaries stated, can't be checked — and can't be usefully disagreed with either, since there's no specific claim to test.

What this looks like in practice

The alloy-lab screening pipeline is built around exactly this discipline, as working code rather than an abstract principle:

• VEC / δ / ΔH_mix are interpretable physical heuristics, not fitted models — every number has a direct physical meaning, and the reasoning behind each threshold is documented, not hidden behind training weights.
• The synthesis feasibility check (melt/boil-point based) explicitly separates its outputs into three tiers: a hard physical rule where confident, a comfortable "no concern" case, and — deliberately — a caution tier for the genuinely ambiguous middle, where the honest answer is "check further," not a confident guess dressed up as one.
• The XRD analysis pipeline was validated against a certified external reference standard (NIST SRM 660, LaB6), not just checked for internal self-consistency — internal consistency alone can't distinguish a correct pipeline from one that's consistently wrong.
• A peak-matching result was walked back mid-project, when a second, independent check (intensity ratio) contradicted an initial, single-criterion conclusion (position match alone) — documented in the project history rather than quietly dropped.

The full technical manifesto behind these choices — including what this project asks of any AI/ML tool it adopts — lives alongside the Alloy Lab Database write-up.

Case study: a small experiment in the opposite direction

To see the failure mode this position guards against, I ran a small experiment on a completely different genre: the confident, beautifully designed post claiming to lay out the definitive workflow or stack for doing something, styled to look hard-won. To be upfront: this isn't a criticism of open source, which I genuinely believe in — it's a criticism of a content genre that happens, right now, to often be dressed up in open-source clothing.

I asked a chatbot to draft a layered "blueprint" the same way these posts present themselves — confident categories, named components, a clean structure. It produced the whole thing in well under a minute. Checking the individual pieces it named gave mixed results, though a more thorough recheck afterward showed the split wasn't as even as it first looked: most of what it named turned out to be real, if often small, obscure, or very recently published — genuine repositories, and in one case a peer-reviewed paper. A couple of pieces pointed to something real but not at all what the post implied (a personal write-up rather than an established framework, in one case; a well-known, legitimate name attached to invented specifics, in another). Only one had no independent trace at all.

None of that means anything sinister happened. It means an LLM asked to produce a confident stack will blend genuine, if obscure, references with names it's pattern-completing into existence, closely enough that the two aren't distinguishable without someone actually checking — and, in this case, checking properly took more than one pass to get right.

The corrected, fully-checked version of that experiment is written up separately, as its own reference rather than a footnote here: A Verified Open-Source Stack for AI Agents — the same four layers, but with every claim actually confirmed against its source.

Three checks before trusting any "ultimate" post

• Can you find independent evidence for the specific things it names? Not just a plausible-sounding name — an actual trace of it existing and being used by someone other than the author.
• Does the post distinguish "established" from "brand new"? Early, experimental, and solo-built things are fine — but a post presenting all of them with identical confidence is hiding exactly the information you need.
• Would the claim survive you asking an AI to generate a competing version of it in a minute? If a chatbot can produce an equally confident alternative "definitive" answer just as fast, "definitive" was never really on the table.

Why this matters more than it might seem

The same discipline applies whether the claim is a "definitive tech stack" post or a materials-discovery model's prediction: a validity envelope, independent verification, and auditability aren't bureaucratic overhead — they're what makes a claim checkable at all. I'd rather move slower and stay checkable than move fast on a claim nobody can verify, whether that claim came from a minute of pattern-completion or a well-trained model quietly extrapolating past the edge of what it actually knows.