MolJSON: Teaching Language Models to Speak Molecule
Deep Dive | Edition 22
Welcome back to the deep dive, where we break down the AI tools and data reshaping how new drugs are discovered. In each edition, we speak directly with the teams behind these tools to explain what they solve, how they work and where they are going next.
Why the molecular format you feed a language model matters as much as the model itself, and what a five-day experiment over New Year’s proved about it.
For computational chemists, drug discovery scientists, and anyone building or evaluating agentic AI systems that touch molecular structures.
Language models can now reason about chemistry, yet they still routinely fail at writing molecules correctly. The bottleneck is not intelligence. It is representation: SMILES and IUPAC were designed for humans and databases, not for models that generate text token by token.
Across a benchmark of 78,000 questions, MolJSON consistently outperformed existing formats. On the constrained generation task, GPT-5 achieved 95.3% accuracy outputting MolJSON (a simple atom-and-bond list in JSON), compared with 64.0% for SMILES and 76.3% for IUPAC. The format appears in zero training corpora. The advantage is purely structural.
This piece covers where existing representations break down, how MolJSON works, why the numbers are not close, and what it means for anyone building tools that ask language models to propose or modify molecules
Kiin Pioneer Programme
We built a platform that helps researchers speed up their entire science, from literature review and biomarker discovery to bioinformatics and computational chemistry. If your workflow involves pulling findings from five different places before you can actually act on any of them, this is for that.
The Pioneer Programme gives academic labs and non-profits one year of free access, plus support from our science team. No cost, no data transfer, all IP stays with your institution. Applications close August, cohort starts September.
This week we spoke with Nicholas Runcie, a DPhil student in statistics at Oxford, about a format for molecular representation that started as a five-day sprint over New Year’s and ended up reshaping how his group thinks about language models in chemistry.
The problem: models that can reason but cannot write
In January 2025, Nicholas Runcie set out to prove that language models could not do chemistry. He built ChemIQ, a benchmark of 816 questions spanning everything from counting carbon atoms to solving NMR spectra. The results surprised him. When DeepSeek R1 and OpenAI’s o3-mini arrived with chain-of-thought reasoning, they scored 50-57% on tasks where previous models had achieved 3-7%. On IUPAC naming, a task where earlier models scored near zero, reasoning models reached 29-44%. They were not just pattern-matching. They were building internal representations of molecular graphs and reasoning about them step by step.
The problem was reliability. A model might reason beautifully about a molecule’s structure, then botch the output because SMILES notation requires you to linearise a graph into a single traversal path with nested parentheses and ring-closure digits. Get one bracket wrong and the molecule is invalid. IUPAC names are just as problematic: they demand perfect recall of nomenclature rules, locant numbering, and substituent priority. These formats were designed for chemists reading printed pages and for databases storing canonical strings. They were never designed for sequence models generating text token by token.
Any agentic system that needs to propose, modify, or communicate molecules needs its language model to write structures reliably. If the representation introduces systematic errors, the pipeline breaks downstream.
Figure 1: Six ways to say ‘acetic acid’. Only MolJSON explicitly lists atoms and bonds without requiring the model to linearise the molecular graph or recall nomenclature rules.
The idea: just write out the atoms and the bonds
Instead of encoding a molecule as a traversal (SMILES) or a name (IUPAC), write it as a JSON object with two arrays: one listing atoms with identifiers and elements, another listing bonds with source, target, and order. No traversal path to maintain. No nomenclature rules to recall. No ring-closure digits to track.
Want to add a methyl group to a benzene ring? Add one entry to the atoms array and one to the bonds array. In SMILES, you would rewrite the entire string. In IUPAC, you would recalculate every locant and potentially rename the parent chain.
{
“atoms”: [
{”id”: “C1”, “element”: “C”},
{”id”: “C2”, “element”: “C”},
{”id”: “O1”, “element”: “O”},
{”id”: “O2”, “element”: “O”}
],
“bonds”: [
{”source”: “C1”, “target”: “C2”, “order”: 1},
{”source”: “C2”, “target”: “O1”, “order”: 2},
{”source”: “C2”, “target”: “O2”, “order”: 1}
]
}Acetic acid (CH3COOH) in MolJSON. Two carbons, two oxygens, three bonds. The entire molecule is a readable list with no special syntax to memorise.
The format also exploits something about how modern language models are trained. Structured JSON output is a first-class capability in every major commercial model because it underpins function calling and agent tool use. Models have been explicitly optimised to produce valid JSON conforming to a schema. MolJSON rides that existing capability rather than asking models to master a chemistry-specific syntax they encounter rarely in training data.
Runcie had the idea in January 2025 but shelved it. SMILES and IUPAC were good enough at the time. The format only came to life because OpenAI gave him a grant of API credits with a New Year’s Eve deadline. With five days and $4,000 to spend, he designed and ran every experiment without iteration. The results in the paper are exactly what he obtained on his first attempt.
Why it’s different: the numbers are not close
Runcie’s benchmark tested three tasks across 78,045 questions using GPT-5-nano, GPT-5-mini, GPT-5, and Claude Haiku 4.5. The molecules were sampled from PubChem deposits made after October 2025 to reduce contamination from training data. Three findings stand out.
Translation. When asked to convert a molecule from one representation to another, GPT-5 achieved 71.0% accuracy translating IUPAC names to MolJSON, compared with 43.7% translating the same IUPAC names to SMILES. The gap held across all model sizes and both directions.
Shortest-path reasoning. Given a molecule and asked to count bonds between two halogen atoms, GPT-5 answered 98.5% of questions correctly when the molecule was presented in MolJSON, versus 92.2% for SMILES and 82.7% for IUPAC. MolJSON also used roughly 1.8 times fewer reasoning tokens than SMILES on this task, meaning the model spent less compute parsing the structure before it could reason about it.
Constrained generation. When asked to generate a molecule satisfying a set of structural constraints (specific atoms, ring counts, shortest-path requirements), GPT-5 achieved 95.3% accuracy outputting MolJSON, versus 64.0% for SMILES and 76.3% for IUPAC. For molecules with a fused ring system, GPT-5 hit 92% accuracy in MolJSON, compared with 42% for SMILES.
The pattern is consistent: MolJSON outperforms formats that language models have been heavily exposed to in training. SMILES strings are ubiquitous in PubChem, ChEMBL, and ZINC. IUPAC names appear in millions of papers. MolJSON appears in none of their training corpora. The advantage comes from structural alignment between the format and how models process information, not from data familiarity.
SMILES and IUPAC accuracy also degrades with molecular size and ring complexity. Fused ring systems cause particular problems for IUPAC. MolJSON remains robust because it represents the graph directly rather than encoding it through a traversal or naming convention.
The future: from format to fully autonomous chemistry
Runcie is clear that MolJSON as published is not a finished standard. He has already heavily modified the format for his current work, keeping the core principle (explicit atom and bond lists) while optimising for specific use cases. He has moved from JSON to YAML for compatibility with Codex agents, and he is currently exploring strategies for encoding stereochemistry and other molecular features
The more interesting signal is where this is heading. Runcie is building agents for autonomous chemistry research, and MolJSON is a core part of that framework. Reliable molecular I/O is what lets an agent propose a structure, check it, modify it, and hand it to the next step without a human correcting format errors along the way.
This fits within Runcie’s broader research programme: developing systems capable of fully automated chemistry research. If a computer were to design a drug, how would it do it? His hypothesis is that language models combining broad knowledge with structured reasoning may solve chemistry problems humans have not yet imagined. MolJSON is one piece of that infrastructure, ensuring the model can reliably read and write molecular structures so that higher-level reasoning is not bottlenecked by format errors.
Kiin’s view.
The most important thing about this paper is not MolJSON itself. It is the demonstration that molecular representation is a first-order design decision when building LLM-based chemistry systems, and that the defaults everyone reaches for (SMILES, IUPAC) are measurably suboptimal. The field has spent years fine-tuning models and building agentic scaffolding while leaving the input/output format as an afterthought. This paper puts a number on what that costs: a 30-percentage-point accuracy gap on constrained generation, same model, same molecules.
These are also first-attempt results with no prompt engineering or iterative refinement. Someone thinking harder about schema design or hybrid representations could likely push them further. That is Runcie’s explicit point: this is a starting direction, not a destination. For anyone building tools that ask language models to propose or modify molecules, the takeaway is immediate. Test your representation. The gap between “works sometimes” and “works reliably” may be a schema change, not a model upgrade.
GitHub: github.com/oxpig/MolJSON
Thanks for reading Kiin Bio Weekly!
💬 Get involved
We’re always looking to grow our community. If you’d like to get involved, contribute ideas or share something you’re building, fill out this form or reach out to me directly.
Subscribe now to stay at the forefront of AI in Life Science and keep up with this upcoming season of deep dives.
Connect With Us
Have questions on this or suggestions for our next deep dive? We’d love to hear from you!
📧 Email Us | 📲 Follow on LinkedIn | 🌐 Visit Our Website