M MolBench
For human evaluators

Evaluator gallery

What the grader scored vs what a human sees. MolBench uses an escalating grader: a fast scene-tree match, then a pixel-level visual diff, then a VLM judge for the ambiguous cases. Tree-match is a deliberately conservative approximation — the rendered image is the real ground truth. Each card below shows the reference and the model output with both scores, so you can judge whether the grader got it right.

STAGE 1
Tree-match
Deterministic F1 over the MolViewSpec scene tree. Fast, engine-neutral, conservative.
STAGE 2
Visual diff
Pixel/structural similarity of the two renders. Catches tree-mismatches that look identical.
STAGE 3
VLM judge
A vision model adjudicates the remaining ambiguous cases: same scene, or genuinely different?
Models: 15 models (spec run, 2026-10-18) VLM judge: claude-opus-4-8 Rows: 12
Read the featured card first. Its tree scored 0.39 by tree match, yet the scene was judged the same scene by the VLM. Do you agree? The VLM-judged cards are where a human check matters most: the VLM’s verdicts have not yet been validated against human raters.

How cards are picked: 4 per verdict, distinct tasks: lowest tree F1 for rescues, highest tree F1 for VLM-different; ties by model, task.

res-lysozyme claude-haiku-4-5 Most dramatic gap

Show me hen egg white lysozyme

Reference scene
Reference render for: Show me hen egg white lysozyme
Model output
Model output render for: Show me hen egg white lysozyme
MVS score
0.39
Visual score
0.92
VLM: same

Picked: lowest tree F1 among scenes the VLM judged the same. 3,576 pixels differ.

VLM rationale: Both show the same lysozyme structure in cartoon representation with identical secondary structure features (helices, beta sheets, loops). The only difference is color (white/gray vs blue), which is a minor styling difference; the structure, representation type, and focus are equivalent.

grd-ligand-4ins-zn gpt-5.4-nano

Load 4INS; show the protein as a grey cartoon and the ZN as orange ball-and-stick.

Reference scene
Reference render for: Load 4INS; show the protein as a grey cartoon and the ZN as orange ball-and-stick.
Model output
Model output render for: Load 4INS; show the protein as a grey cartoon and the ZN as orange ball-and-stick.
MVS score
0.72
Visual score
1.00
Rendering-equivalent

Picked: lowest tree F1 among renders matching the reference (≤ 2 pixels differ). 0 pixels differ.

int-004 gpt-5.4-nano

Load 1A6M with the protein as a grey cartoon and show all the non-covalent interactions (hydrogen bonds, metal contacts) around the bound oxygen molecule.

Reference scene
Reference render for: Load 1A6M with the protein as a grey cartoon and show all the non-covalent interactions (hydrogen bonds, metal contacts) around the bound oxygen molecule.
Model output
Model output render for: Load 1A6M with the protein as a grey cartoon and show all the non-covalent interactions (hydrogen bonds, metal contacts) around the bound oxygen molecule.
MVS score
0.92
Visual score
1.00
Rendering-equivalent

Picked: lowest tree F1 among renders matching the reference (≤ 2 pixels differ). 0 pixels differ.

int-003 meta-llama/llama-4-maverick

In trypsin (3PTB), show the protein as a grey cartoon and the catalytic triad (His57, Asp102, Ser195) as ball-and-stick. Draw the two triad hydrogen bonds as dashed lines with their lengths.

Reference scene
Reference render for: In trypsin (3PTB), show the protein as a grey cartoon and the catalytic triad (His57, Asp102, Ser195) as ball-and-stick. Draw the two triad hydrogen bonds as dashed lines with their lengths.
Model output
Model output render for: In trypsin (3PTB), show the protein as a grey cartoon and the catalytic triad (His57, Asp102, Ser195) as ball-and-stick. Draw the two triad hydrogen bonds as dashed lines with their lengths.
MVS score
0.93
Visual score
1.00
Rendering-equivalent

Picked: lowest tree F1 among renders matching the reference (≤ 2 pixels differ). 2 pixels differ.

gen-cartoon-2hhb gpt-5.4-nano

Load deoxyhaemoglobin (2HHB) and show the protein as a teal cartoon.

Reference scene
Reference render for: Load deoxyhaemoglobin (2HHB) and show the protein as a teal cartoon.
Model output
Model output render for: Load deoxyhaemoglobin (2HHB) and show the protein as a teal cartoon.
MVS score
0.94
Visual score
1.00
Rendering-equivalent

Picked: lowest tree F1 among renders matching the reference (≤ 2 pixels differ). 0 pixels differ.

res-gfp claude-haiku-4-5

show green fluorescent protein from Aequorea victoria

Reference scene
Reference render for: show green fluorescent protein from Aequorea victoria
Model output
Model output render for: show green fluorescent protein from Aequorea victoria
MVS score
0.52
Visual score
0.83
VLM: same

Picked: lowest tree F1 among scenes the VLM judged the same. 7,201 pixels differ.

VLM rationale: Both images show the GFP beta-barrel structure in cartoon representation with the same fold and secondary structure elements. The only difference is coloring (grayscale vs green), but both depict the same protein, same components, and equivalent cartoon representation. The green color in the output is actually appropriate for GFP.

res-pcsk9 gemini-3.5-flash

show the structure of PCSK9

Reference scene
Reference render for: show the structure of PCSK9
Model output
Model output render for: show the structure of PCSK9
MVS score
0.52
Visual score
0.89
VLM: same

Picked: lowest tree F1 among scenes the VLM judged the same. 6,048 pixels differ.

VLM rationale: Both images show the same PCSK9 structure in cartoon representation with identical fold, including the dashed loop region and beta-sheet domain. The topology and components match; only difference is coloring (grayscale vs. teal/orange), which is an acceptable representational variation of the same visualization.

res-cftr openai/gpt-oss-120b

show me the CFTR channel

Reference scene
Reference render for: show me the CFTR channel
Model output
Model output render for: show me the CFTR channel
MVS score
0.52
Visual score
0.93
VLM: same

Picked: lowest tree F1 among scenes the VLM judged the same. 4,237 pixels differ.

VLM rationale: Both show the same CFTR structure in cartoon representation with identical fold, including the characteristic dashed-line disordered region at the bottom. The only difference is coloring (gray vs blue), which is a minor stylistic variation; the structure, representation, and highlighted regions are equivalent.

grd-disulfide-1lyz gpt-5.4-nano

Load 1LYZ; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.

Reference scene
Reference render for: Load 1LYZ; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.
Model output
Model output render for: Load 1LYZ; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.
MVS score
0.93
Visual score
0.97
VLM: different

Picked: highest tree F1 among scenes the VLM judged different. 376 pixels differ.

VLM rationale: Both show the same lysozyme structure as grey cartoon with yellow ball-and-stick cysteines, but the reference highlights all disulfide-bonded cysteines (multiple clusters throughout), while the output only shows a couple of cysteine residues, missing most of the disulfide bonds.

grd-disulfide-3ptb deepseek/deepseek-v3.2

Load 3PTB; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.

Reference scene
Reference render for: Load 3PTB; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.
Model output
Model output render for: Load 3PTB; protein as grey cartoon, and highlight the disulfide-bonded cysteines as yellow ball-and-stick.
MVS score
0.93
Visual score
0.97
VLM: different

Picked: highest tree F1 among scenes the VLM judged different. 312 pixels differ.

VLM rationale: Both show 3PTB as grey cartoon with yellow cysteine highlights, but the reference displays multiple disulfide-bonded cysteine clusters (6+ yellow sites) distributed across the structure, while the output shows far fewer yellow ball-and-stick residues (only ~2 visible). The output appears to be missing most of the highlighted disulfide cysteines.

int-001 google/gemma-3-27b-it

In oxymyoglobin 1A6M, show the protein as a grey cartoon, His64 and the bound O2 as ball-and-stick, and draw the hydrogen bond from His64 NE2 to the O2 molecule's terminal oxygen (atom O2) as a dashed line labelled with its length.

Reference scene
Reference render for: In oxymyoglobin 1A6M, show the protein as a grey cartoon, His64 and the bound O2 as ball-and-stick, and draw the hydrogen bond from His64 NE2 to the O2 molecule's terminal oxygen (atom O2) as a dashed line labelled with its length.
Model output
Model output render for: In oxymyoglobin 1A6M, show the protein as a grey cartoon, His64 and the bound O2 as ball-and-stick, and draw the hydrogen bond from His64 NE2 to the O2 molecule's terminal oxygen (atom O2) as a dashed line labelled with its length.
MVS score
0.93
Visual score
0.99
VLM: different

Picked: highest tree F1 among scenes the VLM judged different. 98 pixels differ.

VLM rationale: Both show the grey protein cartoon of myoglobin. However, the reference shows the His64/O2 region as a faint grey ball-and-stick with a dashed hydrogen-bond line, whereas the output simply shows an orange blob with no visible dashed hydrogen-bond line or length label. The key highlighted feature (the labelled dashed H-bond) is missing, so the visualizations differ.

gen-focus-1hho deepseek/deepseek-v4-flash

Load 1HHO; grey cartoon protein, magenta ball-and-stick ligands, and focus the camera on the ligand.

Reference scene
Reference render for: Load 1HHO; grey cartoon protein, magenta ball-and-stick ligands, and focus the camera on the ligand.
Model output
Model output render for: Load 1HHO; grey cartoon protein, magenta ball-and-stick ligands, and focus the camera on the ligand.
MVS score
0.90
Visual score
0.58
VLM: different

Picked: highest tree F1 among scenes the VLM judged different. 25,748 pixels differ.

VLM rationale: The reference shows grey cartoon protein with magenta ball-and-stick ligands visible. The output shows only a few dark grey cartoon fragments zoomed in far too close, with no magenta ligands visible at all. The required ligand focus and magenta ball-and-stick representation are absent.