This story starts with the OpenADMET PXR challenge, and with an approach that worked before I really understood why it worked.
I had come across a Nature Communications paper by Buterez et al. on using low-fidelity measurements to improve predictions for a more expensive, high-fidelity endpoint. The idea was appealingly simple: do not throw away a cheap, imperfect assay merely because it is not the assay you ultimately care about. Use it to shape a representation, then learn the expensive endpoint on top.
For PXR, the cheap assay was a single-concentration high-throughput screen. I fine-tuned ChEMeleon on those measurements, extracted its molecular embeddings, and used them as the inputs to TabPFN for the final pEC50 prediction. It worked.
I was not the only person to find variations on this idea. During the challenge post-mortem I saw one high-ranking approach use the single-concentration model’s predictions as features rather than only its embeddings. The winning approach used an SVR to map the single-concentration signal to pEC50 and generated pseudo-labels. Different implementations, but the same intuition: the lower-fidelity assay knows something useful about the higher-fidelity one.
That should have been the end of the story. But it left me with the more interesting question: what was the model actually learning?
Was ChEMeleon benefiting because the single-concentration labels are genuinely related to the dose-response curve from which pEC50 is estimated? Or was it simply seeing many more molecules near the chemistry we eventually wanted to predict? That second possibility was not far-fetched. An OpenADMET post-mortem had found that increasing the density of nearby analogue-like molecules—even with generated analogues—could improve ChEMeleon predictions. Perhaps the extra PXR screen was doing the same thing.
This post is my attempt to separate those explanations.
The data: one expensive endpoint, one cheaper proxy
The endpoint I ultimately care about is pEC50, estimated from a proper dose-response curve. These data are expensive; the PXR training set contains 4,139 compounds with usable curves.
Alongside it is a larger, weaker source: 10,747 compounds screened once at 8.25 µM, reported as log2 fold-change. A single measurement is not pEC50. It samples one point on a response curve, with all the usual limitations of dynamic range, baseline, and assay context. But it is an activity measurement of the same target, and it has a direct—if imperfect—relationship to the curve that produces pEC50.
There is also a three-concentration version of the auxiliary task (8.25, 33, and 99 µM). It gives the model a little more of the shape of the response, without becoming a full dose-response experiment.
Two evaluation surfaces, for two different questions
I evaluated every condition on two surfaces because one split cannot answer both parts of the question.
S1 is a per-seed 80/20 holdout from the 4,139 dose-response compounds. It is the mechanism surface: a conventional in-distribution test, useful for asking whether a condition changes prediction under matched chemistry. Some S1 test compounds also have their own single-concentration measurement. I call these subset A. The rest, subset B, have no weak label. This lets us see how much of S1 is direct transfer of a measurement from the exact compound, and how much is genuine generalisation.
S2 is the 513 challenge analogues. This is the surface closest to the original challenge: potency-enriched, distribution-shifted chemistry. Crucially, none of these 513 molecules has a single-concentration label. Any improvement on S2 therefore cannot be the model seeing a lower-fidelity measurement for that same molecule; it has to be mediated through the learned representation and chemically related training compounds.
So S1 helps us dissect directness. S2 asks whether the trick is useful for the deployment setting that motivated it in the first place.
Two explanations, and controls designed to break them
I began with two competing explanations.
Hypothesis 1: activity semantics. The single-concentration value carries a real PXR activity signal. Fine-tuning on it teaches the representation something genuinely useful for pEC50.
Hypothesis 2: chemical exposure. The large auxiliary screen expands the chemistry ChEMeleon sees. The benefit arises, at least partly, from denser and better-aligned chemical coverage—not necessarily from the activity values themselves.
The conditions change one ingredient at a time, so the contribution of each can be tested directly.
The experimental logic. Each column changes one possible explanation while holding the downstream TabPFN model and evaluation design fixed.
Before asking why: is the effect real?
The first possibility to rule out was much more mundane: perhaps the apparent transfer benefit was just a favourable challenge split, model stochasticity, or noise in the comparison.
For each condition, I used the same five random seeds and the same S1 train/test split for every model. The frozen baseline was also run on those shared seeds, so comparisons are paired rather than comparisons of unrelated averages. S2 contains the identical 513 molecules for every seed and condition. I tested each condition against the frozen baseline with Dunnett-adjusted comparisons, controlling the multiple-comparison problem across the condition grid; paired contrasts support the specific planned comparisons.
The result survives that test. With the single-concentration activity labels, mean MAE improves from 0.479 to 0.417 on S1 (Dunnett-adjusted p = 3.8 × 10⁻⁵), and from 0.527 to 0.477 on S2 (p = 6.3 × 10⁻⁹). The three-concentration labels do better again: 0.398 on S1 and 0.469 on S2. This is a repeatable effect across shared seeds, not a one-off favourable split.
Raw MAE across the five shared seeds. Lower is better. The plot establishes that both real-label transfer conditions beat the frozen encoder on both evaluation surfaces.
With an effect worth explaining, I could move on to the mechanism.
Hypothesis 1: does the activity label itself matter?
The full single-concentration condition is the reference: 10,747 PXR molecules, each paired with its real log2 fold-change.
I then made two deliberately unhelpful variants.
Shuffled activity keeps the same molecules, the same label distribution, and the same optimisation budget, but randomly permutes the activity values across molecules. If this still helped, the story would be generic extra training rather than transfer of a meaningful signal.
Descriptor targets on the same PXR molecules replace activity with Mordred descriptors—information-rich targets that are close to ChEMeleon’s native pretraining task. This is a stronger control than simply withholding labels: it asks whether any structured supervision on the correct PXR chemistry is enough.
Neither substitutes for real activity. Shuffled activity is actively harmful: in the paired-delta view it raises S2 MAE by +0.109 relative to frozen (p = 1.7 × 10⁻¹²). Descriptor supervision sits at baseline on both surfaces. The full real-label condition is therefore not winning because it receives more updates, or because the encoder is merely re-learning generic molecular descriptors. The relationship between molecular structure and PXR activity matters.
This figure is plotted as paired ΔMAE versus frozen: negative is an improvement and positive is harm. Destroying the structure–activity pairing reverses the benefit; descriptor supervision recovers none of it.
That supports Hypothesis 1. But it does not tell us whether the identity of the molecules also matters.
Hypothesis 2: is deployment-aligned chemical exposure necessary?
To test that, I kept real activity labels but changed which auxiliary molecules were allowed into the fine-tune.
Test-scaffold-disjoint removes weak-label molecules near the S1 holdout scaffolds. Analogue-disjoint removes molecules near the 513 S2 challenge analogues. All-DRC-disjoint removes molecules near every dose-response compound, leaving a deliberately chemistry-misaligned auxiliary set. Finally, size-matched random retains the same number of auxiliary molecules as the all-DRC-disjoint condition, but selects them randomly. That last pair isolates composition from simple sample count.
The pattern is exactly what Hypothesis 2 predicts. Removing S2-near chemistry reduces the S2 benefit. More decisively, all-DRC-disjoint training raises S2 MAE by +0.023 versus frozen, while the equally sized random subset is effectively neutral (−0.002). Those numbers are the quantities shown in the paired-delta plot below—not raw MAEs—and the difference is not a size effect. A small random sample is not useful, but systematically excluding challenge-relevant chemistry is worse: it steers the representation away from where it needs to work.
Paired ΔMAE versus frozen. Negative values are gains. The equal-size random and all-DRC-disjoint conditions separate dataset size from chemical composition.
Hypothesis 2 is therefore necessary, but not sufficient. The descriptor controls had the right molecules without an activity target and did nothing. Real activity on chemistry that is irrelevant to the eventual task is harmful. The model needs both ingredients at once.
Direct transfer is part of the story, not the whole story
S1 lets us see why the effect is larger in some places than others. In subset A, the test compound has its own weak label. The model has seen a low-fidelity measurement of that exact molecule, so a strong gain there is entirely expected. Under the one-concentration condition, A improves by 19.6% in MAE relative to frozen.
Subset B closes that route: those S1 test compounds have no single-concentration observation. S2 closes it completely: all 513 analogues lack one. The gain becomes smaller, but it does not disappear on S2. This is not hidden leakage—it is the point of evaluating both surfaces. The design quantifies the direct-measurement route separately from chemistry-mediated transfer.
Direct measurement transfer explains the largest gain. The non-overlap subsets show what remains once that direct route is unavailable.
A more informative weak assay helps more
The three-concentration condition is a final consistency check. It uses the same 10,747 molecules as the one-concentration condition, but preserves activity at 8.25, 33, and 99 µM. It is therefore a clean test of whether a richer low-fidelity view of the response helps.
It does. Relative to the one-concentration condition, mean S1 MAE falls by a further 0.019 (paired p = 4.1 × 10⁻⁴); S2 improves from 0.477 to 0.469. The full-train replication points in the same direction: on the 513 analogues, three concentrations reaches MAE 0.461, versus 0.477 for one concentration and 0.519 for frozen ChEMeleon.
The richer auxiliary measurement improves both surfaces. A single concentration is useful, but it is not equivalent to a dose-response curve.
The ordering persists when all dose-response training compounds are available: frozen < one concentration < three concentrations.
What I think the original trick was buying us
The answer is not “more data helps”. It is more specific: weak-to-strong transfer needs activity semantics on chemistry that overlaps the eventual deployment space.
Real activity labels are necessary: shuffle their molecular pairing and performance gets worse; replace them with generic descriptor supervision and the gain vanishes. Relevant chemical exposure is also necessary: remove it systematically and the gain shrinks or becomes harm. The three-concentration result strengthens the interpretation, because retaining more response information gives the model a better starting point.
This is not a claim that ChEMeleon has uncovered some deep biological mechanism of PXR activation. The evidence supports a practical joint dependency: the auxiliary task must carry a target-relevant activity signal, and it must be sampled over chemistry that connects to the molecules we later care about.
The complete grid
Here are the full results: every condition and reported metric, first as raw performance and then as a change from frozen ChEMeleon.
Raw performance. Lower MAE is better; higher R² and Spearman correlation are better. Colours encode magnitude and cells retain the values.
Change relative to frozen ChEMeleon. This is the compact record of what each intervention did, rather than just which condition scored best.
It will be interesting to see whether the same relationship holds in the upcoming CYP inhibition challenge, or whether it exposes a deeper question about assay harmonisation: when do measurements from related assays become mutually useful, and when do their differences overwhelm the shared signal?