We've now post-trained four Nemotron open models on Harvey's Legal Agent Bench (LAB): Nemotron 3 Super, Ultra, 3.5 Lightning, and now Nemotron 3.5 Super VL. Same harness, same data, on top of our SOTA recipes. At this point, picking up a new base is basically a config change, all towards the goal of building a learning layer that doesn't care which model sits underneath it.
This time we wanted to show a run taken to 100 steps and trained with the intelligence density work we previewed in the Lightning post. Around step 60 you can watch it kick in quite clearly. The model first learned to solve more tasks, and then it started solving them in half the tokens.
The Headline Results
Post-trained Nemotron 3.5 Super VL passes 86% of rubric criteria on the tasks it completes, which puts it level with Opus 4.7 (86%) and GPT 5.5 (84%). It does that at $1.34 per million tokens. Opus is $15. (The closed-model numbers come from our July evals, which used a different judge, so read this as "same neighborhood," not a strict leaderboard.)
On the strictest measure, all-pass, where a task only counts if every single criterion is met, 3.5 Super goes from 4.2% to 14.2%, a 3.4x increase. Most of the movement is in the "bad" tail: before training, about 16% of tasks scored below 60%. Midway through, truncations spike and this rises to 28%. By step 100, it's down to 10%.
A look at a 100 Step Training Run
The run splits cleanly into two halves.
Steps 0 to 60: get the answer, whatever it costs. All-pass climbs and peaks at 14.2% at step 55. However, in this phase, output length goes the wrong way. Median tokens per task go from 19k to 54k, and 25 of 120 tasks run straight into the 131k output cap and score zero. This is the predictable outcome of training purely on an all pass reward - it gets rewarded for checking its work more than necessary and never gets told to stop.
Steps 60 to 100: same answers, shorter path. Pass rate holds between 9% and 14%. However, median token length now falls from 54k to 28k.
The token distribution is where you can really see it. At step 60 there's a large lump of attempts sitting on the 131k wall. By step 100 that lump is basically gone, and the bulk of the distribution sits about where step 42 was, except the model is now passing more of it.
Rubric coverage tells the same story from the other side. It dips to 67% at the peak of its yapping behavior, then climbs to 86% once those attempts start finishing. Step 100 is the best checkpoint of the run on this metric.
This is what we mean by intelligence density: how much work you get per token, not per parameter. Base 3.5 Super spends about 707k output tokens for every task it fully passes. Step 100 spends about 305k, so 2.3x denser. Our bet is that this second phase is the part that matters for anyone actually paying for inference, and that most RL runs get stopped right before it shows up.
Where it got better
Take the 83 tasks graded at both base and step 100, and compare each one to itself. 71% improve, 13% get worse, and the rest don't move. Averaged by practice area, 20 of 23 areas go up. The biggest well-sampled gains are firm knowledge (+19 points across 20 tasks) and IP (+11 across 11). Most areas only have a handful of test tasks, so we'd read this as "the gains are broad" rather than putting weight on any single area.
What we're working on next: graders without labels
At Trajectory, our goal is to give companies the tools to own what good looks like, and with it their own intelligence. Harvey gave us expert rubrics for every task, but this isn't often the case for most companies. Most teams can tell you exactly which workflows they want an agent to handle, but writing a rubric good enough to train against takes real domain expertise and a lot of skill in writing representative yet non-hackable graders.
So we ran the version of this experiment that looks more like real life. Let's pretend Harvey only gave us rubrics for 100 tasks, and the other 1,790 training tasks are just tasks with environments. Can a model write the graders for the rest?
We gave GPT-6-Astra a sandbox with all 100 tasks, each with a single attempt by the base model, and asked it to build a grader: a program that takes a trajectory and returns a score, and is only allowed to call GPT-6-Luna (so it's cheap enough to run inside RL). We tried three shapes: one that outputs a score directly, one that writes a rubric and grades against it, and one that just picks the better of two attempts. We're tuning the thing that builds graders, not the graders themselves, because the goal is something that works for the next customer too. We call this object a metagrader.
Early read: correlation with the true scores isn't that high (r between 0.25 and 0.61). But that turns out not to matter much. GRPO and GSPO only care which attempt at a task is better than another attempt at the same task, and there the best build of each grader type gets it right 65–72% of the time. Chance is 50%, and Luna grading against Harvey's real rubrics gets 81%.
Next we trained 3.5 Super on the synthetic rewards from the generated rubric grader (Build 1) and checked it against the real rubrics.
The model can hill-climb synthetic rubrics. Its best all-pass accuracy, 15.8% at step 25, is as strong as the 14.2% it reached when trained on expert rubrics. Although the synthetic criteria are phrased differently than the expert criteria, they often capture similar requirements. Explore some of the generated synthetic rubrics for a sample task in the visualizer below.
There’s still a lot of work to do. The Harvey LAB environments are specifically designed to evaluate models on their ability to solve legal tasks. The filesystem provides clues about which criteria are needed to solve the question. Production tasks, environments, and filesystems are much messier. But this is just the beginning - we’re actively working to build metagraders that can extract rubric criteria for noisy production trajectories. This is precisely the setting where the metagrader will be most useful, since expert rubrics will be sparsely, if at all, available.
Why we care
The open model you deploy today is going to be replaced by a better one in three months. What carries over is the loop around it: your tasks, your graders, your traces, and a training stack that can pick up the next base the day it ships. 3.5 Super is the fourth Nemotron model through that loop, and the first one where we watched it get cheaper as well as better in the same run.
Nemotron 3.5 Super VL is available to train and customize on the Trajectory Platform today. And if this is the kind of problem you want to work on, we're hiring :)
Setup
- Base:
nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-BF16 - Algorithm: GSPO, LoRA rank 32, Adam at 1e-4
- Batch: 8 tasks × 8 samples per step, variable token budgets up to 131k
- Eval: the same 120 held-out LAB tasks every 5 steps, one attempt each, 131k output cap
- Judge: GPT-6-Luna against Harvey's rubrics or synthetic rubrics. Token-limit failures score zero; infra failures are excluded.