Source: Nvidia — 2026-08-11
Summary
Nvidia released Nemotron 3.5 Lightning, a 30B-parameter mixture-of-experts model that only activates 3B parameters per token, positioned as an efficiency play for long-running agentic workloads on RTX and DGX hardware. Alongside it, Nvidia shipped NeMo Switchyard, an open-source library for smart routing across multiple models inside a single agent tool — deciding per-request which model actually needs to handle a given call.
Key Takeaways
- Nemotron 3.5 Lightning's 30B/3B active-parameter split is a 10x gap between total and active weights — the efficiency story is specifically about keeping long agentic sessions cheap to run, not just chat latency.
- NeMo Switchyard is a separate, open-source routing layer — its job is deciding which model (Lightning or something else in the stack) should actually handle a given tool call or step, rather than sending everything to the biggest available model.
- Targeting both RTX (workstation/consumer) and DGX (datacenter) hardware in the same release signals Nvidia wants this efficiency profile usable from a developer's desk up through production clusters.
- The pairing of a sparse MoE model with a routing library is a deliberate architecture choice: cheap model + smart dispatcher, rather than one model trying to be cheap and capable at every step.
Reel Script
Hook (16s)
Running an AI agent for hours at a time gets expensive fast if every single step calls a full-size model. Nvidia's answer: a model that only wakes up a tenth of itself per request, plus a router that decides when it even needs to.
Core Concept (110s)
Mixture-of-experts is the key idea here, and it's simpler than it sounds — instead of one giant neural network that processes every request with all of its parameters, you build a model out of many smaller "expert" sub-networks, and a gating mechanism picks just a few of them to actually run for any given input. Nemotron 3.5 Lightning has 30 billion parameters total, but only 3 billion of them get activated for any single token — the rest sit dormant, like a company with thirty specialists on staff where any given task only pulls in the three people who actually know that domain, instead of convening the whole company for every question. That matters enormously for agentic workloads specifically, because an agent doesn't make one request — it makes dozens or hundreds of calls across a long session, and if each of those calls only needs to wake up a tenth of the model's weights, your compute cost per session drops accordingly. Nvidia paired this with NeMo Switchyard, a separate open-source routing library whose job is deciding, per request, which model in your stack should even handle that step — not every call needs Lightning's reasoning power; some just need a cheap pass-through.
Hands-On (75s)
The architecture worth sketching is two layers: a request comes into your agent tool, Switchyard's router looks at what kind of step this is and picks the right model for the job, and only then does the request actually hit Nemotron 3.5 Lightning's sparse MoE structure — which itself only activates 3 of its 30 billion parameters to answer it. Two efficiency wins stacked on top of each other: routing decides if you need the big gun at all, and the model itself is a tenth-lit lightbulb even when you do. Shipping this simultaneously for RTX workstation cards and DGX datacenter hardware means the same efficiency profile is supposed to hold whether you're prototyping an agent on your desk or running it at production scale.
Takeaway (24s)
If your agent's compute bill is climbing because every tool call hits your biggest model, the fix Nvidia is betting on isn't a smaller model — it's a smarter dispatcher in front of a sparse one. Worth testing Switchyard's routing logic against your own agent's call patterns before assuming you need to downgrade capability to cut cost.