Nemotron 3.5 Bolt - Run everyday agentic tasks locally on a 24GB GPU

Big AI models are impressive, but running them locally without giving up too much speed or capability is still a challenge. NVIDIA is taking a shot at that with Nemotron 3.5 Bolt 74B-A9B 24GB.

The problem

Running capable AI agents locally usually means making trade-offs between model intelligence, speed, and memory. Larger models can be difficult to fit alongside your other applications, while smaller models may struggle with complex tasks.

The solution

Nemotron 3.5 Bolt is an open-weight vision-language model designed to fit within a 24GB memory budget and run everyday agentic workloads locally on NVIDIA RTX and DGX Spark systems.

What makes it different

Instead of training a small model from scratch, NVIDIA compresses a much larger model and uses aggressive 2-bit quantization to preserve more of its capabilities within a tight memory budget. It also uses speculative decoding to speed up generation.

What you get:

• A 74B-parameter model with 9B active parameters

• A 24GB memory footprint for local AI workloads

• Around 84 tokens per second on NVIDIA's reported RTX Spark benchmark

• Designed for agentic tasks such as web browsing, tool use, and terminal workflows

• Open-weight checkpoints, including a version for further customization

• NVIDIA reports #1 in PinchBench and #2 in WebArena and BrowseComp among models with similar memory footprints

Who it's for

Developers building local AI agents, experimenting with private workflows, automating research, or running coding and browsing agents without sending every task to the cloud.

Try it:

Explore the official NVIDIA announcement, get the model on Hugging Face, or check the technical report.