Open Source

Hermes 3 70B Is the Best Open-Weight Model for Agent Tasks Right Now

mrmolsen · June 7, 2026 ·6 min read

If you’re building agentic workflows on open-weight models, Hermes 3 70B is where the benchmarks and the real-world results align. NousResearch has spent two years training models specifically for agentic use cases, and the 70B version of Hermes 3 shows it.

What Hermes 3 Is

Hermes 3 is a fine-tuned series from NousResearch built on top of Meta’s Llama 3.1 base models. The key differentiation isn’t raw benchmark performance — it’s the training emphasis on:

Tool use reliability: structured JSON output for function calling with low hallucination rate
Instruction following: following multi-step, conditional instructions without drift
Role consistency: maintaining assigned personas and task focus across long conversations
Context utilization: actually using information from the full context window

These are precisely the properties that matter for agent tasks and don’t show up clearly in standard academic benchmarks.

Where It Wins

Tool use: Hermes 3 70B produces consistent, valid JSON for function calling with noticeably fewer malformed outputs than base Llama 3.1 70B in the same tasks. In multi-tool schemas the gap is meaningful — invalid calls require retry logic that burns tokens and slows pipelines.

Long instruction chains: Where base Llama 3.1 tends to drop conditions by the fourth or fifth step of a complex instruction, Hermes 3 follows through. NousResearch attributes this to deliberate instruction-following training rather than raw benchmark optimization.

Roleplay consistency: For agent personas — a specialized analyst, a strict code reviewer, a cautious planner — Hermes 3 maintains the assigned role across long contexts without drifting back to generic assistant behavior. This matters for multi-agent systems where role discipline is load-bearing.

Running It

Hermes 3 70B is available on Hugging Face in GGUF format for local inference via llama.cpp or Ollama:

ollama pull nous-hermes3:70b

At Q4_K_M quantization, it runs on a 48GB GPU (A6000, RTX 6000 Ada) or dual 24GB consumer GPUs. The Q6_K version (better quality) needs ~60GB VRAM.

For API access without running your own hardware: Fireworks AI and Together AI both host Hermes 3 70B with OpenAI-compatible endpoints.

The Tradeoffs

Hermes 3 70B is not Claude Sonnet. At raw reasoning and coding tasks, Sonnet wins. The case for Hermes 3 is cost and privacy: $0 at inference if you run it locally, no data leaving your infrastructure, and performance close enough to frontier models for most agentic use cases.

For workflows that process sensitive data or run at scale where API costs matter, Hermes 3 70B is the open-weight default worth reaching for first.