Benchmark data from real deployments — then test a live model yourself. The chat below runs OpenAI's open-weight gpt-oss-120b via Groq. It's the same model we can deploy on your own hardware, so you know what you're getting.
Typical generation throughput with llama.cpp / Ollama at Q4, from published 2026 community benchmarks. Real speed depends on context length, backend and drivers, so we measure it on your hardware during the audit.
* Hosted API speed depends on the model, provider load, region and tier, and every request adds a network round-trip. Local TTFT for a model held in VRAM is typically under 100ms. Local figures are community-reported; your results will vary.
Throughput (tok/s) is only half the story. Time To First Token (TTFT) determines how responsive your application feels — and this is where local wins most dramatically.
| TTFT (avg) | THROUGHPUT | RATE LIMITS | CONSISTENCY | DATA LEAVES NETWORK | |
|---|---|---|---|---|---|
| Local · RTX 4090 · Qwen3 27B | <100ms | ~72 tok/s | NONE | DETERMINISTIC | NEVER |
| Local · 2× RTX 3090 · 70B | <200ms | ~18 tok/s | NONE | DETERMINISTIC | NEVER |
| OpenAI API | + NETWORK | VARIES | TPM / RPM CAPS | VARIABLE | ALWAYS |
| Anthropic API | + NETWORK | VARIES | TPM / RPM CAPS | VARIABLE | ALWAYS |
This is gpt-oss-120b, OpenAI's open-weight model: 117B parameters, but only about 5B active per token, so it runs on a single 80GB GPU or a 128GB Mac. Here it runs on Groq so you can try it without your own hardware. Ask it anything about local AI deployment.
You've seen the numbers and tested the model. We'll get gpt-oss, Qwen3, Gemma 4 — whatever fits your use case — running on your own infrastructure. No more API bills.