What it is

An inference host running open-weight models on custom LPU hardware, not a model lab. It serves Llama, Qwen, GPT-OSS and Whisper at several hundred tokens per second — often an order of magnitude faster than GPU-based providers — behind an OpenAI-compatible endpoint.

Why it earns a slot

When latency is the product — voice agents, autocomplete, live chat — Groq makes an open model feel instant. The free tier needs no credit card, so it is the fastest path to a working LLM call.

Know before you commit

No GPT, Claude or Gemini — open weights only, so quality tops out below frontier. Free-tier limits apply per organization, and extra API keys will not raise them.

Field notes

0 from the community

Sign in to add a field note. Reading is always open.

Loading notes…

Field check

Unverified

Sign in to confirm this entry is still accurate.

Pricing snapshot

Per token: Llama 3.1 8B $0.05/$0.08 per M, GPT-OSS 120B $0.15/$0.60, Llama 3.3 70B $0.59/$0.79. Free tier ~30 RPM / 6k TPM / 14.4k RPD. Dev tier adds 10x limits and 25% off.

Snapshots move fast — verify at checkout, and post a field note if it changed.

Drop into your Lovable chat

Evaluate Groq (https://groq.com) for this project, then use it only if it fits what we are already building.
Context: Ultra-fast open-model inference. An inference host running open-weight models on custom LPU hardware, not a model lab. It serves Llama, Qwen, GPT-OSS and Whisper at several hundred tokens per second — often an order of magnitude faster than GPU-based providers — behind an OpenAI-compatible endpoint.
Reviewer notes (general, not written for this codebase): When latency is the product — voice agents, autocomplete, live chat — Groq makes an open model feel instant. The free tier needs no credit card, so it is the fastest path to a working LLM call.
Watch out for: No GPT, Claude or Gemini — open weights only, so quality tops out below frontier. Free-tier limits apply per organization, and extra API keys will not raise them.
Pricing: Per token: Llama 3.1 8B $0.05/$0.08 per M, GPT-OSS 120B $0.15/$0.60, Llama 3.3 70B $0.59/$0.79. Free tier ~30 RPM / 6k TPM / 14.4k RPD. Dev tier adds 10x limits and 25% off.
Rules: treat the notes above as background only. Do not assume a theme, style, layout or feature that is not already in this project — inspect the existing code first and ask me before changing visual direction.

/api/public/tools/groq

Also in Model APIs, routers & observability

Anthropic API / Claude Platform

platform.claude.com

Frontier model provider API

PaidModel APIs, routers & observability

Per token. Opus 5 $5/$25 per M in/out; Sonnet 5 $2/$10; Haiku 4.5 $1/$5; Fable 5 and Mythos 5 $10/$50. Batch API 50% off; cache reads 0.1x input.

Open dossier

Braintrust

braintrust.dev

LLM eval and experiment platform

Free tier + paidModel APIs, routers & observability

Starter free ($10/mo model credits, 1GB data, 10k scores, 14-day retention), Pro $249/mo ($249 credits, 5GB, 50k scores), Enterprise custom. Overage $3-4/GB and $1.50-$2.50 per 1k scores.

Open dossier

Fal.ai

fal.ai

Generative media inference host

PaidModel APIs, routers & observability

Per output or per second for media models: images ~$0.02-$0.04 at 1MP, video $0.05-$0.40 per second. Serverless GPU rates $1.10-$8.50/hr with committed-use discounts.

Open dossier

Fireworks AI

fireworks.ai

Open-model inference host

Free tier + paidModel APIs, routers & observability

Per token serverless across Standard/Priority/Fast tiers; embeddings $0.008-$0.10/M. On-demand GPUs billed per second: H100/H200 ~$7-8/hr, B200 ~$10-13/hr. Fine-tuning $0.50-$40 per M training tokens. $1 free credit.

Open dossier