The AI Inference Stack
As AI moves from model development to widespread production usage, inference is becoming its own infrastructure stack. Performance increasingly depends not only on access to GPUs, but on specialized chips, compilers, runtimes, serving systems, and routing layers that optimize latency, throughput, energy consumption, hardware utilization, and cost per request.
Explore
Categories
- Inference chips & systems
Companies designing specialized hardware and system architectures that make AI inference faster, more power-efficient, and more economical at scale.
- Inference engines, compilers & runtimes
Software that turns trained models into efficient executable workloads by adapting them to the underlying hardware and improving how inference is scheduled and run.
- Model serving & deployment platforms
Platforms that help teams deploy and operate models as reliable production services while handling scaling and infrastructure complexity.
- Inference gateways, routing & control
Infrastructure that connects applications with model providers and provides a shared control layer for production inference.
- On-device & local inference
Infrastructure built to run AI models directly on user-owned or edge hardware rather than relying primarily on remote cloud inference. These platforms make local models easier to deploy and…
Companies
- Baseten
- BentoML
- Blackfuel
- Callosum
- Concentrate AI
- Conifer
- ContextLane
- Daisytuner
- DiscreteStack
- d-Matrix
- Etched
- fal
- Fireworks AI
- Fractile
- FriendliAI
- FuriosaAI
- Gimlet Labs
- Groq
- Helicone
- Inferact
- Infinity
- Kog
- Lamb Labs
- LiteLLM
- LM Studio
- Maxim AI
- Modal
- Modular
- Not Diamond
- OLIX
- Ollama
- OpenRelay
- OpenRouter
- Portkey
- Positron
- Q.ANT
- RadixArk
- Replicate
- Requesty
- Respan
- RunAnywhere
- Runpod
- Simplismart
- Sparxity
- Thesean
- Together AI
- TrueFoundry
- TrustedRouter
- Volantis
- Wafer
- Weflayr
- ZML