Early access · engines live in minutes
Your own AI engine.
Plugged in.
Open models on dedicated GPUs, behind one OpenAI-compatible endpoint. Flat monthly price, no token meter.
POST https://your-engine.node.inferenceplug.com/v1/chat/completions
user › Summarize our refund policy in one sentence.
assistant ›
Built on the open models teams already trust — and the ones they'll trust next.
0%
dedicated hardware
Every engine is its own GPU instance. No shared compute, no noisy neighbours, no co-mingled data.
100
per-token charges
One flat monthly price. Run it flat out at 3 a.m. and the invoice doesn't move.
0
models curated to start
Plus any other text-generation model on Hugging Face, private or gated, using your own token. Version-pinned and matched to hardware that fits.
Five choices. Then an engine.
No cloud console, no CUDA, no YAML. Hover to pause; click any step.
Pick a cloud
Pick a region
Pick a model
Pick a GPU
Plug in
Pick a cloud. DigitalOcean today; Vultr, Hetzner and AWS next. Our account, our problem.
One engine. Four layers. Open at every layer.
From the GPU to the agent runtime, every layer is built for production and runs on infrastructure dedicated to you. Most platforms cover one layer, or scatter them across dozens of disconnected services.
Managed Compute
AvailableA dedicated GPU running vLLM behind an OpenAI-compatible endpoint.
One customer per instance — never shared. CUDA, drivers, TLS, monitoring and updates are handled. You get a URL and a key.
Products
- GPU Nodes
- Models catalog
- Automatic TLS
- Health & self-heal agent
Open source integrations
vLLM, Caddy, NVIDIA DCGM, Prometheus
NewInference Router
Many models. One endpoint. Your rules.
Running a 70B for hard questions, an 8B for cheap chat and a fine-tune of your own means three base URLs and a routing problem in your application code. Put a router in front and it becomes one base URL and a list of rules you can read.
A request arrives
Router decides
prod-router · least_queue- model
- "fast"
- path
- /v1/chat/completions
- prompt_tokens
- 42
- header
- —
No rule matched. A fresh router already works: the model field is resolved against your aliases.
No rule matched, so the router resolves model "fast" — two replicas serve it. Least-queue takes the idle one.
Handled by fast on L40S 48GB.
Qwen2.5 7B Instruct
L40S 48GB
Qwen2.5 7B Instruct
L40S 48GB
Llama 3.3 70B Instruct
2× H100 160GB
Your model names, not ours
Give each node an alias — “fast”, “big”, “drafting” — and your application asks for that. Swap the model behind an alias without touching a line of code.
Rules you can read top to bottom
Ordered, first match wins. Route on the requested model, the path, a header, an API key, prompt length or whether the call streams. No scoring to reverse-engineer.
Load-aware, not round-robin-and-hope
When several nodes serve one alias, the router places each request by live vLLM queue depth or free KV cache, and can pin a conversation to keep its prefix cache warm.
It keeps serving without us
The router holds its own routing table and its own key hashes, and probes your nodes itself. If our control plane goes dark your traffic is unaffected — only changes to configuration pause.
# one base URL, your own model names
BASE = "https://prod-a1b2c3d4.router.inferenceplug.com/v1"
client = OpenAI(base_url=BASE, api_key="ipr_live_…")
client.chat.completions.create(model="fast", messages=[…])
client.chat.completions.create(model="big", messages=[…])A router costs nothing extra when it rides on a node you already run. Give it its own instance when you want it to outlive any single node.
Why an engine beats an API
It's yours, not a shared API
No rate limits, no queueing behind other tenants, no deprecations on someone else's schedule. Pin the model version you tested and ship it.
Production serving out of the box
vLLM with continuous batching and paged attention on H100 and L40S. Sub-second time-to-first-token on 8B models; 70B on dual H100. Health checks, auto-restart and Prometheus metrics on every engine.
One line to switch
Every engine speaks the OpenAI API. Change the base URL in the SDK you already use and keep everything else. No CUDA, no Kubernetes, no YAML.
A price you can put in a budget
Hardware is bought at reseller rates and bundled with the platform into one number. No meter, no egress surprises, no minimum commitment. Cancel anytime.
Priced like hosting. Not like tokens.
Per engine, per month. Hardware, TLS, monitoring and updates included. Any model on Hugging Face.
An Inference Router in front of them is free on a node you already run, or the price of a small CPU instance if you give it one of its own.
L40S 48GB
1× NVIDIA L40S
$1,290/mo
- 48 GB VRAM · 8 vCPU · 64 GB RAM
- Models up to 12B parameters
- Unmetered requests and tokens
- Dedicated instance, automatic TLS
H100 80GB
Most popular1× NVIDIA H100
$2,690/mo
- 80 GB VRAM · 20 vCPU · 240 GB RAM
- Models up to 32B parameters
- Unmetered requests and tokens
- Dedicated instance, automatic TLS
2× H100 160GB
2× NVIDIA H100
$5,290/mo
- 160 GB VRAM · 40 vCPU · 480 GB RAM
- Models up to 70B parameters
- Unmetered requests and tokens
- Dedicated instance, automatic TLS
Plug in your first engine
From a single 8B assistant to a 70B production agent — pick a cloud, pick a model, and you're live in minutes.