Count concurrent turns
Each agent runtime admits at most two active turns. Plan GPU slots for the agents generating at the same moment, not the agents installed.
Deployment · AMD cloud
Run open-weight models on AMD Instinct GPUs rented by the hour or by contract, and keep RatioCore's governance, audit and approvals in your own cloud account.
What your users see
Your people work with agents in the messaging apps you approve, Microsoft Teams in this example. Where RatioCore runs is your choice. Each channel you enable is an outbound path, so review it like any other connector.
RatioCore PlatformApp
New agent version ready to release.
Release approved by Tomás Ruiz · 12:41
RatioCore PlatformApp
Scheduled for 15:00. I'll post here when it's live.
When it fits
AMD Instinct GPUs carry 192 to 288 GB of memory each, so larger open-weight models fit on fewer GPUs. Cloud capacity lets you start a pilot quickly and scale the inference pool without a procurement cycle.
Reference stack
RatioCore keeps the agent layer separate from model serving. The same three tiers apply on every path; only where each one runs changes.
Control plane, agent runtimes, gateways and dialog and workflow engines on a Linux virtual machine in the same region and private network.
Durable records, memory and audit. Use the provider's managed PostgreSQL where it offers pgvector on version 17, or run it yourself on a private VM.
Model servers expose OpenAI-compatible endpoints. vLLM publishes an official ROCm image, and SGLang and llama.cpp also provide ROCm builds.
Register each endpoint in the model registry and map agents to routes. Give each model family at least two nodes if availability matters; failover works within a family.
Where to find AMD Instinct
Confirmed on provider documentation in November 2026. Availability varies by region and plan; check before you size.
| Provider | GPU | Instance or plan | GPUs per node | Memory per GPU |
|---|---|---|---|---|
| Microsoft Azure | MI300X | ND MI300X v5 (Standard_ND96isr_MI300X_v5) | 8 | 192 GB |
| Oracle Cloud (OCI) | MI300X | BM.GPU.MI300X.8 (bare metal) | 8 | 192 GB |
| Oracle Cloud (OCI) | MI355X | BM.GPU.MI355X.8 (bare metal) | 8 | 288 GB |
| DigitalOcean | MI300X | GPU Droplets, on demand or contract | 1 or 8 | 192 GB |
| DigitalOcean | MI325X | GPU Droplets, contract | 8 | 256 GB |
| DigitalOcean | MI350X | GPU Droplets, contract or spot | 8 | 288 GB |
| AMD Developer Cloud | MI300X | 1 or 8 GPU instances, for development and testing | 1 or 8 | 192 GB |
| Crusoe | MI355X | Crusoe Cloud | — | 288 GB |
| TensorWave | MI300X, MI325X, MI355X | Bare metal, managed Kubernetes or Slurm | — | — |
A dash means the provider page does not state it. DigitalOcean also lists MI355X as a spot-only public preview. Sources: Azure, OCI, DigitalOcean, AMD Developer Cloud, Crusoe, TensorWave
Software on ROCm
RatioCore needs an OpenAI-compatible chat endpoint and an embeddings service. These serving stacks document ROCm support.
| Serving stack | AMD support | Notes |
|---|---|---|
| ROCm | Production release supports MI355X, MI350X, MI325X and MI300X | Ubuntu, RHEL, SLES, Debian, Rocky and Oracle Linux |
| vLLM | Official support for MI300-series and MI350-series | Official image vllm/vllm-openai-rocm; OpenAI-compatible server |
| SGLang | Official ROCm Docker images | MI300X is the documented example |
| llama.cpp | HIP/ROCm build and a server-rocm image | RatioCore's reference chat server; GPU images are built but not CI-tested upstream |
| Ollama | Supports MI300X on Linux with the ROCm v7 driver | Used by RatioCore for embeddings and summaries |
Validate each model on the chosen stack: tool calling, context length and tokenizer behaviour differ between models and servers. Sources: ROCm system requirements, vLLM GPU installation, SGLang, llama.cpp, Ollama
Sizing
Each agent runtime admits at most two active turns. Plan GPU slots for the agents generating at the same moment, not the agents installed.
Model size, context length and the number of parallel sequences decide how much GPU memory each server needs. Large-memory GPUs let one node hold bigger models.
Start with a single-GPU instance, measure turn latency, queue wait and token volume on real work, then size the production pool.
These are reference patterns, not certified packages. Provider availability, quotas and regions change; confirm them, and run your own security, load and restore tests before production.
Other paths
Serve open-weight models on AMD Instinct GPUs from Azure, Oracle Cloud, DigitalOcean and other providers, with the Core in your cloud account.
Explore AMD cloud Your data centerInstall RatioCore on your own servers with AMD Instinct, NVIDIA, or both, for regulated and network-restricted environments.
Explore on premises Fully cloudRun every tier in your AWS, Azure, Google Cloud or Oracle account, with self-served models, managed model APIs, or both.
Explore fully cloudFAQ
Still deciding? Bring your questions to a walkthrough.
No. RatioCore talks to any OpenAI-compatible endpoint. AMD Instinct is one option for serving open-weight models; NVIDIA GPUs and managed model APIs are others.
Yes, and it is the simplest layout. Place the Core host, PostgreSQL and the GPU nodes in one region and private network so inference traffic never crosses the internet.
It is meant for development and testing. Use it to evaluate models on AMD Instinct, then move production to a provider plan that fits your requirements.
See it on your business
Bring a process your team runs today. In a demo we show how agents could run it, where your people stay in charge, and what a focused pilot could prove.