RatioCore Book a demo
RatioCore
Book a demo

Deployment · AMD cloud

Agents on AMD Instinct, in the cloud.

Run open-weight models on AMD Instinct GPUs rented by the hour or by contract, and keep RatioCore's governance, audit and approvals in your own cloud account.

YOUR ENVIRONMENT
RatioCore Core services
PostgreSQL 17 + pgvector
AMD Instinct model servers
Audit records
Identity & roles
Outbound: only the channels, connectors and model routes you approve

What your users see

Same experience, wherever it runs.

Your people work with agents in the messaging apps you approve, Microsoft Teams in this example. Where RatioCore runs is your choice. Each channel you enable is an outbound path, so review it like any other connector.

When it fits

Large-memory GPUs, no hardware to buy.

AMD Instinct GPUs carry 192 to 288 GB of memory each, so larger open-weight models fit on fewer GPUs. Cloud capacity lets you start a pilot quickly and scale the inference pool without a procurement cycle.

  • Open-weight models on AMD Instinct GPUs
  • Core and data in your own cloud account
  • Scale the inference pool independently
  • Start with a single-GPU instance for a pilot

Reference stack

Three tiers, one boundary.

RatioCore keeps the agent layer separate from model serving. The same three tiers apply on every path; only where each one runs changes.

  1. 01

    Core services

    Control plane, agent runtimes, gateways and dialog and workflow engines on a Linux virtual machine in the same region and private network.

    • Linux with systemd, Python 3.12
    • Private network to the GPU nodes
    • TLS reverse proxy for the console
  2. 02

    PostgreSQL 17 with pgvector

    Durable records, memory and audit. Use the provider's managed PostgreSQL where it offers pgvector on version 17, or run it yourself on a private VM.

    • WAL archiving and off-host backups
    • Private database network only
  3. 03

    AMD Instinct inference pool

    Model servers expose OpenAI-compatible endpoints. vLLM publishes an official ROCm image, and SGLang and llama.cpp also provide ROCm builds.

    • vLLM: vllm/vllm-openai-rocm image
    • SGLang: lmsysorg/sglang ROCm images
    • Embeddings served alongside, e.g. Ollama on ROCm
  4. 04

    Routes and failover

    Register each endpoint in the model registry and map agents to routes. Give each model family at least two nodes if availability matters; failover works within a family.

    • Model-drift check per server
    • Fair queue shares GPU slots across agents

Where to find AMD Instinct

AMD GPU capacity by provider.

Confirmed on provider documentation in November 2026. Availability varies by region and plan; check before you size.

ProviderGPUInstance or planGPUs per nodeMemory per GPU
Microsoft AzureMI300XND MI300X v5 (Standard_ND96isr_MI300X_v5)8192 GB
Oracle Cloud (OCI)MI300XBM.GPU.MI300X.8 (bare metal)8192 GB
Oracle Cloud (OCI)MI355XBM.GPU.MI355X.8 (bare metal)8288 GB
DigitalOceanMI300XGPU Droplets, on demand or contract1 or 8192 GB
DigitalOceanMI325XGPU Droplets, contract8256 GB
DigitalOceanMI350XGPU Droplets, contract or spot8288 GB
AMD Developer CloudMI300X1 or 8 GPU instances, for development and testing1 or 8192 GB
CrusoeMI355XCrusoe Cloud—288 GB
TensorWaveMI300X, MI325X, MI355XBare metal, managed Kubernetes or Slurm——

A dash means the provider page does not state it. DigitalOcean also lists MI355X as a spot-only public preview. Sources: Azure, OCI, DigitalOcean, AMD Developer Cloud, Crusoe, TensorWave

Software on ROCm

Serving stacks that run on AMD.

RatioCore needs an OpenAI-compatible chat endpoint and an embeddings service. These serving stacks document ROCm support.

Serving stackAMD supportNotes
ROCmProduction release supports MI355X, MI350X, MI325X and MI300XUbuntu, RHEL, SLES, Debian, Rocky and Oracle Linux
vLLMOfficial support for MI300-series and MI350-seriesOfficial image vllm/vllm-openai-rocm; OpenAI-compatible server
SGLangOfficial ROCm Docker imagesMI300X is the documented example
llama.cppHIP/ROCm build and a server-rocm imageRatioCore's reference chat server; GPU images are built but not CI-tested upstream
OllamaSupports MI300X on Linux with the ROCm v7 driverUsed by RatioCore for embeddings and summaries

Validate each model on the chosen stack: tool calling, context length and tokenizer behaviour differ between models and servers. Sources: ROCm system requirements, vLLM GPU installation, SGLang, llama.cpp, Ollama

Sizing

Size for concurrency, not agent count.

01

Count concurrent turns

Each agent runtime admits at most two active turns. Plan GPU slots for the agents generating at the same moment, not the agents installed.

02

Match memory to models

Model size, context length and the number of parallel sequences decide how much GPU memory each server needs. Large-memory GPUs let one node hold bigger models.

03

Pilot small, then measure

Start with a single-GPU instance, measure turn latency, queue wait and token volume on real work, then size the production pool.

These are reference patterns, not certified packages. Provider availability, quotas and regions change; confirm them, and run your own security, load and restore tests before production.

FAQ

Deployment questions.

Still deciding? Bring your questions to a walkthrough.

Does RatioCore require AMD GPUs?

No. RatioCore talks to any OpenAI-compatible endpoint. AMD Instinct is one option for serving open-weight models; NVIDIA GPUs and managed model APIs are others.

Can the Core run on the same cloud as the GPUs?

Yes, and it is the simplest layout. Place the Core host, PostgreSQL and the GPU nodes in one region and private network so inference traffic never crosses the internet.

Can we use the AMD Developer Cloud for production?

It is meant for development and testing. Use it to evaluate models on AMD Instinct, then move production to a provider plan that fits your requirements.

See it on your business

Which function would you reinvent first?

Bring a process your team runs today. In a demo we show how agents could run it, where your people stay in charge, and what a focused pilot could prove.

See it on your workflowBook a demo