RatioCore Book a demo
RatioCore
Book a demo

Deployment · On premises

Your servers. Your GPUs. Your agents.

Install RatioCore inside your own data center on AMD, NVIDIA or a mix of both. Prompts, memory and audit records stay on infrastructure you operate.

YOUR ENVIRONMENT
RatioCore Core services
PostgreSQL 17 + pgvector
AMD or NVIDIA GPU nodes
Audit records
Identity & roles
Outbound: nothing by default; only the channels and connectors you approve

What your users see

Same experience, wherever it runs.

Your people work with agents in the messaging apps you approve, Microsoft Teams in this example. Where RatioCore runs is your choice. Each channel you enable is an outbound path, so review it like any other connector.

When it fits

Regulated data. Restricted networks.

On premises suits organizations that must keep prompts, documents and decisions on infrastructure they control, or that already own GPU capacity. Models run locally, so no prompt leaves the network unless you route an agent to an external provider.

  • Prompts and memory stay on your network
  • Use GPUs you already own
  • AMD and NVIDIA in the same pool
  • Works in network-restricted sites

Reference stack

Three tiers in your racks.

RatioCore keeps the agent layer separate from model serving. The same three tiers apply on every path; only where each one runs changes.

  1. 01

    Application host

    A dedicated Linux server for the control plane, agent runtimes and gateways. Today the Core scales vertically on one host, so give it ample CPU and memory.

    • Linux with systemd, Python 3.12
    • TLS reverse proxy on the internal network
  2. 02

    PostgreSQL 17 with pgvector

    On its own server or the application host for a pilot. For production, add WAL archiving, off-host backups and a rehearsed restore.

    • Private database network
    • Least-privilege database roles
  3. 03

    GPU inference pool

    AMD and NVIDIA nodes can serve side by side. Each model family gets its own route; failover works within a family, so give each at least two nodes when availability matters.

    • OpenAI-compatible servers on every node
    • Embeddings on a small GPU or CPU node
  4. 04

    Operations

    Your monitoring, secrets, logging and backup tools wrap the platform. RatioCore exposes health and readiness endpoints and Prometheus metrics.

    • Self-healing supervision
    • Health, readiness and /metrics

AMD hardware

AMD Instinct and Radeon PRO.

Data-center platforms for production pools, and workstation-class cards for small sites and pilots.

System or cardGPUsMemory per GPUBest for
Dell PowerEdge XE97858x Instinct MI355X, with 2x EPYC 9005 CPUs; air-cooled288 GBLarge production pools
Supermicro AS-4126GS-NMR-LCC8x Instinct MI355X; liquid-cooled, 4U288 GBDense production pools
Supermicro AS-8126GS-TNMR8x Instinct MI350X; air-cooled, 8U288 GBProduction pools
Instinct MI325X / MI300X servers8-GPU platforms from server vendors256 GB / 192 GBExisting Instinct fleets
Radeon PRO W7900Workstation or server card48 GBPilots, small sites, embeddings
Radeon AI PRO R9700Workstation or server card32 GBPilots, small models, embeddings

All listed AMD GPUs appear in ROCm's production support matrix. Radeon cards are supported on Ubuntu and RHEL. Sources: Dell XE9785, Supermicro AMD, ROCm GPU specs, ROCm system requirements

NVIDIA hardware

From one card to a rack.

NVIDIA systems for every scale, served through open servers such as vLLM and SGLang or through NVIDIA's own serving software.

System or cardGPUsMemoryBest for
NVIDIA GB200 NVL7272 Blackwell GPUs and 36 Grace CPUs; liquid-cooled13.4 TB per rackVery large model fleets
DGX B300 / HGX B3008 Blackwell Ultra GPUs2.1 TB per systemLarge production pools
DGX B200 / HGX B2008 Blackwell GPUs1.4 TB per systemLarge production pools
H200 (SXM or NVL)Hopper GPU141 GBProduction pools, existing Hopper fleets
RTX PRO 6000 Blackwell Server EditionPCIe server GPU, up to 4 MIG instances96 GBMid-size pools, mixed workloads
L40SPCIe server GPU48 GBPilots, smaller models, embeddings

NVIDIA NIM serves models behind industry-standard APIs and needs an NVIDIA AI Enterprise license in production. NVIDIA Dynamo is open source and offers an OpenAI-compatible front end over vLLM, SGLang and TensorRT-LLM. Sources: GB200 NVL72, DGX B300, HGX, H200, RTX PRO 6000, L40S, NIM, Dynamo

Sizing

Plan the pool around real work.

01

Start from the workload

Users, peak requests, simultaneous turns, model family, context length and tools per turn drive the GPU count. A developer's setup is not a sizing guide.

02

Avoid single-slot bottlenecks

A single-slot node shared by many agents is the most common bottleneck. Prefer multi-slot servers or more nodes per model family.

03

Mix vendors by role

Large-memory nodes can serve the main reasoning model while smaller cards handle embeddings, summaries and pilot agents.

Installation is a guided procedure today. Hardware lists are examples confirmed on vendor pages in November 2026, not a certified compatibility matrix; validate drivers, models and serving software in your environment.

FAQ

Deployment questions.

Still deciding? Bring your questions to a walkthrough.

Can we mix AMD and NVIDIA GPUs?

Yes. RatioCore routes each agent to a model endpoint, not to a GPU vendor. AMD and NVIDIA servers can sit in the same pool, each serving its own model family.

Does on premises mean no internet access at all?

It can. With local models and no external channels or connectors enabled, nothing needs to leave your network. Each channel, connector or external model you enable adds its own outbound path.

Can we start with one server?

For a pilot, yes: one application host with PostgreSQL in a container and one or two GPU nodes. Production separates the database, adds backups and sizes the GPU pool to peak concurrency.

See it on your business

Which function would you reinvent first?

Bring a process your team runs today. In a demo we show how agents could run it, where your people stay in charge, and what a focused pilot could prove.

See it on your workflowBook a demo