Self-served open models
Most control over model versions and data paths. You size and operate the GPU instances.
Deployment · Fully cloud
Run the Core, the database and the models in your own cloud account. Serve open-weight models on cloud GPUs, call managed model APIs, or route different agents to each.
What your users see
Your people work with agents in the messaging apps you approve, Microsoft Teams in this example. Where RatioCore runs is your choice. Each channel you enable is an outbound path, so review it like any other connector.
Elena CostaService desk
RatioCore summarise today's open requests
RatioCore Service DeskApp
Here is today's summary.
When it fits
Fully cloud suits teams that run on a hyperscaler already and want managed databases, automated infrastructure and elastic GPU capacity. Agents still act only through RatioCore's identity, policy and approval checks, whichever model answers.
Reference stack
RatioCore keeps the agent layer separate from model serving. The same three tiers apply on every path; only where each one runs changes.
The control plane, agent runtimes and gateways run on a Linux virtual machine in a private subnet, behind your load balancer and TLS.
Each major cloud offers managed PostgreSQL with pgvector. Enable the extension, keep the database private, and use the provider's backups and point-in-time recovery.
Serve open-weight models on GPU instances behind an OpenAI-compatible server, or call managed model APIs that offer OpenAI-compatible chat completions.
Keep credentials in your secret manager and reference them from RatioCore; use cloud logging and monitoring around the platform.
Managed database
| Cloud | Service | pgvector | Notes |
|---|---|---|---|
| AWS | Amazon RDS for PostgreSQL / Aurora PostgreSQL | Available on PostgreSQL 17 | Extension version does not upgrade automatically with the engine |
| Microsoft Azure | Azure Database for PostgreSQL flexible server | Available on PostgreSQL 13 to 18 | Add vector to the allowed extensions first |
| Google Cloud | Cloud SQL for PostgreSQL / AlloyDB | Available | AlloyDB also offers ScaNN indexes |
| Oracle Cloud | OCI Database with PostgreSQL | Available on PostgreSQL 14 to 17 | Enable through a custom configuration |
Cloud GPUs
Examples of GPU instances for open-weight models behind vLLM, SGLang or another OpenAI-compatible server.
| Cloud | NVIDIA examples | AMD examples |
|---|---|---|
| AWS | P5 (H100, 80 GB), P5e/P5en (H200), P6 (B200, B300) | — |
| Microsoft Azure | ND H100 v5, ND H200 v5, ND GB200 v6, ND GB300 v6 | ND MI300X v5 (8x MI300X, 192 GB each) |
| Google Cloud | A3 (H100, H200), A4 (B200), A4X (GB200, GB300), G4 (RTX PRO 6000), G2 (L4) | — |
| Oracle Cloud | H100, H200, B200, B300, GB200, GB300 and L40S shapes | BM.GPU.MI300X.8, BM.GPU.MI355X.8 |
Smaller instances such as single-GPU H100 or L4 suit pilots and embeddings; 8-GPU nodes suit large models and high concurrency. Sources: AWS P5, AWS P6, Azure ND family, Google Cloud GPUs, OCI shapes
Managed model APIs
Managed APIs remove GPU operations. RatioCore still decides what each agent may do; the model only proposes.
| Service | OpenAI-compatible access | Published data commitments |
|---|---|---|
| Azure OpenAI in Microsoft Foundry | v1 API works with the standard OpenAI client | Prompts and outputs not used to train foundation models without permission; processing stays in your geography unless you choose Global or DataZone deployments |
| Amazon Bedrock | OpenAI Chat Completions endpoint per region | Model providers have no access to customer prompts and completions |
| Google Vertex AI | OpenAI-compatible Chat Completions for Gemini and Model Garden vLLM deployments | Customer data not used to train models without permission; regional endpoints keep cached data in the chosen location |
Each managed API adds an outbound data path. Review the provider's terms, regions and retention settings before routing an agent to it. Sources: Azure OpenAI API, Azure data privacy, Bedrock chat completions, Bedrock data protection, Vertex AI OpenAI, Vertex AI data governance
Choosing a model path
Most control over model versions and data paths. You size and operate the GPU instances.
No GPUs to run and fast access to frontier models. Data leaves your network under the provider's terms.
Route sensitive agents to self-served models and others to a managed API. Each agent keeps its own route, budget and approvals.
These are reference patterns, not turnkey or certified cloud packages. Infrastructure templates, quotas and service compatibility are confirmed per project, along with security, backup and restore tests.
Other paths
Serve open-weight models on AMD Instinct GPUs from Azure, Oracle Cloud, DigitalOcean and other providers, with the Core in your cloud account.
Explore AMD cloud Your data centerInstall RatioCore on your own servers with AMD Instinct, NVIDIA, or both, for regulated and network-restricted environments.
Explore on premises Fully cloudRun every tier in your AWS, Azure, Google Cloud or Oracle account, with self-served models, managed model APIs, or both.
Explore fully cloudFAQ
Still deciding? Bring your questions to a walkthrough.
Yes. Any approved model that offers an OpenAI-compatible chat endpoint can be registered as a route, including managed services such as Azure OpenAI, Amazon Bedrock and Vertex AI. Model routes never carry authority to execute actions.
Yes. Each agent maps to its own model route, so a sensitive workflow can use a self-served model while another uses a managed API.
Not by itself. The same identity, policy and approval checks apply. What changes is where data is processed, so review each outbound path, region and retention setting.
See it on your business
Bring a process your team runs today. In a demo we show how agents could run it, where your people stay in charge, and what a focused pilot could prove.