In one sentence. ComputeFlux translates between the mutually incompatible formats OpenAI, Anthropic, and Google use, so existing code can reach any of them by changing one URL.
Picture it like this. Simultaneous interpretation at the UN. With three languages you could hire an interpreter for every pair — six of them. At nine languages you'd need seventy-two. So instead everything routes through one pivot language, and a tenth language costs two interpreters rather than eighteen. The catch is the same one the UN lives with: an idiom that exists in one language but not the pivot doesn't survive the trip.
Why it matters. Switching AI providers is supposed to be a business decision. In practice it's a rewrite. This layer turns it back into a configuration change — and it's also where a surprising amount of quiet data loss can hide, which is why this article names what gets dropped instead of glossing over it.
Here for the architecture? Skip to the hub-and-spoke matrix.
When ComputeFlux's Gateway takes an OpenAI-formatted request bound for Claude, or routes an Anthropic SDK call to Gemini, it does something most AI proxies avoid: real protocol translation. Not header rewriting, not URL remapping. Semantic transformation across wire formats that genuinely disagree with each other — ten client-facing endpoints across two inbound protocol families, plus a third provider protocol, Gemini, supported outbound only.
Three things make that tractable: a ProviderClient adaptor interface, an intermediate canonical representation, and a deliberately minimal conversion matrix. Article 9 showed how a request survives a failing provider. This article picks up the same retry loop and asks what happens when the fallback provider doesn't even speak the same protocol as the one that just failed.
Why the Adaptor Pattern, Not Middleware Transformation
A common naive approach to multi-provider support is middleware chaining: each provider-specific transformation is a middleware layer, and a request passes through the stack until it matches. This fails in three ways for ComputeFlux's use case.
First, middleware chains introduce ordering dependencies. If Anthropic→OpenAI conversion runs before OpenAI→Gemini, the intermediate representation must survive the first transformation intact — which it often does not, because OpenAI's function tool-calling syntax is structurally different from Anthropic's tool_use blocks. Middleware doesn't compose cleanly; each layer must know what came before.
Second, middleware implies the request body is the integration point, but AI protocols diverge at every layer: authentication (Bearer tokens vs. x-api-key vs. x-goog-api-key), URL structure, error envelope format, and — most critically — streaming semantics. Middleware that only transforms the body leaves the surrounding HTTP machinery to ad-hoc per-provider code, defeating the abstraction.
Third, and most practically, the Gateway must support symmetric conversion: a user sending an Anthropic-format request should receive an Anthropic-format response, regardless of which provider actually served it. Middleware stacks struggle with this bidirectional state — request and response paths must be independently reversible, which requires pairing request converters with their inverse response converters.
The Adaptor pattern solves all three by putting every provider behind a uniform interface. The Scheduler calls just two methods — Do for synchronous requests, Stream for streaming — plus a small GetProviderName helper. There's no separate init or URL-building step in the interface, because each Adaptor builds its own upstream URL and headers inside Do and Stream.
So the Scheduler picks an endpoint without knowing its protocol, and the Adaptor absorbs every protocol-specific detail. Conversion happens at the Gateway layer before the Adaptor runs, which means an Adaptor only ever sees requests already in its native format.
The Endpoint Surface: Ten Endpoints, Two Inbound Protocol Families
Understanding why each endpoint exists — and why some deliberately don't — illuminates the Gateway's design philosophy.
OpenAI-compatible (/v1/*), the de facto industry standard, covers the core inference surface in seven endpoints. GET /v1/models is the discovery endpoint — unauthenticated, cached for 7 days (Cache-Control: max-age=604800), returning every model available through the Gateway. It's the only unauthenticated endpoint on this surface. POST /v1/chat/completions is the flagship endpoint, discussed in depth below. POST /v1/completions is the legacy text-completions endpoint, translated internally into chat-completions format by wrapping the prompt in a single user message — lossless for simple cases, but it loses completions-specific features like suffix prompting and logprobs customization. POST /v1/embeddings is stateless — one input text, one output vector — and, like the other non-chat endpoints (images, audio), has no Anthropic equivalent to convert to, since Anthropic has no embeddings API. POST /v1/images/generations returns a JSON wrapper around a URL or base64-encoded image, and POST /v1/audio/speech returns binary audio; neither is streaming JSON, both carry correspondingly higher latency, and the streaming path is disabled for them. POST /v1/audio/transcriptions is the reverse of TTS and requires multipart form-data handling, a different content-type path from the JSON endpoints.
Anthropic-compatible (/anthropic/*) exposes three endpoints. POST /anthropic/v1/messages is the primary one, and it diverges from OpenAI in exactly the ways that hurt: a top-level system field, plus a messages array whose content may be either a string or an array of typed content blocks. That divergence is the single largest source of complexity in the translation layer, detailed below. POST /anthropic/v1/messages/count_tokens returns token counts without running inference. POST /anthropic/v1/complete is Anthropic's legacy completions endpoint, deliberately not convertible to OpenAI format (errUnsupportedConversion), which nudges clients toward the Messages API.
Gemini is not a third client-facing family. There is no inbound /gemini/* or /v1beta/* surface for clients to call. It exists only as a third provider-side protocol. When a route points at a Gemini-hosted model, the Gemini Adaptor reshapes the already-converted request into Gemini's own wire format — /v1beta/models/{model}:generateContent, authenticating with the API key through the x-goog-api-key header — forwards it, and translates the response back on the way out.
This is the most involved compatibility layer of the three, because Gemini diverges most structurally from both inbound formats. System prompts live in a top-level systemInstruction field rather than in the messages array. A safetySettings array controls per-category content-filtering thresholds, which the Gateway fills with sensible defaults when the source protocol has no equivalent. And streaming uses yet another SSE variant that needs its own parser.
The Hub-and-Spoke Matrix: Converters, Not a Combinatorial Explosion
Two inbound protocols (OpenAI, Anthropic) times three provider protocols (OpenAI, Anthropic, Gemini) yields six conversion pairs, each needing request and response conversion — twelve paths. Same-protocol routing (OpenAI-in→OpenAI-provider, Anthropic-in→Anthropic-provider) is pass-through with zero conversion cost, reducing to eight active non-streaming paths; add streaming variants for the four cross-protocol response paths and the total lands at twelve.
ComputeFlux does not implement sixteen independent converter functions. Instead, every conversion routes through a canonical intermediate representation — the Bifrost chat request/response model — which is the critical architectural insight: hub-and-spoke conversion, not point-to-point. When an Anthropic request targets a Gemini provider, rather than a bespoke AnthropicToGemini converter handling every field permutation, the code composes two existing converters:
Anthropic → BifrostChatRequest → OpenAI → BifrostChatRequest → Gemini
ConvertAnthropicRequestToGemini is a three-line function: call ConvertAnthropicRequestToOpenAI to produce an OpenAI-format byte slice, then feed that into ConvertOpenAIRequestToGemini. The intermediate OpenAI format never touches the wire — it's purely an in-memory pivot point.
This yields a profound reduction in surface area. Each converter only needs to know two formats: its source and the canonical Bifrost model. Adding a fourth protocol (say, Cohere) requires only two new converters (Cohere↔Bifrost), not converters to every existing protocol — the matrix grows O(n) in protocols, not O(n²) in pairwise combinations.
The trade-off is fidelity. Every hop through the canonical model risks information loss: Anthropic's stop_reason has no exact OpenAI equivalent, OpenAI's logprobs has no Anthropic counterpart. The Bifrost model necessarily represents the intersection of all protocols' feature sets, not their union, and fields it doesn't support are silently dropped. For plain text-in-text-out chat completions this is invisible; for edge cases like Anthropic's computer-use tool or OpenAI's structured outputs, the Gateway must either extend the canonical model or reject the conversion. ComputeFlux chose rejection over silent data corruption — trading compatibility for correctness.
Field-Level Divergence: Content Blocks, Tool Calls, System Prompts
The deepest architectural difference between OpenAI and Anthropic is in the message content model. OpenAI models content as a string or an array of {type, text/image_url} objects. Anthropic models content as an array of typed content blocks — text, image, tool_use, tool_result, and potentially future block types. This has concrete implications for the translation layer:
Forward conversion (OpenAI → Anthropic): an OpenAI content string wraps into a single {"type": "text", "text": "..."} block; an OpenAI content array maps block-by-block (text → text, image_url → image). OpenAI's tool_calls field on the assistant message converts into Anthropic's tool_use content blocks embedded within the message content array.
Reverse conversion (Anthropic → OpenAI): multiple text blocks concatenate (or stay as an array, depending on client expectations). tool_use blocks become tool_calls on the assistant message — OpenAI keeps tool calls separate from content, Anthropic embeds them in the content array. This structural mismatch, a reorganization of the message tree rather than a field rename, is the most common source of bugs in protocol translation.
Tool results are particularly tricky: Anthropic represents them as user messages with tool_result content blocks matching the originating tool_use ID; OpenAI represents them as messages with role: "tool" and a tool_call_id. The Gateway must track tool call IDs across the whole conversation and consistently map them between the two ID namespaces.
Gemini's system prompt handling is a cautionary tale about API divergence all on its own. Three providers, three different mental models. OpenAI makes the system prompt a role: "system" message inside the array. Anthropic makes it a top-level system field covering the whole conversation. Gemini makes it a systemInstruction field sitting explicitly outside the contents array, applied at the generation level rather than the message level.
The problem surfaces when a client alternates system and user messages mid-conversation. System: you are helpful. User: hello. System: now you are unhelpful. User: hello again. The Gateway has to pick: merge every system message into one systemInstruction, keep only the last, or reject the request outright.
The current implementation keeps the last one. That's pragmatic and correct for the overwhelming majority of use cases, and it silently discards information for the minority that genuinely rely on mid-conversation system-prompt changes. It's the semantic-loss problem in miniature. Syntax translates cleanly — messages become contents. Semantics, like conversation-level versus message-level instruction, doesn't always survive the hop.
/v1/chat/completions earns its status as the most complex endpoint precisely because it's where all of this collides at once: streaming vs. non-streaming response pipelines, tool/function calling with three incompatible schemas, multi-modal content with three incompatible image encodings (image_url vs. Anthropic's image block vs. Gemini's inlineData), and per-provider optional response fields (system_fingerprint, logprobs, finish_reason vs. stop_reason) that must be preserved, stripped, or documented as provider-dependent on a per-field basis.
Streaming: A Brief Mention
Streaming conversion is where this architecture spends its complexity budget. The converter can't buffer a full response before forwarding — that would defeat the point of streaming — so it has to carry incremental state across chunk boundaries. It tracks whether a content block has started, accumulates tool-call arguments fragment by fragment, and holds final usage statistics until the stream ends.
Article 14 is the canonical deep-dive: chunk reassembly, backpressure handling, and the tool-call state machine across three different SSE dialects.
Gateway-Side vs. Scheduler-Side Conversion
A fundamental architectural question is where conversion happens.
Gateway-side conversion puts it in the HTTP handler in relay_handler.go. That handler reads the inbound request's Content-Type or URL path to identify the protocol, compares it against the selected endpoint's protocol, and calls convertRequest/convertResponse around the forward. The Scheduler and Adaptor then work purely in the provider's native format.
Scheduler-side conversion would instead give each route a "preferred protocol" field and let the Scheduler convert before dispatch. That would let it treat endpoints speaking different protocols as interchangeable fallbacks.
ComputeFlux chose gateway-side conversion, and the reasoning runs straight back to Article 9's retry loop. Because conversion happens at the edge, the Scheduler's retries operate on provider-native requests. When a request fails on OpenAI and falls back to Anthropic, the Gateway re-converts the original user request rather than the already-converted one.
That's what stops conversion errors from compounding. If the OpenAI→Anthropic conversion drops a field, the loss doesn't carry forward into an Anthropic→Gemini conversion on the next retry. The cost is keeping the original raw bytes around for the request's lifetime — negligible for a typical sub-1MB payload.
Error handling differs too. Suppose a request uses an Anthropic feature Gemini doesn't support. Gateway-side conversion catches it and returns a 400-level client error before any provider is contacted, which is immediate and actionable. Scheduler-side conversion would turn the same case into a runtime routing failure, potentially burning retries against providers that would all fail identically.
HTTP Client Architecture and Timeout Design
ComputeFlux doesn't share one HTTP client across every outbound provider call. It runs two purpose-built http.Clients. They share transport settings — IdleConnTimeout: 90s, MaxIdleConns: 100, HTTP/2 forced off — and diverge on exactly the timeouts each traffic shape needs.
The non-streaming client behind Do sets an overall Timeout: 300s and disables ResponseHeaderTimeout completely. A non-streaming LLM response can legitimately take minutes to generate before a single byte comes back, and a header-arrival timeout would fire on perfectly healthy requests.
The streaming client behind Stream inverts that. No overall Timeout at all — the caller's context governs a stream's lifetime — but it keeps ResponseHeaderTimeout: 90s. A streaming response should start emitting within seconds, so 90 seconds is a generous bound for catching a connection where the TCP handshake succeeded but the upstream never sent a first byte.
The shared settings matter too. IdleConnTimeout: 90s trades connection-pool memory against the latency of establishing new TLS connections, and since Gateway→Provider connections are long-lived and reused across many user requests, pooling earns its keep. MaxIdleConns: 100 caps the pool globally rather than per host — in practice the binding constraint is the provider's own rate limit, not the Gateway's pool.
Attack Surface and Security Considerations
The conversion layer introduces a specific attack surface: prompt-injection-via-protocol-confusion, where an attacker crafts a request valid in both OpenAI and Anthropic formats but carrying semantically different meaning depending on which converter processes it first. ComputeFlux mitigates this with strict protocol detection — the inbound protocol is determined by URL path prefix (/v1/chat/completions vs. /v1/messages) and the anthropic-version header, never by heuristic content inspection. There is no "auto-detect" mode; the request is unambiguously classified before conversion begins, eliminating protocol confusion as an attack vector entirely. A secondary concern is information leakage through error messages: if a conversion error echoed raw field values, a malicious user could probe for the existence of specific fields in the intermediate representation and reverse-engineer the conversion logic. Conversion failures are wrapped generically — the detailed error is logged internally, but only a sanitized version reaches the client.
Where Management Traffic Goes
Everything above concerns the inference hot path. Account management, provider discovery, and billing history don't go through this REST surface at all — they go through a dedicated GraphQL endpoint (/gql), chosen precisely because management queries need field-level flexibility that a REST endpoint-per-view-model can't offer cheaply. We cover that design, including why the schema is split by domain into several independently-generated files, in Article 15, GraphQL API Layer.
Key Takeaways
- Everything pivots through one canonical format. Adding a new provider costs two converters, not one for every provider already supported. The work grows in a straight line instead of exploding.
- The pivot's price is fidelity, and it's paid in features. The middle format can only carry what all the formats share. Anything unique to one provider has nowhere to go — and ComputeFlux chooses to reject such a request rather than silently mangle it.
- One documented casualty: changing the system prompt mid-conversation. Only the last one survives the translation. That's named here rather than left for someone to discover in production.
- Translation happens at the edge, not in the scheduler — for a specific reason. When a request fails on one provider and falls back to another, the original request gets re-translated. Convert the already-converted version and errors compound with every fallback.
- The protocol is read from the URL, never guessed from the content. No auto-detection means a whole class of protocol-confusion attack simply has nowhere to live.
The Adaptor pattern and hub-and-spoke matrix are a pragmatic trade between generality and complexity. Protocol conversion is a data transformation problem, not a routing problem. Isolating it at the Gateway layer is what buys clean retry semantics, unambiguous protocol detection, and centralized error handling.
None of this, though, addresses how fast anything moves once it's inside the chain rather than at the HTTP edge. The next article drops altitude sharply — from what a request means to what it costs in nanoseconds.
Next — Article 13: vtproto — High-Performance Serialization: the least glamorous article in the series, and one of the most honest about what it can and can't prove.