# An API in front of a chat backend

**Published:** September 15, 2026 | **Author:** Noppakorn Kaewsalabnil

---

TL;DR: An OpenAI-compatible endpoint is not one translation. The backend behind mine takes a single flattened string and answers with text deltas, so structured tool calls, token counts, credentials, and error semantics all had to be built in the adapter. This post covers the four gaps, the streaming parser that cannot take text back, and the sealed-cookie token that keeps the gateway stateless.

Every AI SDK ships with a list of things it assumes the server does. It can be handed tool definitions and will answer with structured calls. It reports `prompt_tokens` on the response. It authenticates with a key you can revoke without touching anything else you own. None of that is in the protocol because a protocol says so. Each one is a promise some backend made first, and the SDK wrote it down.

I spent the past two weeks building a gateway that speaks three of those protocols in front of a backend that makes none of those promises. [thaipass](https://github.com/PunGrumpy/thaipass) serves `/v1/chat/completions`, `/v1/responses`, and `/v1/messages` over the chat backend that sits behind a web UI. That backend accepts a message with one text part and answers with a server-sent event stream of text. Everything else an SDK expects is the adapter’s problem.

One thing before the engineering: thaipass is a reverse-engineered adapter over an undocumented private API, and I run it against one account, which is mine. It is not a service and this post is not an invitation to point it at someone else’s programme. What transfers is the inventory: the contract items you notice only when the backend does not have them.

> **Where the code comes from:** Every snippet is quoted from thaipass at commit 6b1b54d, and the refusal strings in the last section were measured on 12 September 2026 against one account.

## The backend takes a string

Three protocols come in with different message shapes, and all three reduce to one internal turn before anything goes upstream. A turn is a role, text, the tool calls the assistant made, and the id of the call a tool result answers. Files ride beside the text because they are uploaded separately, not flattened into it.

**What the adapter does between the SDK and the backend**

```trazo
client[Client SDK] --> turns[One turn shape]
turns --> flat[Flatten to one string]
flat --> backend[Chat backend]
backend --> sse[SSE text deltas]
sse --> splitter[Fence splitter]
splitter --> calls[Tool calls]
splitter --> text[Text]
calls --> reply[Reply in the client protocol]
text --> reply
```

The system prompt and the tool guide become blocks above the conversation, and every remaining turn becomes a labelled line:

```typescript packages/core/src/translate.ts
const roleLabel = (turn: ChatTurn, names: Map<string, string>): string => {
  if (turn.role === "assistant") {
    return "Assistant";
  }
  if (turn.role === "tool") {
    const name = turn.callId === undefined ? undefined : names.get(turn.callId);
    return name === undefined ? "Tool" : `Tool (${name})`;
  }
  return "User";
};
```

The labels change how a model answers, so a request that does not need them does not get them. One user message, no system prompt, and no tools goes upstream as the bare text the caller sent. Anything else gets the transcript treatment, because a model reading `Assistant:` and `Tool (read_file):` can at least tell whose words it is looking at.

That is the whole translation layer, and it is the least interesting part. The cost shows up in what the flattening throws away.

## Tools become prose, and prose has to become tools again

A tools API gives the model a slot that is not the text. Without one, the tool contract has to be written into the prompt and read back out of the answer. The guide that goes upstream names the fence the model should write, then lists what it may call:

```typescript packages/core/src/tools.ts
export const renderToolGuide = (tools: readonly ToolDefinition[]): string =>
  [
    "You can call tools. To call one, write a block exactly like this, one block per call, then stop and wait for the result, which arrives in the next turn:",
    `${FENCE_OPEN}{"name": "<tool name>", "input": {<arguments matching the tool's input schema>}}${FENCE_CLOSE}`,
    `Tools:\n\n${tools.map(renderTool).join("\n")}`,
  ].join("\n\n");
```

The reply then goes through a splitter that turns any fenced block naming an offered tool into a call, and leaves everything else as text. A block that never closes is text. A block holding a tool nobody offered is text. The caller sees `tool_calls` in its own protocol and never learns that the contract was a paragraph.

With a real tools API, a malformed call is the provider’s bug. Here it is mine, and the failure modes are the ones prose has.

## A streaming parser cannot take text back

The interesting constraint is not parsing the fence. It is parsing the fence in a stream, where every character released to the caller is already on someone’s terminal.

The fence that opens a call is a code block tagged `tool_call`. A chunk ending in two backticks might be the first characters of one, or it might be inline code in an ordinary answer, and nothing in the chunk itself decides which. So text is released only once it can no longer open a fence, and the tail that could still turn into one is held:

```typescript packages/core/src/tools.ts
// How many trailing characters could still turn out to open a fence.
const heldLength = (tail: string): number => {
  const longest = Math.min(tail.length, FENCE_OPEN.length - 1);
  for (let length = longest; length > 0; length -= 1) {
    if (FENCE_OPEN.startsWith(tail.slice(-length))) {
      return length;
    }
  }
  return 0;
};
```

Non-streaming, this whole feature is an `indexOf` and a `slice`. Streaming, it is a decision on every chunk about what can never be retracted. Release too eagerly and half a fence flashes in the output before the call is recognised. Hold too much and the answer arrives in visible steps. The bound here is 12 characters, which is the longest thing the parser is allowed to owe the reader.

## The model that dropped the fence

Some models write the object the guide asks for and leave the fence off. The reply then arrives as ordinary prose with a `stop` finish reason, and it reads like this:

```text
{"name":"load_skill","input":{…}}I can begin once the load_skill tool result is available.
```

The caller sees a finished answer that narrates a call nobody ran. That is worse than an error, because an error is retried and this is believed. The retry path cannot catch it either, since that path waits for a `tool-calls` finish reason, and a streaming caller cannot be retried at all.

The recovery rule is deliberately the narrowest one that covers what I observed. The object must be the first thing in the reply, it must parse, and it must name a tool the caller offered, which is the same bar a fenced block has to pass. Prose that quotes a call later in the answer is left alone, which matters when the caller is reviewing code and quotes one on purpose.

## Numbers the backend never sends

The backend reports no token counts, and `prompt_tokens` is not an optional field in practice: clients graph it, budget against it, and cut conversations with it. So the adapter counts characters and divides.

One divisor would have been wrong for the traffic I actually send. ASCII runs about four characters per token, while Thai, other non-Latin scripts, and emoji sit nearer one and a half, so the two are counted apart:

```typescript packages/core/src/tokens.ts
const ASCII = /[\u0020-\u007E\s]/gu;
const ASCII_CHARS_PER_TOKEN = 4;
const OTHER_CHARS_PER_TOKEN = 1.5;

export const charTokens = (count: CharCount): number =>
  Math.ceil(
    count.ascii / ASCII_CHARS_PER_TOKEN + count.other / OTHER_CHARS_PER_TOKEN
  );
```

Money works the same way, with one line I would not cross. The backend meters in credits and publishes no per-token price, so the credit figure is passed through exactly as it arrives, and the dollar figure is derived from OpenRouter’s public price for the same model. The derived number is documented as something to budget against and never as an invoice.

An estimate in a field named after a measurement is fine. An estimate in that field with nothing telling the reader it is one is a lie with good formatting.

## A key that is not a key

![A strip of punched paper tape printed with tiny type runs into a cracked black wax seal, a green printed line continuing through the crack](https://cdn.sanity.io/images/1ezd5mjg/production/3195714889abcb2c15db3306fced3731163f3c7e-1536x864.png?w=1600&fm=webp&q=80&auto=format)

The default credential for this backend is the session cookie, which every client copies into an environment variable. That cookie is the whole account. It cannot be scoped, it cannot be revoked on its own, and it has to be pasted again on every machine each time the session rotates.

A thaipass token is that cookie sealed under a key only the gateway holds. The app carries bytes it cannot read, the gateway unseals per request and forwards the cookie upstream as before, and nothing is stored anywhere:

```typescript packages/core/src/auth/seal.ts
const iv = randomBytes(IV_BYTES);
const cipher = createCipheriv(ALGORITHM, key, iv);
cipher.setAAD(Buffer.from(purpose, "utf-8"));
const body = Buffer.concat([
  cipher.update(JSON.stringify(payload), "utf-8"),
  cipher.final(),
]);
const sealed = Buffer.concat([iv, body, cipher.getAuthTag()]);
```

The same envelope carries the authorization code in the middle of a login. An app sends the account holder to the consent screen on the dashboard, which is where the session already is, and the gateway issues a code sealed the same way, with the PKCE challenge and the redirect inside it. The exchange swaps that code for a token, and neither step writes a row anywhere.

**Signing an app in, with nothing stored**

```trazo
sequenceDiagram
participant App
participant Dashboard
participant Gateway
App->>Dashboard: authorize, code_challenge
Dashboard->>Gateway: approve with the session cookie
Gateway-->>App: tp_c1_ code, sealed
App->>Gateway: exchange with code_verifier
Gateway-->>App: tp_v1_ token, sealed cookie
```

The algorithm is AES-256-GCM with a 32-byte key from the environment. The expiry travels inside the envelope, so an expired token is refused without a lookup. The purpose is authenticated rather than encrypted, which is what stops an authorization code in the middle of a login from being presented as an access token: the two carry different prefixes, `tp_c1_` and `tp_v1_`, and each opens only under its own additional data.

That design is the piece I would lift into another project unchanged, and it is worth being clear about what it buys and what it costs. It buys a stateless deployment, scopes an app cannot widen, and a token that dies with the session it stands for. It costs per-token revocation: the only levers are signing out upstream, which ends every token at once, and rotating the key, which does the same. There is no refresh token, and there cannot be one, because no proxy can renew someone else’s browser session.

## When the edge answers before the model

The last gap is error semantics. The backend’s edge scores some requests itself and answers `403` before the model runs, on what the prompt contains rather than how long it is. Resending the same body gets the same verdict, so the adapter reports it to the caller as a request error with the edge’s own body attached, rather than as something a retry loop should keep hitting.

The list of strings that trip it is measured, not theorised. As of 12 September 2026, `document.cookie`, `document.write`, and `eval()` were each refused on their own, while `dangerouslySetInnerHTML` and the bare word `cookie` passed. `eval()` is listed with its parentheses because that is the form that was sent, and the real rule is certainly wider than the three.

A caller who wants those strings dropped before the prompt goes upstream sets `x-thaipass-withhold: 1`, and each match is replaced by a marker saying something was removed. It is off by default. Editing someone’s prompt without telling them is worse than the refusal they came to avoid, and a marker the model can see is also a marker the reader can be shown.

## The inventory

Four promises, none of them in the protocol document: structured calls, usage numbers, a revocable credential, and errors that mean what a client’s retry logic thinks they mean. Each one had a place to live in the SDK long before I had anything to put there.

A compatibility layer is not a translation of a protocol. It is an inventory of promises, and you find out which ones you had been leaning on at the moment you have to write them yourself. The pieces above are what that inventory cost for one backend that only ever wanted a string.

---

**More posts:** [View all posts](https://www.pungrumpy.com/writing) | [Site map](https://www.pungrumpy.com/sitemap.md)