How to use a decentralized AI API with the OpenAI SDK

What OpenAI-compatible means here

Gonka Broker accepts the familiar chat-completions request shape, including message roles, model selection and streaming. Compatibility makes migration small, but it does not promise that every OpenAI endpoint, parameter or model-specific feature is available. Test the exact request shape your application needs.

Python setup

Install the official SDK and export the key in your shell:

python -m pip install openai
export GONKA_API_KEY="<your Gonka Broker key>"

Then run the Python client separately:

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.gonka.broker/v1",
    api_key=os.environ["GONKA_API_KEY"],
    timeout=60,
)

reply = client.chat.completions.create(
    model="gonka-auto",
    messages=[
        {"role": "user", "content": "Summarize this in three bullets."}
    ],
    max_tokens=300,
)
print(reply.choices[0].message.content)

JavaScript setup

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.GONKA_API_KEY,
  baseURL: "https://api.gonka.broker/v1",
  timeout: 60_000,
});

const reply = await client.chat.completions.create({
  model: "gonka-auto",
  messages: [{ role: "user", content: "Return one short answer." }],
  max_tokens: 200,
});
console.log(reply.choices[0].message.content);

Migration checklist

1. Keep the old provider configured until the new route passes your tests.
2. Change only the base URL, API key and model name first.
3. Test a short non-streaming request and inspect its usage record.
4. Test streaming separately if your application depends on it.
5. Set output limits, timeouts and bounded retries.
6. Test the actual prompts, response parsing and tool behavior used by your application.

Errors and safe retries

Treat authentication failures as configuration problems, not retryable traffic. Retry temporary rate-limit or server errors only with a small attempt limit and backoff. A retry that later succeeds can create additional billed usage, so never use an unbounded loop.

Cost and operating limits

Input and output tokens use the published Gonka Broker rate. Successful requests are charged against the prepaid balance. Begin with short prompts and a small allowance, then review usage before enabling scheduled or autonomous workloads. See current pricing, the gonka-auto model page, and the verification methodology.

When to keep a fallback

Keep another route for workloads that require a specific model capability, fixed latency behavior or a feature you have not verified through Gonka Broker. OpenAI compatibility reduces integration work; it is not a guarantee that two providers behave identically.

Next step: Create an account or read the integration docs.