Short answer: keep one OpenAI-compatible client, move the base URL and model name into configuration, and test capabilities before switching. The application code stays the same while the provider changes.
Separate the client from the provider
Most SDKs accept a base URL and an API key. Point both at a gateway that exposes an OpenAI-compatible interface, then select the model by name. Your code calls chat.completions.create; the gateway routes to DeepSeek, Qwen, Kimi or another provider.
client = OpenAI(
base_url=os.environ["AI_BASE_URL"],
api_key=os.environ["AI_API_KEY"],
)
resp = client.chat.completions.create(
model=os.environ.get("AI_MODEL", "deepseek-chat"),
messages=messages,
)Use model aliases
Hard-coded model IDs spread across a codebase make migration painful. Define aliases such as fast, reasoning and long-context in one configuration file. Changing a provider becomes a one-line edit, and you can run A/B tests by pointing an alias at a different model.
Check capabilities before switching
| Capability | Test | Failure symptom |
|---|---|---|
| Streaming | Stream a 200-token response | Empty chunks or a single blob |
| Tool calling | Request a function with a JSON schema | Tool call returned as plain text |
| JSON mode | Ask for a schema-valid object | Markdown fences or missing fields |
| Context | Send a document near the limit | Truncation or 400 errors |
Run these tests in CI against every model you enable. A model that passes chat but fails tool calling will break agent workflows in production.
Add a fallback path
Wrap the call in a retry with a secondary model for timeouts and 5xx responses. Retry 429s with exponential backoff first; switch models only when the primary is unavailable. Log the model, latency and token counts so cost and quality stay visible.
Keep prompts portable
Provider-specific prompt tricks reduce portability. Write system instructions in plain language, keep examples short, and validate output with a schema. If a model needs a special format, apply it in a thin adapter rather than across the application.
For cost tracking, read AI token cost math. For a broader comparison, see DeepSeek vs Qwen vs Kimi. Access is available through the TOKEN category.
Frequently asked questions
Do I need a new SDK for each provider?
No. An OpenAI-compatible gateway lets you keep one SDK and change the base URL and model name.
Will prompts behave the same on every model?
Mostly, but tool calling and JSON output vary. Validate structured output with a schema and keep prompts provider-neutral.
How do I roll back a model change?
Keep model names in configuration or aliases so a rollback is a config change, not a code deploy.
No comments yet. Be the first to share your thoughts!