How I Built Multi-Provider AI Routing in CodeIgniter 4

When you start integrating AI into a real product, the first temptation is to call the OpenAI SDK directly from a controller. It works. For a week. Then one of three things happens: OpenAI rate-limits you in the middle of a demo, the cost line on the invoice gets uncomfortable, or a customer asks "can you run this on Gemini instead?" and you realise every controller in the codebase has openai.com hardcoded into it.

I've now built provider-agnostic AI orchestration into three different PHP projects on CodeIgniter 4. The pattern that survived contact with reality looks like this: a single AIManager service, a thin Provider interface, database-backed config, and a controller layer that does not know which model it's talking to. This post is the version I wish I'd read before I started.

The shape of the problem

In a single product I have at any given time:

• Long-form content generation (blog drafts, work entries) — wants a smart model, latency is fine
• Chat replies — wants a fast model, latency matters more than quality
• Embeddings — wants a cheap, specialised model
• Translation — wants something fluent in 12 languages
• Lead scoring and intent classification — wants a small, deterministic model
• Sentiment / emotion — wants a small model

That is at minimum two providers and three models. If I want a fallback path, it is two providers, three models, and a routing table. If I want to swap providers per environment (a free local model in dev, a paid hosted model in prod), it is a routing table that reads from config.

You cannot keep this in your head. You cannot keep it in if/else blocks either, because every controller that does AI work ends up duplicating the same provider-selection logic — and the day you change defaults, you change every controller.

The shape of the solution

Controller / Skill
       │
       ▼
  AIManager  ──► resolveProvider(task) ──► [provider table]
       │
       ▼
  Provider interface
   │     │      │
  OpenAI Gemini Ollama

Three pieces. The controller asks AIManager for a thing. AIManager looks at the task type and consults the provider table to pick a provider. The provider speaks one common interface. The controller never knows.

That is the whole pattern. Everything else is bookkeeping.

The interface

Every provider implements one tiny contract:

namespace App\AI\Providers;

interface ProviderInterface
{
    public function complete(string $prompt, array $options = []): array;
    public function embed(string $text): array;
    public function isAvailable(): bool;
}

Three methods. complete() for chat/text generation. embed() for vectors. isAvailable() for the routing logic to know whether to skip this provider before it makes the call.

Implementing OpenAIProvider is about 80 lines. Implementing GeminiProvider is another 80. Adding a third provider — Anthropic, Ollama, an internal one — is a copy of the same skeleton. The whole point of the interface is to keep "talks to LLMs" out of every other layer.

The manager

namespace App\Services;

class AIManager
{
    public function __construct(
        private AiProviderModel $providers,
        private RoutingService $routing,
        private AiLogModel $logs,
    ) {}

    public function complete(string $task, string $prompt, array $options = []): array
    {
        $provider = $this->resolveProvider($task);

        try {
            $start = microtime(true);
            $result = $provider->complete($prompt, $options);
            $this->logs->record($task, $provider->name(), $result, microtime(true) - $start);
            return $result;
        } catch (\Throwable $e) {
            return $this->fallback($task, $prompt, $options, $provider, $e);
        }
    }

    private function resolveProvider(string $task): ProviderInterface
    {
        $row = $this->routing->routeFor($task) ?? $this->routing->defaultRoute();

        $providerRow = $this->providers->find($row['provider_id']);
        if (!$providerRow || !$providerRow['is_active']) {
            throw new \RuntimeException("No active provider for task {$task}");
        }

        return $this->buildProvider($providerRow);
    }
}

The interesting part is what is missing: the controller doesn't pass a model name. It passes a task name. blog_draft, social_post_short, lead_intent, embedding. The routing table maps task → provider → model. Change the table, change the behaviour, no code change.

The provider table

CREATE TABLE ai_providers (
  id INT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
  name VARCHAR(80) NOT NULL,            -- 'openai', 'gemini', 'ollama'
  display_name VARCHAR(120),
  base_url VARCHAR(255),
  api_key_encrypted TEXT,
  default_model VARCHAR(120),
  is_active TINYINT(1) DEFAULT 0,
  priority INT DEFAULT 100,             -- lower = preferred
  rate_limit_per_min INT DEFAULT 60,
  created_at DATETIME,
  updated_at DATETIME
);

CREATE TABLE ai_routes (
  id INT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
  task VARCHAR(80) NOT NULL,            -- 'blog_draft', 'embedding', ...
  provider_id INT UNSIGNED NOT NULL,
  model VARCHAR(120) NOT NULL,
  fallback_provider_id INT UNSIGNED NULL,
  is_active TINYINT(1) DEFAULT 1,
  UNIQUE KEY uniq_active_task (task, is_active),
  FOREIGN KEY (provider_id) REFERENCES ai_providers(id)
);

Two tables. ai_providers is "where can we talk to a model." ai_routes is "for this task, talk here, and if that fails, talk there."

The API key is encrypted at rest. In CodeIgniter 4, that's the built-in Encryption service with a key from .env. Never store keys in plain text. Never. Even in dev.

The fallback that actually fires

This is the bit that everyone gets wrong on the first try. The try { call() } catch { fallback() } pattern looks right and is dangerously naive in production. Three failures that don't throw exceptions:

1. Provider returns 200 OK with an error envelope. OpenAI does this when content moderation kicks in. The HTTP request succeeded; the model "refused."
2. Provider returns 200 OK with empty content. Truncation, content filter, or just a flaky day. You get an empty string and your downstream code happily saves an empty blog post.
3. Provider returns 200 OK with a different schema than you expected. Gemini's parts[] structure looks nothing like OpenAI's choices[0].message.content. Without normalisation in the provider class, your fallback never triggers because the call "succeeded."

The fix is at the provider level: every implementation normalises the response into a common shape and validates it before returning. Empty content is an exception, not an empty string. The manager only sees "it worked" or "it threw."

// Inside OpenAIProvider::complete()
$body = json_decode($response->getBody(), true);

if (!isset($body['choices'][0]['message']['content'])) {
    throw new ProviderException('OpenAI returned no content');
}

$content = trim($body['choices'][0]['message']['content']);
if ($content === '') {
    throw new ProviderException('OpenAI returned empty content');
}

return [
    'content' => $content,
    'model'   => $body['model'] ?? null,
    'tokens'  => $body['usage']['total_tokens'] ?? 0,
];

Now the fallback in AIManager works as intended: the catch block fires for both transport errors and semantic failures, and the next provider in the routing table gets the call.

The pinning bug nobody sees

A bug I shipped into one project and only noticed in production: I cached the resolved provider on the conversation object. Worked great until I wanted to swap models mid-conversation. Every conversation was permanently pinned to whichever provider it started with — even after I rewired the routing table.

The lesson: resolve the provider at the call level, not the session/conversation level. If you want to remember which provider answered a specific message, log it. If you want to keep a conversation on the same provider for consistency, pass that explicitly in the call options. Don't let the cache make routing decisions you didn't intend.

A real-world version of this in another project of mine: chat latency dropped from ~5800 ms to ~500 ms after fixing a provider-pinning bug like this and switching the default route to a faster model. No infrastructure change. One config row.

The logs you'll actually need

Every AI call writes a row:

CREATE TABLE ai_logs (
  id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
  task VARCHAR(80) NOT NULL,
  provider VARCHAR(80) NOT NULL,
  model VARCHAR(120),
  prompt_tokens INT,
  completion_tokens INT,
  total_tokens INT,
  cost_estimated_usd DECIMAL(10,6),
  latency_ms INT,
  was_fallback TINYINT(1) DEFAULT 0,
  status VARCHAR(40),
  error_message TEXT,
  created_at DATETIME
);

The two columns nobody adds on day one and everybody adds on day thirty: was_fallback and cost_estimated_usd. The first tells you when the primary provider is misbehaving. The second tells you which task is silently eating your budget.

You don't need a real-time observability platform for this. A weekly query on ai_logs will tell you:

• Which provider has been falling back the most
• Which task is consuming the most tokens
• Whether your routing decisions still make economic sense

That's enough to drive every meaningful adjustment.

The configuration surface that scales

Provider configuration lives in the admin panel, not in code. A simple form:

• Provider name (OpenAI / Gemini / Ollama)
• Base URL
• API key (encrypted on save, masked on render, preserved if left blank)
• Default model
• Active toggle
• Priority

When you add a new provider — Anthropic last quarter, Mistral last week — you don't deploy. You go to /admin/ai-providers, paste the key, hit save, then update the routing table to send some tasks there.

This is the only sane way to operate. Engineering hours are expensive. Admin clicks are free.

What this gives you, in business terms

A founder pitch I've used: "We're not locked in to any AI provider. If GPT-4o doubles in price tomorrow, we can move 80% of our traffic to Gemini Flash through an admin form, and our customers won't notice."

That is technically true and economically powerful. It's also why the abstraction is worth the effort even when you're "just using OpenAI." The cost of building it once is small. The cost of not having it on the day the bill arrives is large.

What I'd do differently next time

1. Build the interface first, the OpenAI provider second. I started with raw OpenAI calls "to ship faster" and refactored later. Refactoring later was twice the work of writing the interface first.
2. Don't skip isAvailable(). I treated it as nice-to-have. It is critical. A provider with a dead key shouldn't be tried — it should be skipped silently. Otherwise every request pays the latency of one timeout before the fallback fires.
3. Log tokens and cost from day one. You will never go back and reconstruct historical cost data. Log it now even if you don't query it yet.
4. Routing is config, not code. The single biggest improvement to operational sanity. Once routing is in a table, swapping behaviour becomes a data change with an audit trail.
5. Normalise the response in the provider. The manager should never see provider-specific shapes. If you find yourself doing if ($provider->name() === 'gemini') anywhere outside the provider class, the abstraction has leaked and you have a maintenance bomb.

Wrap-up

Multi-provider AI is not a science project. It's table stakes for anyone shipping AI into a real business — because the providers will change underneath you whether you plan for it or not. CodeIgniter 4 happens to be a perfectly reasonable framework for this; the pattern works equally well in Laravel, Symfony, Django, Rails. What matters is the discipline: one manager, one interface, two tables, one log.

Build it on day one. Be glad on day a hundred.

If you want to see the rest of the stack this AIManager sits inside — workflow engine, hybrid RAG, multi-language content — there's more in this series at marufsheikh.net.