Why I Built My Own Workflow Engine

Every few months a developer asks me a version of the same question. "You're on PHP. Why didn't you just use Camunda? Or n8n? Or Temporal? Or even just a cron job?" The honest answer is that I tried most of them, in product after product, and they all eventually broke against the same wall: workflows that need to know about my data, my events, and my AI calls — without a layer of indirection that costs more than the engine saved.

So I built my own. It's simple, it's small enough to fit in your head, it lives in the same codebase as the rest of the product, and it has been stable enough to run lead routing, AI replies, comment moderation, scheduled posts, and CRM sync in production. This is what it does, what it doesn't try to do, and why I would build it again.

The shape of the problem

A "workflow" in business software usually means one of three things:

1. A trigger-driven sequence. "When a lead is created, score it, then if score > 80 notify Telegram, else add to nurture sequence."
2. A scheduled batch. "Every 15 minutes, scan pending social comments, classify intent, generate AI reply drafts."
3. A long-running process with human steps. "Receive contract, send to legal, wait up to 7 days, escalate if no response."

These are three different problems. Off-the-shelf engines try to solve all three at once. That ambition is exactly the cost. Camunda's BPMN modeller is gorgeous and produces XML that nobody wants to write by hand and a Java runtime nobody wants to operate. Temporal is genuinely good and brings a steep learning curve and an operational footprint that's overkill for a 50-employee SaaS. n8n is a beautiful no-code product and an awkward fit when your "nodes" need to call internal services that don't have HTTP endpoints.

For a small-to-medium PHP product, what you actually need is a system that supports cases 1 and 2 cleanly, with case 3 representable as a chain of case-1 triggers tied together by database state. That's a much smaller piece of software.

What I built

Three concepts:

• Workflow — a named, versioned definition. Has a trigger (event name or cron expression), a list of steps, and a status.
• WorkflowExecution — an instance of a workflow running. Records start time, current step index, status, payload, and results.
• Step — a single unit of work in a workflow. Has a type (skill, model, query, webhook, wait, branch) and a payload.

The whole runtime is one service:

namespace App\Services;

class WorkflowService
{
    public function __construct(
        private WorkflowModel $workflows,
        private WorkflowExecutionModel $executions,
        private AIManager $ai,
        private QueueService $queue,
    ) {}

    public function execute(int $workflowId, array $payload, array $options = []): int
    {
        $workflow = $this->workflows->getWithSteps($workflowId);
        if (!$workflow || !$workflow['is_active']) {
            throw new \RuntimeException("Workflow {$workflowId} not active");
        }

        $execId = $this->executions->start($workflowId, $payload);

        try {
            foreach ($workflow['steps'] as $step) {
                $this->executions->updateProgress($execId, $step['index']);
                $result = $this->executeStep($step, $payload);
                $payload = array_merge($payload, $result);

                if ($result['__halt'] ?? false) {
                    break;
                }
            }

            $this->executions->markCompleted($execId, $payload);
        } catch (\Throwable $e) {
            $this->executions->markFailed($execId, $e->getMessage());
            throw $e;
        }

        return $execId;
    }

    public function executeAsync(int $workflowId, array $payload): int
    {
        return $this->queue->dispatch('workflow', [
            'workflow_id' => $workflowId,
            'payload' => $payload,
        ]);
    }
}

That's the engine. Synchronous execute for cases where the caller wants the result. Async execute that pushes onto the same generic job queue the rest of the product uses. Steps are dispatched by type. Results merge into the payload so later steps can reference earlier output.

Step types I actually use

Six. That's the whole vocabulary:

private function executeStep(array $step, array $payload): array
{
    return match ($step['type']) {
        'skill'   => $this->runAISkill($step, $payload),
        'model'   => $this->runModelOp($step, $payload),
        'query'   => $this->runQuery($step, $payload),
        'webhook' => $this->callWebhook($step, $payload),
        'wait'    => $this->scheduleResume($step, $payload),
        'branch'  => $this->evaluateBranch($step, $payload),
        default   => throw new \RuntimeException("Unknown step type: {$step['type']}"),
    };
}

• skill — run an AI skill (blog draft, lead intent classifier, sentiment, etc.). The skill registry maps names to classes; AIManager handles provider routing.
• model — perform a CI4 ORM operation. Insert, update, find, delete. The payload tells the step which model and which method.
• query — raw SQL or query builder when ORM-via-model isn't expressive enough.
• webhook — emit a signed webhook to a registered URL. HMAC-SHA256 over the payload.
• wait — schedule the workflow to resume at a later time. Backed by the same scheduled-tasks system the rest of the cron jobs use.
• branch — evaluate a condition on the payload, select a sub-sequence of steps.

Six types fit on a back-of-envelope diagram. Anything that doesn't fit one of these isn't a workflow step — it's application code that the step calls into.

How a real workflow looks

Lead intake from a social comment, end to end:

{
  "name": "Social comment to qualified lead",
  "trigger": {"type": "event", "name": "comment.received"},
  "steps": [
    {
      "type": "skill",
      "name": "lead_detection",
      "input": "comment.content",
      "output": "is_lead"
    },
    {
      "type": "branch",
      "condition": "is_lead == true",
      "if_false_halt": true
    },
    {
      "type": "skill",
      "name": "lead_intent",
      "input": "comment.content",
      "output": "intent"
    },
    {
      "type": "skill",
      "name": "lead_temperature",
      "input": ["comment.content", "intent"],
      "output": "temperature"
    },
    {
      "type": "model",
      "model": "SocialLeadModel",
      "method": "create",
      "payload": {
        "comment_id": "comment.id",
        "intent": "intent.value",
        "temperature": "temperature.score"
      }
    },
    {
      "type": "branch",
      "condition": "temperature.score >= 90",
      "if_true": [
        {"type": "webhook", "url": "notification.telegram_hot_lead"}
      ]
    }
  ]
}

JSON, stored in a column, edited via an admin form. It is not the most beautiful DSL ever invented. It is small, easy to read, easy to validate, and a junior engineer can write one in twenty minutes.

What this gives you that the alternatives don't

Same codebase. Workflow steps that call your models, your services, your AI manager are one method call away. No HTTP round-trip to your own server to do something the server already knows how to do.

Same database. Workflow executions, results, errors, queue state — all in the same MySQL. Cross-joining "show me all workflow executions for this lead" is a normal query, not a multi-system data integration project.

Same auth. A workflow doesn't have to authenticate against itself to invoke its own code. Permissions are the application's permissions.

Same audit log. Every step writes to the existing admin audit log. No "where did the workflow do that?" mystery.

Trivial deploy. It's one service class and a couple of tables. No new daemon. No new container. No new operator paging on call.

What it doesn't try to be

Not a BPMN engine. No swim lanes, no compensating actions, no formal model of timeouts beyond the wait step. If your business genuinely needs BPMN, you have problems that Camunda exists to solve and you should pay them.

Not a no-code platform. Admins can edit workflows in JSON; that is not the same thing as Zapier. If your customers need to author their own integrations, n8n is a better product.

Not a distributed task orchestrator. There is one queue worker. It runs on the web server. If the server falls over, the worker falls over. That is fine for a single-server product. It is not fine for a system that needs multi-region failover.

These are exactly the boundaries where off-the-shelf engines start to earn their keep. I have not crossed them. When I do, I'll pay for them.

Bugs that taught me things

Method case. I shipped a create() action that silently never fired. The cause: CodeIgniter 4.4+ changed getMethod() to return uppercase 'POST' while my check was === 'post'. The form submission rendered the GET view instead of saving. Three weeks of "huh, the workflow editor doesn't seem to persist." Now I use is('post') everywhere and the type system catches this kind of drift.

Null name. Workflow creation hit the database with a blank name and threw a NOT NULL exception. The admin saw an unfriendly 500. Fix: validate the name before the insert, redirect back with a friendly error. Cheap, obvious, easy to forget.

Missing-row 200. A delete() call on a non-existent workflow returned 200 OK from the database (zero rows affected, no error). The admin UI thought everything worked. Fix: check existence first, return a real 404 if it isn't there.

JSON in step config. Storing step config as JSON was the right call. But early on I parsed it inline in the executor on every step. For a 20-step workflow that ran 1,000 times in a batch, that was 20,000 redundant json_decode calls. Parse once at workflow load, pass the array down.

Cycle detection. Or rather, lack of it. A workflow with a webhook step that triggered another workflow that triggered the first one ran until the queue worker fell over. I now plan to add a depth check on workflow_executions — flag rows as in-progress, refuse to start the same workflow recursively beyond N levels. Not done yet. On the list.

When you should not build this

You should not build a workflow engine if any of these are true:

• You have one workflow. Write it as a service class. You don't need an engine.
• You have customers who author workflows themselves. Buy n8n or Make (Integromat) and don't reinvent it.
• You need distributed execution, exactly-once delivery, or formal compensating actions. Pay Temporal.
• Your team is two people and the current cron-job-and-some-glue setup is working. Don't pre-build infrastructure.

You should build it if all of these are true:

• You have more than three "if X then Y, sometimes Z" flows.
• They all run inside one product.
• They all reach into your application's code and data.
• Operating an external orchestrator would be a noticeable percentage of your team's attention.

That's the exact case I was in. The engine pays for itself the moment you stop writing one-off cron scripts and start composing them.

What I'd watch out for

A workflow engine that grows uncontrolled is its own technical debt. Two rules that have saved me:

1. Steps stay dumb. A step calls one function. If a step needs more than that, you don't need a fancier step — you need a service method that the step calls.
2. No business logic in the engine. The engine routes, executes, logs. The decision what to do lives in skills, models, and services. If you find yourself adding if/else to executeStep, refactor.

Following these two rules, the engine has stayed at roughly the size of a single, readable PHP class for a year now. Most additions have been step types, not logic. New step types are small, isolated, and reviewable.

Wrap-up

The framing I'd push back on hardest is "build vs buy." For workflow engines specifically, the right framing is "what is the smallest engine my product actually needs, and can I keep it under the bar where it starts to compete with grown-up tools?"

If you can keep it under that bar — and most small-to-medium products can — building is the cheap, fast, controllable answer. If your usage is heading north of that bar, switch. Don't try to grow your own engine into Temporal. Use Temporal.

A small workflow engine that knows your data is one of the highest-leverage pieces of internal infrastructure a product can have. It is also one of the easiest to over-build. Build only what you'll use this quarter. Add the seventh step type the day you actually need it.