You added read replicas so the primary database could rest. Then you look at the metrics and the primary is still doing most of the reads. If you run queue workers with sticky turned on, the workers may be the reason.
sticky doesLaravel lets you split a database connection into a read side and a write side. Selects go to the read hosts, and inserts, updates and deletes go to the write host (Laravel docs).
Replicas lag a little behind the primary. So if you write a row and read it back right away from a replica, it may not be there yet. The sticky option fixes that. The docs describe it this way: if a write has run \"during the current request cycle\", any further reads use the write connection, so you can read back what you just wrote (Laravel docs).
'mysql' => [\n 'driver' => 'mysql',\n 'read' => ['host' => ['10.0.0.11', '10.0.0.12']],\n 'write' => ['host' => ['10.0.0.10']],\n 'sticky' => true,\n // ...\n],\nIn a web request this is a good trade. The request ends, the process state resets, and the next request starts on the replicas again.
\nA queue worker does not end after one job. php artisan queue:work boots once and keeps running jobs until it is restarted. There is no \"request cycle\" boundary between jobs.
The Laravel News write-up of the release puts it plainly: \"a single job that writes can send the reads of every later job in that worker to the write connection\" (Laravel News).
\nThe pull request that added the fix gives a concrete picture. For a worker with a high memory limit, one job that writes \"could mean for the next 10,000 jobs it uses the write connection\", or until the next deploy restarts the worker, \"which means we actually lose the benefit of having replicas\" (PR #61915).
\nMost job workloads write something. So in practice, almost every long-running worker ends up reading from the primary.
\nLaravel v13.36.0 was released on October 13, 2026 (GitHub release). It includes a new static flag on the queue worker, merged on October 7 (PR #61915):
\nuse Illuminate\\Queue\\Worker;\n\n// AppServiceProvider::boot()\nWorker::$flushStickyConnections = true;\nWith the flag on, the worker resets the sticky state before each job. Each job starts on the replicas, and only switches to the primary for reads after it writes something itself.
\nIt is off by default. The reason given is that \"a job could depend on reading what an earlier job in the same worker wrote\" (Laravel News). The PR author suggests it could become the default in Laravel 14, but did not want to break existing apps in a minor release (PR #61915).
\n\nJob 1 writes. Jobs 2 to N in that worker read from the primary until the worker restarts.
\nJob 1 writes. Job 2 starts fresh on the replicas. It only reads from the primary after its own first write.
\nAsk one question: does any job read data that a different job wrote moments before, and only works because both ran in the same worker?
\nIf the answer is yes, that is a hidden dependency. The PR author makes the same point: if jobs depend on other jobs, \"they should be a chain or a batch, not quietly relying on whichever worker process happened to pop them\" (PR #61915). With several workers, the second job can already land on a different process, so the dependency is fragile today.
\nIf the answer is no, turning the flag on is a small change with a clear upside: your workers start using the replicas you already pay for.
\nA simple rollout:
\nAppServiceProvider::boot().php artisan queue:restart).The flag does not remove replica lag. It only stops one job's write from deciding where the next job reads.
\nIf a job is dispatched right after a web request writes a row, and the job reads that row from a replica, it can still miss it. That was true before this change for the first job in a fresh worker, too. For those jobs, read from the write connection on purpose, for example with Model::onWriteConnection() (API docs), or dispatch the job with afterCommit() so it only runs once the transaction is committed (Laravel docs).
The same release has a few more changes for people running workers (Laravel News, GitHub release):
\n| Change | \nWhat it means | \n
|---|---|
#[DeleteWhenMissingModels] and #[FailOnTimeout] now work on queued mailables | \nA mail for a deleted model is dropped instead of failing, and a timed out mail is not retried and sent twice. | \n
| Redis queue notification pushes are batched | \nFewer round trips to Redis when pushing many notifications. | \n
| PgBouncer SSL EOF errors count as lost connections | \nLaravel can reconnect instead of failing the job. | \n
| Mercure broadcasts run synchronously by default | \nSet MERCURE_ASYNC=true to queue them again. | \n
Have you checked where your workers actually read from?
\nAn AI can write 10,000 lines of code in a minute. I don't care. I care whether those 10,000 lines survive my constraints.
\n\n- AI made writing code cheap, so the scarce work is deciding what is allowed to merge and ship.\n- Stack your gates from cheap to expensive: static analysis, unit and integration tests, mutation testing, required CI checks, then monitoring with a fast rollback.\n- Keep human review for design and risk, and turn every comment you repeat into a rule the pipeline enforces.\n\nCode generation stopped being the slow part of my week. An agent can scaffold a Laravel module, its migrations and its tests before my coffee cools.
\nThat speed is real, and I use it every day. It also changed the question. \"Can we write this?\" is cheap now. \"Should this reach production?\" is the expensive one.
\nSo I stopped judging my work by how much code I write. I judge the system around the code: what it refuses, how fast it refuses it, and how fast it recovers when something slips through.
\nWhen code was slow to write, the writing itself filtered out bad ideas. You thought twice before typing 400 lines by hand.
\nThat filter is gone. Volume goes up, and every weak spot in your process gets hit more often.
\nI covered the trade-off in AI vs traditional development and the loop itself in agentic workflows. This post is the next step: the concrete gates I put between generated code and real users.
\nThink of them as a row of doors. Each one is cheap to run, catches one kind of failure, and says no without caring who wrote the code. An agent, a teammate, or me at 1am: same doors.
\n\nThe cheapest gate runs before any test. PHPStan reads your code and finds whole classes of bugs without executing it. Larastan teaches it about Laravel: Eloquent models, facades and container bindings.
\nPHPStan has levels from 0 to 10. Level 10 arrived with PHPStan 2.0 and treats every mixed type strictly, including the implicit ones you get from missing types.
On an existing app you won't jump to level 10 on day one, and you don't need to.
\nGenerate a baseline first, then add it to includes. Old errors are recorded and new code has to meet the higher bar.
vendor/bin/phpstan analyse --generate-baseline\n# phpstan.neon\nincludes:\n - vendor/larastan/larastan/extension.neon\n - phpstan-baseline.neon\n\nparameters:\n paths:\n - app/\n level: 8\nThen ratchet. Every few weeks, clear part of the baseline, and raise the level by one when it gets small. That file is a to-do list that can only get shorter.
\nAgents are happy here. Point one at a failing PHPStan run and it will fix types all afternoon without complaint. That's exactly the work I want to hand over.
\n\nRun PHPStan one level above your current setting with `--generate-baseline`, commit the baseline, and raise the level in CI. From today, new code meets the stricter rule.\n\nUnit tests pin down the rules: pricing, proration, state changes, the logic that costs money when it's wrong. Integration tests (Laravel calls them feature tests) prove the pieces work together: the route, the policy, the database and the queued job.
\nI lean on feature tests for AI-written code. An agent can make every unit test pass and still wire the controller to the wrong policy. A feature test that hits the real route as a real user catches that.
\nPest also gives you architecture tests, which turn \"we don't do that here\" into a failing build:
\n// tests/ArchTest.php\narch()->preset()->laravel();\narch()->preset()->security();\n\narch('no debug calls ship')\n ->expect('App')\n ->not->toUse(['dd', 'dump']);\nOn an older app, start with ->ignoring() for the folders you haven't cleaned up yet.
When I write the spec first, I write it as tests. The agent gets a red suite and a clear finish line. If it can't turn the suite green honestly, I usually learn something about my spec.
\nCoverage tells you a line ran. It doesn't tell you a test would notice if that line were wrong. That gap is where generated tests like to hide, with assertions like \"the result is not null\".
\nMutation testing closes it. The tool changes your code on purpose (flips a > to >=, drops a method call, returns an empty array) and reruns your tests. If the tests still pass, that mutant escaped, and you just found a test that checks nothing.
Pick the tool for your stack:
\n| Stack | \nTool | \nFail the build with | \n
|---|---|---|
| PHP on Pest | \nPest's built-in --mutate | \n--min=70 | \n
| PHP on PHPUnit | \nInfection | \n--min-msi=70 | \n
| JavaScript or TypeScript | \nStrykerJS | \nthresholds.break | \n
One detail to know before you wire this up: Infection's Pest support covered Pest 1 only. From Pest 2 on, Pest runs its own mutation engine, so a Pest suite uses pest --mutate, and it mutates the classes your tests name with covers() or mutates().
On a PHPUnit suite, Infection is the usual choice. Its --git-diff-lines flag mutates only the lines a pull request touched, which keeps CI fast on a big codebase.
For JavaScript, Stryker exits with code 1 when the score drops below break:
{\n \"thresholds\": { \"high\": 80, \"low\": 60, \"break\": 60 }\n}\nThe 70 and 60 in these examples are starting points. Pick the score your suite hits today, gate on it, and raise it as the tests improve. Why this matters so much for agent-written tests is in trust is not a QA strategy.
\nAll of this is a suggestion until the merge button respects it. Here is the workflow I start with on a Laravel app that uses Pest:
\nname: quality-gates\non: [pull_request]\n\njobs:\n static:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v7\n - uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.4\"\n - run: composer install --no-interaction --prefer-dist\n - run: vendor/bin/phpstan analyse --no-progress\n\n tests:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v7\n - uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.4\"\n - run: composer install --no-interaction --prefer-dist\n - run: cp .env.example .env && php artisan key:generate\n - run: vendor/bin/pest --parallel\n\n mutation:\n needs: tests\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v7\n with:\n fetch-depth: 0\n - uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.4\"\n coverage: pcov\n - run: composer install --no-interaction --prefer-dist\n - run: cp .env.example .env && php artisan key:generate\n - run: vendor/bin/pest --mutate --parallel --min=70\n # PHPUnit suite? The same gate with Infection:\n # vendor/bin/infection --git-diff-lines --git-diff-base=origin/main --min-msi=70 --threads=max\nThe mutation job only has work to do once your tests name the classes they cover with covers() (or mutates()), so add those as you write tests.
Then open branch protection (or a ruleset) for main and turn on \"Require status checks to pass before merging\". Add all three jobs. Also tick \"Require branches to be up to date before merging\", so each pull request is tested against the code it will actually land on.
Now nobody merges red. No agent, no teammate in a hurry, and no version of me on a deadline. The wider picture of the pipeline is in CI vs CD.
\n\nOpen your repo settings and check which jobs are required on `main`. A quality job that isn't on that list is decoration. Add it today.\n\nSome bugs only show up with real data and real traffic. Your tests won't catch the query that is fine with 200 rows and slow with two million. So the last gate sits after the deploy.
\nI want three things there:
\n/up health route that returns 200 when the app boots. Hit it right after deploy and fail the pipeline if it doesn't answer.curl --fail --silent --max-time 10 https://app.example.com/up || ./rollback.sh\nMonitoring is also how the gates improve. When something escapes to production, I fix it, then I ask which earlier gate should have caught it, and I add the test or the rule there. Every incident should leave the doors tighter. The difference between having logs and actually being told is in logging vs monitoring.
\nWith the machines doing the mechanical checks, review changes shape. I don't read every line hunting for missing types anymore. The pipeline already did that, on every file, with the same attention.
\nMy review time goes to the questions no tool answers. Should this feature exist? Does this new queue or table fit the system? What happens to the money if this webhook arrives twice?
\nGitHub can route that attention for you. A CODEOWNERS file plus \"Require review from Code Owners\" means risky paths wait for a human, and everything else merges on green:
# .github/CODEOWNERS\n/database/migrations/ @your-org/leads\n/app/Policies/ @your-org/leads\n/app/Billing/ @your-org/leads\nMy full read list, and how I handle agent-sized diffs, is in stop reading every line of AI-generated code.
\nGenerating code is the easy part now. The craft is in the system that decides what ships.
\n\nWrite a `CODEOWNERS` file with the three paths where a mistake costs real money or data, and turn on code owner review for them. Let the rest merge on green checks.\n\n\"Moving engineers up the stack\" sounds like a conference slide. Here is what it means in my actual week:
\nThe engineers who do well with AI can say exactly what \"correct\" means, then make a machine check it on every push.
\n\nPick one feature from this sprint and write its acceptance tests before any code exists, yours or an agent's. Then compare that review with your last one.\n\n\n- Cheap code moves the hard work to deciding what may merge and ship.\n- Order your gates from cheap to expensive: static analysis, tests, mutation testing, required CI checks, then monitoring and rollback.\n- Use a baseline and a ratchet so standards rise without a big rewrite.\n- Spend human review on design and risk, and route it with CODEOWNERS.\n- Every bug that escapes should add a rule to an earlier gate.\n\nIf you want an outside look at which of these gates your Laravel pipeline is missing, my architecture audit covers the code, the CI and the deploy path.
\nWhich gate is missing from your pipeline right now?
\nChatbots made answers cheap. Bots are making the work around the answers cheap.
\n\n- The chatbot wave made getting an answer cheap, but you still did the work around it.\n- The new bots wake up on their own, use your tools, and stop at a draft for you.\n- Taste, sends in your name, costly decisions and setting the rules still need a person.\n\nA few years ago, AI meant a chat box. You asked a question, you got an answer, and then you did the work yourself.
\nIt was useful, but you were still the one who opened the apps, wrote the email and pressed send.
\nThat is changing fast. In the last few months, several big AI companies shipped some kind of bot that works on its own.
\nIt runs on a schedule, watches your inbox or your repo, uses your apps, and comes back with the work done or a draft ready for you. You don't have to be in the chat for it to happen.
\nI think this is the next big wave, and it will touch a lot more people than the first one did.
\nThe chatbot wave made one thing cheap: getting an answer. Explaining code, summarizing a long document, writing a first draft.
\nAll of that went from minutes or hours to seconds.
\nBut the job around the answer stayed the same. You still had to:
\nThe chatbot waited for you. If you didn't ask, nothing happened.
\nThe new bots flip that. You don't ask them each time. You give them an area and a set of rules once, and they keep going.
\n\nThe chatbot waited for you. If you didn't ask, nothing happened.
\nYou give the bot an area and a set of rules once, and it keeps going.
\nIn practice that looks like:
\nYou can read how two of these products compare in Grok Bot vs OpenAI dots, and how I set mine up in Run your business with one AI assistant per area.
\n\nPick one area and write its rules down once, so a bot could keep going without you asking each time.\n\nI run most of my week this way now. One bot looks after my website and blog. One handles social posts. One handles my inbox.
\nThey talk to each other in a shared chat, and I step in when they need a decision.
\nThe biggest change is not speed.
\nBefore, I was the one carrying context from one app to the next. Now the bots carry it, and I mostly review and decide.
\nA normal day looks like this: I open the chat, read what got done overnight, approve a few drafts, say no to one or two, and give a new idea.
\nThat's most of the admin part of my day.
\nBots doing the work does not mean bots running your life. A few things still need a person, and I think they will for a while:
\nIf anything, the skill that matters most now is being clear: clear about what each bot owns, what it may never touch, and when it should stop and ask.
\n\nFor one bot, write three lines: what it owns, what it may never touch, and when it should stop and ask.\n\nAnyone whose day is mostly moving information between tools. That covers a lot of jobs:
\nThis doesn't mean those jobs go away. It means the job moves. Less doing the steps, more deciding what the steps should be and checking the result.
\nPeople who learn to set up and run a small team of bots will get a lot more done than people who don't.
\n\nLook at your last week and mark the parts that were mostly moving information between tools. Those are the steps to hand over first.\n\nYou don't need a big setup to try this.
\nThat's it. Start with one area, keep the rules strict, and add more once you trust it.
\n\nPick one boring task you do every week and give one bot that task and nothing else.\n\n\nChatbots made answers cheap. Bots are making the work around the answers cheap.\nThe people who do well in this wave won't be the ones who type the best prompts. They'll be the ones who are best at deciding what should happen, setting clear rules, and saying yes or no quickly.\n
\nWhich boring weekly task would you hand to a bot first?
\nOne agent that does everything, or a team where each bot has one job? That is the real choice between OpenAI dots and Grok Bot.
\n\n- Both products now give an agent its own cloud computer, background work and approvals.\n- Dots start as one personal agent. Grok Bot starts as a team of focused bots.\n- The best test is one real weekly task, run for a week.\n\nOn 29 September, OpenAI launched dots at DevDay.
\nA day or two later, one founder posted that he had spent months building his business into a monorepo full of partitioned context, skills, automations and MCP connections. After 36 hours with dots, he thought most of it might not be needed anymore.
\nThat post hit home, because I run my own work the other way: a set of Grok Bot assistants, each with one job.
\nSo the honest question is not \"which one is smarter.\" It is \"which shape of agent fits the way you work.\" This post compares the two based on what each product has published, and where my own daily use backs it up.
\n| \n | Dots | \nGrok Bot | \n
|---|---|---|
| Starting shape | \nOne primary dot, named and personalized | \nA team of Bots, one per area of work | \n
| Control | \nCustom Rules plus an Auto-review layer | \nScopes, drafts and approvals | \n
| Recurring work | \nGoals that keep going in the background | \nNamed skills and routines | \n
| Where you talk | \nChatGPT, voice calls, Slack, Microsoft Teams | \nDesktop app, iPhone app, group chats | \n
The headline idea is now the same on both sides.
\n\nThis is the part that matters most.
\nDots start with one. At launch you get one primary dot, named and personalized.
\nOpenAI says you will be able to add more dots later, and it is piloting \"specialist dots\" for organizations with their own identity and access. But today, for most people, it is one agent doing everything.
\nGrok Bot starts with a team. You create as many Bots as you have areas of work. Each one gets a name, one job and its own memory.
\nBots can message each other, share context in group chats and hand work over, so you are not the router between them.
\nOne agent that does everything slowly becomes a catch-all. Its memory mixes your sales notes with your health plan with your client work.
\nA team lets you keep each area small and focused. In a team setup, the bot that runs your website has no reason to see your finance inbox, and the bot that plans travel has no reason to touch your blog.
\nThat separation is the whole point.
\nMy own week runs like this: one bot for my site, one for social, one for my inbox, and none of them steps into the others' work.
\nWith one dot, there is nothing to design. With a team, you have to decide your areas and write down who owns what.
\nThat takes an afternoon, not a weekend, but it is real work.
\n\nSpend one afternoon listing your areas of work and writing down which bot owns each one.\n\nBoth products take safety seriously, but they put the control in different places.
\nDots use Custom Rules. Each rule describes an action and sets whether the dot can do it alone, must ask first, or must never do it.
\nUnder that, an Auto-review layer checks actions that could affect your accounts or share information. Some tasks, like changing a password, always stay with you no matter what.
\nGrok Bot uses scopes plus approvals. You control what each Bot can reach by which connectors and logins you give it.
\nOutgoing messages start as drafts you send yourself, and risky actions go through a safety check. Some steps, like a sign-in or a payment, are always handed back to you.
\nEither way, an agent can only reach what you connect to it. In Grok Bot, each bot gets only the connectors you give it.
\n\nBefore you connect anything, list the connectors and logins each agent really needs, and give it only those.\n\nThis is where the \"business in a monorepo\" crowd will feel at home.
\nGrok Bot has two named building blocks:
\nYou can also show a Bot a workflow once and let it save the steps.
\nDots work toward goals you set and keep going in the background, and they learn from your feedback. OpenAI's launch material talks more about goals and ongoing work than about named, schedulable recipes.
\nIf your work is a set of repeatable weekly tasks, explicit routines are easier to inspect and debug. If your work is more open-ended, a goal-driven agent may need less setup.
\n\nPick one repeatable weekly task, show a Bot the workflow once, and let it save the steps as a skill.\n\nDots also have an option to connect to your own laptop, which is off by default.
\nDots are rolling out to ChatGPT Pro and Business Premium in eligible markets, with the first dot included in the plan.
\nAt launch, Pro in the EEA, Switzerland and the UK is not included, and Enterprise gets a beta that an admin must turn on. Pricing for extra dots has not been published yet.
\nGrok Bot plans are listed on the Grok Bot page.
\n\nCheck both pricing pages before you decide, because both products are moving fast.\n\nWould you rather have one agent that knows everything about you, or a team where each one only knows its own job?
\nBy the time you know what actually needs you today, an hour is gone and your head is already tired. One assistant per area of work fixes most of that.
\n\n- Split your work into areas and give each one its own assistant, plus a hub that routes requests.\n- Use permissions, not prompts, to keep information apart.\n- Drafts before sends, approval before posting, and quiet unless something needs you.\n\nMost mornings used to start the same way.
\n\nOpen the inbox. Open the team chat. Open the other team chat. Check the calendar, the repo, the tickets, the notes app.
\nEach area assistant watches its own inbox, sorts it and drafts replies. You only hear about it when something needs you.
\nThe problem is not that you have too many tools. The problem is that the context lives in all of them, and you are the only thing connecting them.
\nEvery switch costs focus, and things slip through the gaps between apps.
\nFor the last few weeks I have been running my work through Grok Bot with a setup that fixes most of this. Nothing here depends on what kind of business you run. If you have more than one area of work, you can copy it.
\nDo not build one giant assistant that knows everything. Split your work into areas, and give each area its own assistant.
\nAn area is anything with its own inbox, its own people, or its own rules. For most people it looks something like this:
\nOn top of these, add one hub assistant. The hub does no area work itself. It takes a request, decides which assistant owns it, and hands it over.
\nWhen you are not sure where something belongs, you tell the hub and move on.
\nBecause each assistant stays small. It has a short list of tools, a clear job, and a memory that is only about its own area.
\nIt gets better at that job faster, and it is much harder for it to mix things up.
\n\nName your areas. Anything with its own inbox, its own people, or its own rules is an area.\n\nThis is the most important rule, so it gets its own section.
\nEach assistant connects only to the accounts its area needs. The content assistant gets the website repo and the design tool. The finance assistant gets the billing inbox. The delivery assistant gets the project tools.
\nNone of them get everything. This does two things for you:
\nYou are not trusting a prompt to keep secrets. You are using permissions.
\nIf parts of your work must never mix, this matters even more. Give each one its own accounts and keep them fully separate.
\n\nFor each assistant, list the accounts its area needs and connect only those. Start with read access where you can.\n\nAI assistants are fast. That is the point, and also the risk. A few simple rules keep speed from turning into regret.
\nWrite the scopes down. Keep one short file that says what each assistant owns, in one line each.
\nWhen an assistant gets a request outside its scope, it hands it to the owner or the hub and says nothing more. No advice outside its lane.
\nDrafts before sends. Anything that goes out under your name, like an email, a message or a reply, starts as a draft you review. You press send. The only exceptions are the ones you set on purpose.
\nApproval before posting. Public content gets your OK before it goes live. Blog posts, social posts, anything with your name on it.
\nQuiet by default. Assistants only ping you when something needs a decision, a deadline is at risk, or something is blocked.
\nNo \"all clear\" messages, no empty digests. Silence means things are fine.
\nThese sound strict, and they are. Strict rules are what let you hand over more work without worrying about it.
\n\nWrite the scope file: one line per assistant saying what it owns. Share it with all of them.\n\nOnce the setup is running, the day changes shape.
\nInbox checks run on their own. Each area assistant watches its own inbox on a schedule. It sorts, summarizes, and drafts replies. You only hear about it when something needs you.
\nRoutines handle the repeat work. Weekly reports, content schedules, renewal reminders, the Monday \"what is open\" check.
\nYou write the routine once, in plain words, and it runs on its schedule. If a routine has nothing to report, it stays quiet.
\nRepeat tasks become saved skills. When you notice you are explaining the same steps a second time, save them as a skill. Next time, any assistant can follow the recipe without you walking it through again.
\nAssistants talk to each other. When the content assistant ships a new blog post, it tells the social assistant, which drafts the posts for your review. You do not carry the message between them.
\nYour job shifts from doing every step to reviewing and deciding. That is where your time is worth the most anyway.
\n\nThe next time you explain the same steps a second time, save them as a skill.\n\nYou do not need to build everything at once. In fact, please do not.
\nThe order matters. People who start with ten routines and full access usually end up turning it all off after a bad week. People who start small keep going.
\n\nAdd one routine. Watch it run for a week before you add the next.\n\n| Mistake | \nWhy it hurts | \nDo this instead | \n
|---|---|---|
| One assistant for everything | \nIt gets confused, and one wrong connection can expose all your work. | \nOne assistant per area. | \n
| Too much access too early | \nYou hand over send or delete before trust is earned. | \nGive read access first. | \n
| Noisy assistants | \nIf every assistant pings you every hour, you have just built a new inbox. | \nSet \"quiet unless needed\" from day one. | \n
| No written scopes | \nAssistants drift into each other's work, and you get duplicate or conflicting actions. | \nWrite the scope file. | \n
| Skipping review | \nThe draft step feels slow at first. | \nKeep it. It is still faster than fixing a message you cannot take back. | \n
What is the one task you would hand over tomorrow if you knew it could not touch anything else?
\nAgents are optimistic. They call a tool, see a JSON blob that looks success-shaped, and tell the user \"refund sent.\"
\n\n- Refunds, inventory writes, spend and credential changes need a human or a policy gate.\n- Approval ids come from your system, bound to actor, tenant, action and expiry. Never from the model.\n- After every mutation, a second read proves the world moved before anyone says \"done.\"\n\nSometimes the refund is pending. Sometimes the tool returned a structured error the model ignored. Sometimes the write never happened.
\nHuman-in-the-loop (HITL) gates and post-mutation verification are how you stop early victory. The model can propose. Policy decides. The system checks reality before anyone celebrates.
\nThis is the mutation half of MCP first: tools without judgment burn money.
\nPair with idempotency, structured tool errors, auth and audit, and commerce paths in secure agentic commerce.
\nNot every tool needs a human. Read-only lookups should be fast. Mutations that move money, stock, or trust should slow down.
\n| Risk class | \nExamples | \nDefault gate | \n
|---|---|---|
| Money out | \nRefunds, payouts, store credit | \nApprove above threshold; auto only inside policy | \n
| Inventory | \nForce adjust, unpublish, bulk price | \nApprove or dual-control for large deltas | \n
| Spend / buy | \nAgent checkout complete, ad spend | \nBuyer confirm or spend ceiling | \n
| Credentials | \nRotate keys, invite admins | \nAlways human for production | \n
| Irreversible comms | \nMass email, legal notices | \nApprove + preview | \n
Thresholds are product decisions. A small shipping adjustment is not the same as a large refund. Encode ceilings in policy config, not prompt text.
\nFor Shopify-style claims and refunds, the same idea shows up as queue triage with human resolution (claims pipeline). Auto-clear the easy half; keep edge cases for people.
\n\nList every mutation tool you expose and tag each one with a risk class from the table above.\n\nThere are four shapes that hold up in production:
\nproposed_action payload; UI or Slack asks a human to approve; only then does execute_refund run with the approval id.draft_refund (safe) then commit_refund (requires approval_token).pending record; worker waits for approve/deny; agent polls status.Always bind approvals to:
\nDo not let the model invent an approval_id. Issue it from your system after a real human (or policy engine) decision.
The failure mode is consistent across stacks:
\nVerification is a second read (or event) that proves the world moved:
\ncompleted not open.Put verification in code the agent must call (or that your tool runner calls automatically), not only in a skill paragraph.
\nSkills help sequencing; code enforces it.
\nPattern for a tool runner:
\nauthorize -> idempotency begin -> mutate -> verify -> audit -> respond\nIf verify fails, return a recoverable structured error (verification_failed) with what was expected vs observed. Do not return a soft \"ok\" with a warning buried in prose.
complete_checkout already sits behind trust tiers and permissions (agent profiles, buyer-linked tokens). Still verify order state before telling the buyer \"paid.\"Stateless servers (stateless MCP) make this cleaner: approval records live in your DB, not in a vanished MCP session.
\napproval_id (system-issued) on high-risk commits.When your agent last said \"all done,\" did the ledger agree within a second, or did a human discover the gap in a support ticket two days later?
\nAgent OAuth fails in boring ways: missing PKCE, wrong APP_URL, a new dynamic client on every redirect, a token that has scopes but no tenant.
Laravel MCP 1.0 makes two of those failures explicit. As of the 1.0 upgrade guide and Laravel MCP auth docs:
\nOAuthClient::redirect() throws if the authorization server metadata omits code_challenge_methods_supported (and S256 is the expected method).client_id can be an HTTPS URL to a JSON document your app hosts.This post is the multi-tenant SaaS reading of those rules: how to register clients, where tenant binding belongs, and the auth failures agents hit in production.
\nPair with stateless MCP servers, MCP auth and audit, and Laravel multi-tenancy.
\nPreviously, some servers omitted code_challenge_methods_supported and clients proceeded anyway. Laravel MCP 1.0 rejects that.
If you operate the authorization server (Passport / your IdP), publish:
\n\"code_challenge_methods_supported\": [\"S256\"]\nin /.well-known/oauth-authorization-server.
If you are the MCP client connecting to a third party that will not advertise PKCE, you need another grant that fits (for example pre-issued credentials where appropriate). There is no \"turn off PKCE\" flag in redirect().
For public agents and desktop hosts, PKCE is how you survive without embedding a client secret. Treat missing PKCE metadata as a hard misconfiguration, not a soft warning.
\n\nOpen your `/.well-known/oauth-authorization-server` in a browser and confirm it lists `S256` under `code_challenge_methods_supported`.\n\nMCP revision 2026-07-28 deprecates DCR in favor of metadata documents. Flow in Laravel:
Mcp::oAuthRoutesFor('github', $handler) registers connect, callback, and GET /mcp/oauth/{client}/client-metadata.jsonclient_id_metadata_document_supported, Laravel can use that document URL as client_idtoken_endpoint_auth_method is none, and $token->clientSecret is nullclient_secret must be nullableAPP_URL must be correct in production; the document is built from it, not from the incoming Host headerCustom path / extra fields:
\nMcp::oAuthRoutesFor(\n 'github',\n $handler,\n clientMetadataUri: 'oauth/github/metadata.json',\n clientMetadata: [\n 'client_name' => 'Acme Dashboard',\n 'logo_uri' => 'https://acme.com/logo.png',\n ],\n);\nThis also fixed a nasty 0.x footgun: calling redirect() repeatedly could DCR a new client every time. Metadata documents stop that churn.
Scopes alone do not name a tenant. A token with mcp:use (or your custom scopes) that is not bound to workspace acme-eu will happily call tools until your handler guesses wrong.
Pattern that holds up:
\ntenant_id / workspace_id on the authorization code or token record you issue.user_id + tenant_id + tool + idempotency_key.If you connect outbound as an MCP client to GitHub-like servers per tenant, store the TokenSet per user_id + tenant_id + mcp_client_name. Do not share one refresh across workspaces.
Resource indicators / audience checks (RFC 8707 style) matter when the same IdP protects multiple APIs. Validate that the access token was minted for your MCP resource, not for an unrelated API.
\nResolve tenant from the token, not from a free-form argument the model invents.
\n| Failure | \nSymptom | \nFix | \n
|---|---|---|
| No PKCE advertisement | \nOAuthException on redirect | \nPublish code_challenge_methods_supported: [\"S256\"] | \n
Wrong APP_URL | \nMetadata client_id / redirect mismatch | \nFix env; redeploy metadata | \n
| DCR every connect | \nOrphan clients on IdP | \nPrefer metadata documents | \n
| Null secret surprise | \nInsert fails / refresh breaks | \nNullable columns; auth method none | \n
| Scope without tenant | \nCross-tenant tool calls | \nBind tenant at consent; enforce in tools | \n
| Legacy tests | \n400 / -32020 | \nSend MCP headers + _meta on 2026-07-28 | \n
Agents will narrate these as \"the tool is broken.\" Your job is to make the HTTP and OAuth errors structured enough that the model (or your skill) can tell the human to reconnect.
\nweb (or your chosen) middleware.client_secret storage nullable.If an agent can call refund_order with a token from workspace A while the chat is about workspace B, where did tenant binding fail: consent, token storage, or the tool handler?
Laravel MCP 1.0 leans into protocol revision 2026-07-28: every request stands alone.
The MCP-Session-Id header is gone. Request::sessionId(), setSessionId(), and SessionInitialized are gone. If your tools cached \"conversation state\" under a session id, that design does not survive the upgrade.
That sounds scary until you remember how you already scale Laravel: Octane workers, queues, and explicit keys. Stateless MCP is the same idea with stricter honesty.
\nAs of the 1.0 upgrade guide and Laravel MCP docs, modern requests carry protocol version and client capabilities in params._meta. Older initialize clients still negotiate, but new work should assume no server-side MCP session.
You lose a free place to stash:
\nYou do not lose:
\nIf a feature needed a session, give it a first-class id the client sends (workspace id, run id, approval id) in tool arguments or _meta.
HTTP MCP endpoints are ordinary Laravel routes (Mcp::web). Octane helps when you want warm workers and lower boot cost for chatty agents. Rules that already apply to Octane still apply:
job_id / status URL immediately.get_job_status reads that store.That is more reliable than holding an SSE stream open for ten minutes of report generation, and it survives worker restarts. Pair with Laravel Octane high traffic notes if you are tuning the runtime.
\nStreaming tools (generators + SSE on web servers) still work for progress notifications. Treat them as transport, not as durable session state.
\n\nTake your slowest tool, make it dispatch a job and return a `job_id`, and add a `get_job_status` tool that reads progress.\n\nEvery mutation tool should already:
\nauth:api / Sanctum / your middleware).Without MCP sessions, those five are not optional polish. They are how two Octane workers and a retried agent call stay coherent. See MCP idempotency for Laravel mutations.
\nFor protocol 2026-07-28 HTTP clients, also send matching MCP-Protocol-Version, Mcp-Method, and (for calls) Mcp-Name headers. Mismatches return HTTP 400 with JSON-RPC -32020. Update feature tests that used bare postJson.
Pass your own identifiers:
\n{\n \"params\": {\n \"name\": \"refund-order\",\n \"arguments\": {\n \"order_id\": \"gid://...\",\n \"idempotency_key\": \"8f3c...\",\n \"_meta\": {\n \"io.modelcontextprotocol/protocolVersion\": \"2026-07-28\",\n \"app.correlation_id\": \"run_01J...\"\n }\n }\n }\n}\nLog app.correlation_id (or your chosen key) beside tenant and user ids. Agents can resume by asking for status with that id. Humans can grep it in observability.
Searchable catalogs and cache hints from Laravel MCP 1.0 (searchable tool catalogs) also assume list/read responses may be cached while tool calls stay uncacheable.
\nStateless fits that model: discovery can be hot; mutations stay cold and keyed.
\nsessionId usage and SessionInitialized listeners._meta + MCP headers on 2026-07-28 clients.Stateless MCP does not mean \"forget the user.\" It means the MCP transport does not own a sticky session bag between JSON-RPC messages.
\n\nYour product still has:
\nThose live in application storage. The MCP server becomes a thin, horizontally scalable door into that storage. That is the same split you want for multi-tenant Laravel APIs.
\nThe request carries identity; the database carries history.
\nWhen an agent \"resumes\" work, it should call a tool with the ids you issued earlier (run_id, draft_id, approval_id), not hope the same load balancer owns an in-memory map from last Tuesday.
If you deleted MCP-Session-Id tomorrow, which of your tools would still be safe under two parallel Octane workers retrying the same agent turn?
A Token-tier agent credential identifies your app. A buyer-linked token also carries the signed-in Shop customer.
\n\n- A buyer-linked token turns \"an agent is shopping\" into \"this buyer is shopping through an agent.\"\n- You get it in three server-side hops: Shop OAuth, token exchange for a JWT grant, then a redeem at Shopify.\n- The grant lives about 60 seconds. The token lives about 60 minutes and never refreshes, so plan to remint.\n\nAs of Sep 2026, Shopify documents a UCP delegated identity provider flow: authorize with Shop (accounts.shop.app), exchange the Shop access token for a short-lived JWT grant, redeem that grant at Shopify for a buyer-linked JWT.
The resulting token lasts about 60 minutes and does not refresh. When it expires, remint from the Shop session.
\nOfficial reference: Create a buyer-linked token. Pair with UCP agent profiles, auth and rate limiting, and secure agentic commerce.
\nWith a buyer-linked token you can:
\ndev.ucp.shopping.checkout:manage)Without it, you may still build carts anonymously or with an app-only token, but you miss buyer-aware behavior and you still need the right permission story for complete_checkout.
Think of it as identity linking for agentic checkout, not as a forever API key.
\nConfidential client secret stays on a server you control. Public clients cannot complete this flow safely. Do not ship the secret in browser or mobile binaries.
\nDiscover endpoints at runtime (do not hard-code forever):
\n/.well-known/oauth-protected-resource → Shopify authorization serverdev.ucp.common.identity_linking → Shop as delegated IdP/.well-known/oauth-authorization-server → authorize + token endpointsThen run the three steps below.
\nStandard OAuth authorization code against Shop. Reuse Dev Dashboard client id/secret and redirect URI.
\nRequest every scope you plan to redeem later (include checkout manage if you will call checkout tools).
\nYou receive a Shop access_token (example responses show expires_in: 3600).
Token exchange (RFC 8693) at Shop's token endpoint:
\ngrant_type=urn:ietf:params:oauth:grant-type:token-exchangesubject_token = Shop access tokenrequested_token_type = JWTaudience = api.shopify.com (Global Catalog) or the merchant store domain for store checkoutThe returned access_token is a JWT authorization grant: signed, single-use, about 60 seconds to live. It is not your API bearer yet. To inspect its claims while you debug, paste a test token into my JWT decoder.
JWT bearer assertion (RFC 7523) at Shopify's (or the store's) token endpoint with the scopes you need. Shopify verifies the grant against Shop's keys, resolves the customer, and issues the buyer-linked JWT.
\nFor merchant checkout, discovery points at the store's authorization server. Set audience to the store domain and request scopes such as openid and dev.ucp.shopping.checkout:manage as documented for signed-in checkout tools.
Two clocks, often confused:
\n| Artifact | \nLifetime (documented) | \nRefresh? | \n
|---|---|---|
| JWT authorization grant from Shop | \n~60 seconds, single-use | \nNo; re-exchange if needed | \n
| Buyer-linked token from Shopify | \n~60 minutes | \nNo refresh token; remint via the chain | \n
Build session handling around reminting, not around refresh_token muscle memory from classic OAuth apps. Cache the buyer-linked JWT only within its lifetime, keyed to the buyer and resource server, on the server.
Build session handling around reminting, not around refresh_token muscle memory.
\nIf complete_checkout still fails with a Token-tier client, check purchase permission and merchant/agent eligibility separately. Community threads show scope alone is not always sufficient; treat platform policy as part of the design.
audience (global vs store).Log correlation ids and HTTP status from each hop. Do not ask the model to \"retry checkout\" when the token is simply dead.
\n\n- Buyer-linked tokens bind a Shop customer to your agent credential for personalized and signed-in commerce.\n- Flow: Shop OAuth, then token exchange (JWT grant), then Shopify JWT bearer redeem.\n- Run exchange and redeem only on a confidential server.\n- Grant ~60s single-use; buyer-linked token ~60 minutes with **no refresh**.\n- Discover endpoints; set audience to global catalog or merchant store deliberately.\n- Remint from the Shop session; design UX for re-link, not silent forever tokens.\n\nWhen your agent's checkout fails at minute 61, is the model confused about tools, or did you forget that buyer-linked JWTs die without a refresh token?
\nIn the OAuth decade you registered an app and got a client id. In agentic commerce you host a UCP agent profile: a public JSON document at a URL you control.
\n\n- Every Shopify Cart MCP and Checkout MCP call asks for your agent profile URL.\n- Declare cart and checkout together if you want to turn a cart into a checkout.\n- Anonymous, Signed and Token trust tiers decide your rate limits and whether `complete_checkout` is possible.\n\n\nEvery Cart MCP and Checkout MCP call on Shopify asks for that URL in meta.ucp-agent.profile. The merchant fetches it, negotiates capabilities, and (for signed traffic) checks your published keys.
No profile, no serious cart/checkout conversation.
\nThis post covers what to put in the profile, why cart and checkout belong together, and how Anonymous / Signed / Token tiers change what you can do.
\nCross-read Shopify UCP quick-start, secure agentic commerce, and the Storefront MCP cart → UCP migration.
\nOfficial guides: Define your agent profile and Auth and rate limiting.
\nVersion pins below are as of Sep 2026; check them against the current docs.
\nAn agent profile is a JSON document that declares:
\n2026-08-25)Minimal cart + checkout profile (from Shopify's published examples):
\n{\n \"ucp\": {\n \"version\": \"2026-08-25\",\n \"capabilities\": {\n \"dev.ucp.shopping.cart\": [{ \"version\": \"2026-08-25\" }],\n \"dev.ucp.shopping.checkout\": [{ \"version\": \"2026-08-25\" }]\n }\n }\n}\nHost it somewhere stable and HTTPS:
\nhttps://your-app.example/profiles/agent.json)Then send that URL on every MCP cart/checkout request. Merchants treat it like app metadata: fetch, cache per directives, negotiate intersection of capabilities.
\nFor signed requests, the profile also carries the public key material used to verify HTTP Message Signatures (RFC 9421 / ECDSA P-256 per UCP). That is the \"app credentials\" half of the story without a browser consent screen on every call.
\n\nPublish the minimal profile above on a stable HTTPS URL, then fetch it with `curl` from outside your network to confirm a merchant could read it.\n\nPassing a cart_id into create_checkout requires both capabilities to negotiate successfully. A cart-only profile is fine for browse-and-share (continue_url). It is not enough for the purchase handoff.
Declare what you actually implement. Inflating capabilities you cannot fulfill is how you fail negotiation or fail later in checkout.
\nUnder-declaring is how you debug \"why won't create_checkout accept my cart\" for an afternoon.
\nKeep exploratory line-item work on Cart MCP; open Checkout MCP when the buyer is ready. Rate limits are stricter on checkout at every trust tier.
\nShopify classifies UCP MCP traffic by how the agent identifies itself (as of the Sep 2026 auth docs):
\n| Tier | \nHow you identify | \nCatalog / cart / checkout build | \ncomplete_checkout | \nRelative limits | \n
|---|---|---|---|---|
| Anonymous | \nNo auth or signature | \nYes | \nNo | \nLowest | \n
| Signed | \nHTTP Message Signatures; key in profile | \nYes | \nNo | \nMid | \n
| Token | \nBearer JWT (Dev Dashboard / customer / shop / buyer-linked) | \nYes (buyer-linked can personalize) | \nWhen purchase permission is granted | \nHighest | \n
Order tools (get_order) sit on the Token tier with the right scopes, and only for orders placed through your agent.
Climb tiers on purpose. Do not ask for complete_checkout from an anonymous client and treat the 401 as a model failure.
meta.ucp-agent.profile on every cart/checkout tool call.Think of the profile as your OAuth app listing plus capability manifest.
\nMerchants will judge your profile by what it claims and what your traffic tier can actually do.
\nIf you also host your own MCP servers (Laravel or otherwise), you already know the pain of fat tool lists and weak auth.
\nUCP profiles are the commerce-protocol cousin of that discipline: advertise operable capabilities, then back them with auth that matches risk.
\nSee MCP first and API vs MCP for the product-side framing; this post is the Shopify UCP instantiation.
\n\n- The UCP agent profile URL is how merchants negotiate who you are and what you can do.\n- Declare `dev.ucp.shopping.cart` and `dev.ucp.shopping.checkout` together when you convert carts.\n- Anonymous, Signed, and Token tiers gate rate limits and whether `complete_checkout` is even on the table.\n- Signed proves agent identity via profile keys; Token is required for purchase completion permissions.\n- Keep the profile stable, versioned, and honest. Capability theater fails in negotiation or at checkout.\n\nIf you shipped a shopping agent tomorrow, would your public profile URL survive a merchant fetch right now, and which trust tier are you actually operating on when you say \"we support checkout\"?
\nIf your \"catalog AI\" is a PDF tutorial wearing a Shopify hoodie, you will ship confident answers that recommend out-of-stock, low-margin, or dead SKUs.
\n\n- Document RAG optimizes for passage relevance. Commerce RAG has to optimize for sellable, profitable products.\n- Retrieve with hybrid lexical plus vector search, then rank on live stock, velocity, margin and merchandising rules.\n- Keep price and inventory out of the embedding. Read them live at query time.\n\nDocument RAG optimizes for passage relevance. Commerce RAG optimizes for sellable, profitable, in-policy products. Same family of techniques. Different system design.
\nThis post is the cut line for Shopify Plus (and any large catalog) work: hybrid lexical + vector retrieval, then a commerce ranking layer that uses live stock, velocity, margin, and merchandising rules.
\nPair it with why your RAG is failing, 7 RAG mistakes in production, and agent-facing catalog tools via Shopify UCP quick-start.
\nClassic demo loop:
\nThat works when the truth is mostly static prose. Policies, runbooks, warranty text.
\nIt fails when the \"document\" is a product graph whose price, inventory, and variants change every minute.
\nA product is not a paragraph.
\nIt is title, options, collections, metafields, media, and a set of volatile facts that should never be baked into the only retrieval score.
\nFor storefront search, chat commerce, and agent tools (including UCP search_catalog / get_product on /api/ucp/mcp), the job is closer to:
Shopify's Storefront Catalog MCP gives agents a search surface. It does not replace your ranking philosophy for a custom Plus app, Hydrogen search, or internal shopping agent.
\nTreat MCP as a channel. Own retrieval quality behind it when you build custom.
\nPure vector search misses exact SKUs, model numbers, and brand strings the embedding smoothed away. Pure BM25 / keyword search misses \"shoes for wide feet on wet rock\" when the title says \"trail runner.\"
\nHybrid retrieval runs both and fuses ranks (reciprocal rank fusion or a tuned weighted blend). That is the same medicine as general production RAG (why your RAG is failing), applied to product text fields: title, description, tags, option names, curated metafields.
\nPractical embedding content:
\nIf you are on pgvector in Laravel, the hybrid pattern still holds: SQL tsvector / external search engine for lexical, ANN for semantic, fuse, then rank.
Stack choice is secondary to the split. See picking the right RAG stack when you are still selecting stores.
\n\nSearch your catalog assistant for an exact SKU and for a plain-language need. If one of them fails, you need the other half of hybrid retrieval.\n\nRetrieval returns a candidate set. Commerce ranking decides what a buyer or agent should see first.
\nSignals that belong after hybrid retrieve (illustrative, tune per vertical):
\n| Signal | \nWhy it matters | \n
|---|---|
| In-stock / location available | \nSemantic winners that cannot ship are false friends | \n
| Sales velocity (7/30-day) | \nPreference for what actually converts | \n
| Margin / contribution | \nAmbiguous queries should not default to junk economics | \n
| Return rate | \nHigh semantic fit with high returns hurts trust | \n
| Campaign pins / suppressions | \nMerchandising is policy, not cosine | \n
| Newness / overstock | \nLaunch windows and aging inventory need soft boosts | \n
A product that is \"perfectly similar\" and converts poorly is a bad recommendation. The embedding does not know that unless you engineer the signal into ranking (or into a learned reranker that consumes those features).
\nDo not pretend one cosine score encodes margin. It does not.
\nCommon failure mode: concatenate \"price: $49 in stock: 3 margin: high\" into the embedding text. Overnight a sale and a sell-through change the truth; the vector still argues yesterday.
Better split:
\nInventory especially: listen to inventory webhooks (or poll InventoryLevel), keep a KV / Redis / table of live quantities, filter or demote after retrieval. Regenerating embeddings on every stock change is expensive and still races reality.
\nThis is the ecommerce version of \"don't RAG what should be an API call.\" Policies and size guides can live in document RAG. \"How many left in EU warehouse?\" should not.
\n\nGrep your embedding text builder for price, stock and margin fields, and move each one to a live read.\n\nA human scanning a PLP can mentally skip sold-out cards. An agent that trusts top-k embeddings will propose them unless you enforce availability and policy in the tool layer.
\nFor agentic commerce:
\nget_product (or Admin/Storefront live reads) before stating price or stock.available when the Shopify catalog extension is in play; still verify before checkout.Agent channels (UCP / MCP) amplify whatever retrieval quality you already have. They do not invent merchandising judgment.
\nYou do not need a research lab to start. You need the humility to stop treating SKUs like PDF chunks.
\n\n- Document RAG optimizes passage relevance; commerce RAG must optimize sellable outcomes.\n- Hybrid lexical + vector retrieval catches SKUs and semantic intent; neither alone is enough at catalog scale.\n- Ranking on stock, velocity, margin, and merchandising rules comes **after** retrieval.\n- Keep price and inventory out of embeddings; read them live.\n- Agents will confidently recommend bad stock unless tools and filters enforce reality.\n- Evaluate with commerce outcomes, not only embedding similarity demos.\n\nIf your catalog assistant still chunks product CSVs like a handbook, which false recommendation would hurt more this week: an out-of-stock hero SKU, or an in-stock low-margin substitute that crowds out the product you actually wanted to sell?
\nEvery tool you advertise to a model costs tokens, up front.
\n\n- A fat `tools/list` taxes the context window. Keep a few hot tools and park the long tail behind `ToolSearch`.\n- Cache hints cut how often clients refetch discovery and resources. Tool calls are never cached.\n- Default to private cache scope for tenant-filtered catalogs, and keep auth and idempotency on every mutation.\n\nNames, descriptions, input schemas, output schemas. A SaaS MCP server with forty admin actions can burn a surprising slice of the context window before the user asks a single question.
\nLaravel MCP 1.0 (as of the stable release covering protocol revision 2026-07-28) gives you two levers for that problem: searchable tool catalogs and cache hints. One shrinks what the model sees. The other reduces how often clients re-fetch discovery and resource payloads.
This post is the practical split: which tools stay in the hot list, which go behind ToolSearch, and how to set ttlMs / scope without lying to shared caches.
Pair it with the MCP first framing and the Laravel MCP safety trilogy when you wire real mutations. Official docs: Laravel MCP (Searchable Tool Catalogs and Cache Hints sections).
\nWhen a client calls tools/list, every returned tool definition becomes candidate context for the model. That is fine for five crisp tools. It hurts when you expose:
close_year_end_books)The agent does not need those definitions on turn one. It needs a short path to find them when the user asks.
\nLaravel MCP's answer is ToolSearch: keep hot tools in $tools as normal classes, and park the long tail under a ToolSearch::class => [...] entry.
As documented, a searchable catalog registers two meta-tools for the client:
\nsearch_tools: query by name, description, and input schema; returns matches with names, descriptions, and expected inputs (respect a result limit)execute_tools: invoke one or more tools by name from what search (or prior knowledge) surfacedMinimal server shape:
\nuse Laravel\\Mcp\\Server;\nuse Laravel\\Mcp\\Server\\Tools\\ToolSearch;\n\nclass OpsServer extends Server\n{\n protected array $tools = [\n // Always advertised...\n CreateInvoiceTool::class,\n ListDelayedOrdersTool::class,\n\n // Discovered on demand...\n ToolSearch::class => [\n HistoricalAuditTool::class,\n RotateApiKeyTool::class,\n ExportComplianceCsvTool::class,\n ],\n ];\n}\nCurrentWeatherTool-style examples in the docs make the same point: advertise the common case; search for historical weather and alerts.
Limits for batched execution live in config:
\nmcp.tool_search.max_tool_callsmcp.tool_search.max_output_bytesTune those before an agent can fan out ten heavy tools in one execute_tools call.
Conditional registration still applies. If a catalog tool's shouldRegister returns false, it will not appear in search results and cannot be executed. That matters for plan-gated or tenant-gated features.
Use this as a decision table, not a vibe:
\n| Put it in the hot list when... | \nPut it behind ToolSearch when... | \n
|---|---|
| It is the main outcome users ask for | \nIt is long-tail or seasonal | \n
| Search wording is ambiguous (\"fix order\" could mean five tools) | \nThe name and description search cleanly | \n
| You need zero discovery latency | \nAn extra search turn is acceptable | \n
| It is the safe entry point into a skill | \nIt is a deep follow-up after triage | \n
| Schema is small and stable | \nSchema is large or rarely needed | \n
Examples from a multi-tenant ops server:
\nlist_delayed_orders, get_order, draft_customer_messageissue_partial_refund (after policy skill), export_tax_csv, rotate_webhook_secretExposing every mutation \"so the model can see it\" is how you get eager refunds. Discoverability plus a skill that says when to search is safer.
\nTools without judgment burn money.
\nThat is the same discipline as MCP first.
\nIf your agent already struggles with wrong tool choice, searchable catalogs alone will not fix bad descriptions. Write #[Description] like a product brief. Ambiguous copy poisons both the hot list and search rankings.
Laravel MCP attaches advisory cache hints to responses that may be cached: server discovery, primitive listings, and resource reads. Defaults are conservative: private scope, ttlMs of zero (do not cache).
You can set a server default with #[Cacheable] and override per method via cacheHints():
use Laravel\\Mcp\\Enums\\CacheScope;\nuse Laravel\\Mcp\\Server\\Attributes\\Cacheable;\n\n#[Cacheable(ttlMs: 60_000, scope: CacheScope::Public)]\nclass OpsServer extends Server\n{\n protected function cacheHints(): array\n {\n return [\n 'tools/list' => new Cacheable(\n ttlMs: 30_000,\n scope: CacheScope::Public,\n ),\n ];\n }\n}\nFrom the docs and MCP caching draft:
\nCacheScope::Public: safe to share across users (same tool list for everyone)CacheScope::Private: tied to the authorization context (filtered lists, user-specific resources)#[Cacheable] wins over method and server defaultsttlMs means do not cacheOn the client side, Laravel's MCP client honors hints when you opt in with withCache(). Installing 1.0 does not enable caching by itself.
That is good: accidental public caching of a private tools/list is a tenancy bug waiting to happen.
For multi-tenant SaaS, default private unless you are sure every authenticated caller sees the same primitives. If plan features change the tool list per shop, public scope is wrong even if the HTTP route is authenticated.
\n\nCall `tools/list` as two users on different plans. If the lists differ, make sure its cache scope is private.\n\nHere is a pattern that holds up on real products:
\nToolSearch, with descriptions that include synonyms users actually say.search_tools before inventing a workflow.tools/list only when the catalog is identical for all callers.Searchable catalogs reduce tokens. They do not replace permission checks. An agent that finds rotate_api_key via search still needs the same actor, tenant, and audit line as a hot tool.
If you are connecting agents as MCP clients rather than only hosting servers, the Laravel AI SDK path of spreading Client::web(...)->tools() into an agent still benefits: fewer advertised tools means less noise in the agent loop.
Cache the listed collection when it is stable, as the AI SDK docs suggest for remote OAuth servers.
\nshouldRegister and authorization still decide availability.Public for a cheap cache hit.Protocol note: Laravel MCP 1.0 speaks 2026-07-28 (including server/discover) while still negotiating older initialize clients. If your host is mid-upgrade, check the Laravel MCP docs for your installed package version.
If you listed every tool your admin UI exposes into MCP tomorrow, which five would earn a permanent slot in the hot list, and which ones should stay invisible until search_tools earns them?
If your shopping agent still calls get_cart and update_cart on https://{shop}.myshopify.com/api/mcp, you are running on deprecated tools.
Shopify announced the deprecation on June 24, 2026. Maintenance for those Storefront MCP cart tools ended on August 31, 2026.
\nAgents that have not moved are now on unmaintained tools, and they can fail mid-conversation while buyers still expect a cart.
\nThe replacement is UCP Cart MCP at https://{shop-domain}/api/ucp/mcp. Same commerce job. Different endpoint, tool set, and request shape.
This post is the migration map: what breaks, what replaces it, and a checklist you can run against any agent still wired to /api/mcp.
For the broader agent-ready storefront story, start with the Shopify UCP quick-start and secure agentic commerce on Shopify.
\nOn Storefront MCP (/api/mcp), cart support was essentially two tools:
get_cartupdate_cartThose tools mixed \"create if missing\" behavior into update, patched line items like a classic AJAX cart, and lived next to policy search on the legacy storefront MCP surface.
\nUCP Cart MCP implements the UCP cart capability (dev.ucp.shopping.cart, version 2026-08-25 in current Shopify docs) and exposes four tools on /api/ucp/mcp:
| Tool | \nJob | \n
|---|---|
create_cart | \nStart a cart with line items and optional buyer context | \n
get_cart | \nRead current cart state and estimated totals | \n
update_cart | \nReplace the full cart state (PUT semantics) | \n
cancel_cart | \nDelete the cart; requires an idempotency key | \n
Catalog discovery (search_catalog, lookup_catalog, get_product) already moved toward the UCP endpoint. Cart is catching up. Treat /api/mcp cart calls as a compatibility shim, not a long-term contract.
Official references: deprecation changelog and Cart MCP docs.
\nPoint every cart tools/call at:
POST https://{shop-domain}/api/ucp/mcp\nCart tools accept unauthenticated requests. That lets an agent estimate totals and share a continue_url before the buyer signs in.
Checkout is a different server with stricter auth and rate limits; keep exploratory work on Cart MCP.
\nEvery cart call must include meta in arguments:
{\n \"meta\": {\n \"ucp-agent\": {\n \"profile\": \"https://your-agent.example/.well-known/ucp\"\n }\n }\n}\nThat profile URI is how the merchant negotiates capabilities. Skipping it is not a soft warning; it is a failed request.
\nHosting that profile is the agent equivalent of registering an OAuth app. Pair this post with the secure agentic commerce guidance when you harden trust tiers.
\nFor get_cart, update_cart, and cancel_cart, pass the cart id as a top-level id in arguments. Do not nest id inside the cart object on write.
Responses land in result.structuredContent. Read the cart object there.
Business problems (expired cart, quantity adjusted) often arrive as a successful JSON-RPC result with entries in cart.messages, not as a protocol error. If your client only checks JSON-RPC errors, you will miss quantity adjustments and not_found.
Legacy update_cart often created a cart when no id was sent. UCP splits that out.
Use create_cart when the buyer has selected variants and you want estimated totals, multi-turn edits, or a shareable continue_url without starting checkout. Include:
cart.line_items[] with quantity and item.id (variant GID, which my Shopify GID decoder can unpack)context (address_country, region, postal code) for pricing and availability hintsattribution (UTM / click ids) if you will later convert to checkoutbuyer for personalized estimatesIf you plan to call Checkout MCP create_checkout with this cart, set attribution on the cart before conversion. The merchant uses the cart's line items, context, buyer, and attribution when creating checkout and ignores overlapping fields in the checkout payload.
When the buyer is ready to pay, pass the cart id as cart_id into create_checkout. Conversion is idempotent for the same cart id.
This is the sharp edge of the migration.
\nupdate_cart on UCP Cart MCP replaces the cart. Omit line_items, context, or attribution and those fields are removed. There is no server-side merge.
That differs from Storefront API and AJAX cart mutations, and from the old Storefront MCP patch-style mental model.
\n\nPractical rules for agent tool wrappers:
\ncart object on every update_cart.attribution and context if you still want them.If you teach the model \"just send the field that changed,\" you will empty carts in production.
\nPut that rule in the tool description and in your skill playbook, not only in a human README.
\ncancel_cart requires meta[\"idempotency-key\"] as a UUID in addition to ucp-agent.profile. Cancel removes the cart from storage. Later get_cart returns a business not_found outcome.
Use cancel when the buyer abandons, or when you need to clear a stale cart before starting clean.
\nRetries without a stable idempotency key are how you get confusing double-cancel paths. The same discipline shows up in MCP idempotency for Laravel mutations even when the server is Shopify's, not yours.
\nCarts are long-lived browsing containers. Checkouts are short-lived purchase sessions with stricter freshness and rate limits.
\n| Use case | \nPrefer | \n
|---|---|
| Multi-turn browsing and total estimates | \nCart MCP | \n
| Share a link with the buyer | \nCart MCP continue_url | \n
| Buyer ready to purchase | \nCheckout MCP | \n
| Complete payment in your app | \nCheckout MCP | \n
Rate limits scale with how the agent identifies itself (Bearer, signed, anonymous). Anonymous Cart MCP works for exploration; Checkout MCP does not. Keep chatty quantity tweaks off checkout tools.
\nThis split matches the MCP vs A2A vs ACP idea: pick the protocol surface that matches the job, not the one your first tutorial used.
\nRun this against every agent, skill, and integration that still mentions Storefront MCP carts:
\n/api/mcp, get_cart, and update_cart in agent configs, MCP client URLs, and tool wrappers./api/ucp/mcp for cart tools.create_cart as an explicit step; stop inventing carts inside update.meta.ucp-agent.profile on every cart call.update_cart (PUT semantics).id for get, update, and cancel.idempotency-key for cancel_cart.structuredContent.cart.messages for business outcomes (not_found, quantity_adjusted, and friends).create_checkout only when the buyer commits.If you are still designing the product around agents rather than bolting MCP onto a human-only flow, the MCP first framing helps: ship the cart tools and skills as the capability contract, then let the chat UI sit on top.
\n\nStart with step 1 today: search your agent configs and tool wrappers for `/api/mcp` and list every hit.\n\n\n- Storefront MCP `get_cart` / `update_cart` on `/api/mcp` are deprecated; maintenance ended August 31, 2026. Migrate now.\n- Migrate cart traffic to UCP Cart MCP at `/api/ucp/mcp` with `create_cart`, `get_cart`, `update_cart`, and `cancel_cart`.\n- Every request needs `meta.ucp-agent.profile`; cancel also needs a UUID idempotency key.\n- `update_cart` is full replacement. Partial payloads wipe omitted fields.\n- Keep browsing on Cart MCP; move to Checkout MCP only when the buyer is ready to pay.\n- Treat `cart.messages` as first-class outcomes, not optional log noise.\n\nIf your agent still hard-codes /api/mcp for carts, which step fails first in staging: missing agent profile, empty cart after a partial update, or a checkout call that never saw a real cart id?
Your returns policy PDF is not a returns process.
\n\n- One pipeline can carry returns, damage claims and warranty, as long as the claim type decides rules, evidence and resolutions.\n- A claim counts as done only after a refund, replacement or credit lands in Shopify with an audit record.\n- Let rules handle the routine claims so your team spends its time on the hard ones.\n\nA customer emails a blurry photo of a cracked box. Support digs through orders to confirm the purchase. Someone promises a refund in Slack.
\nThree days later the same claim is still \"waiting on photos,\" and nobody can say who owns it. That is not a claims system. That is an inbox wearing a policy hat.
\nI built Claimify because I kept seeing the same pattern on Shopify stores: every claim type sprawling into its own spreadsheet, helpdesk macro, or ad-hoc Google Form. Returns in one place. Damage in another. Warranty buried in a shared mailbox.
\nThe fix is not a prettier form. It is one end-to-end pipeline that every claim type walks through, from the moment a customer files to the moment Shopify actually refunds, replaces, or credits.
\nA policy answers what you allow. A process answers how it moves.
\n\nIf you only publish the policy, every exception becomes a negotiation in email. If you only build a form without states, you get a ticket graveyard.
\nThe pipeline is the missing middle: typed claim in, typed resolution out, with explicit transitions in between.
\n\"Claims\" is not a synonym for returns. On a real storefront you usually need at least:
\nOne queue can hold all of them if the type is a field on the claim, not a separate product.
\nClaim type drives which eligibility rules run, which evidence is required, and which resolutions are legal for that SKU. That is the difference between a generic \"support form\" and a claims system.
\n\nList every place a claim can arrive today (email, chat, forms, DMs). Each one is a queue nobody owns.\n\nThe loop should be boring and complete.
\nCustomer side. File from the account page, from an order action, or from a theme app block on a help page. Pick the order line, pick the claim type, describe the issue, attach evidence. No copying order numbers into a contact form that pretend-matches later.
\nMerchant side. The claim opens in Shopify Admin with live order context: line items, fulfilment dates, customer tags, previous claims on the same order.
\nAssign an owner. Move it through a Kanban (new, review, resolve) or list view. Comment with history that does not vanish when someone leaves the company.
\nIntegrations. Status changes should fire webhooks and talk to the tools you already run: Shopify Flow, Gorgias, Klaviyo, Zapier, Make. The helpdesk does not need to be the system of record. It needs to stay in sync with it.
\nIf a claim can die in a thread without a state change, you do not have a pipeline. You have chat.
\nWarranty is where most \"claims apps\" quietly fail. They slap a warranty label on a return flow and call it done.
\nThat breaks the moment you need registration, a serial number, a coverage window that starts at fulfilment (or at first activation), or a different resolution set than a 30-day return.
\nTreat warranty as a product surface:
\nI shipped Claimify with warranty as a first-class path for exactly this reason. The App Store listing is under XCO Apps; the product thesis is the same either way: warranty is not a renamed form.
\nDetails and constraints from the build are on the Claimify work page.
\nA claim is not resolved when someone types \"approved\" in Slack. It is resolved when money or inventory moves in Shopify and the audit row says who did it, when, and why.
\nThat means resolution actions should map to real Admin API outcomes:
\nCopy-pasting order IDs between tabs is how refunds get double-issued and how replacements ship without a paper trail.
\nKeep order context live from the GraphQL Admin API. Do not cache a stale snapshot as the source of truth for money.
\nNot every claim needs a human. Auto-approval and eligibility rules can handle the boring cases:
\nThe point is not to remove judgment. It is to stop burning senior support time on \"yes, this is inside the 14-day window and the tag says returnable.\"
\nHumans stay on the edge cases. Rules handle the rest, and every auto decision still writes an audit event.
\nPublic numbers from the Claimify listing, nothing invented: 5.0 from 6 reviews, plans at $9 / $19 / $49, and a 3-day trial. Useful as a product fact. Useless as a substitute for your own ops metrics.
\n\nPull last month's claims and mark the ones a simple rule (window, tag, photo present) could have approved. That share is your first auto-approval target.\n\nA few constraints that mattered when I built this as an embedded Shopify app (Laravel backend, Remix + App Bridge + Polaris admin, evidence on S3/R2):
\nIf you are scoping a similar public or private app, the same shape shows up on Shopify Plus apps: embedded admin, billing, webhooks, and a real resolution path into Shopify.
\n\n- A policy PDF is not a process. Inbox volume is not a claims system.\n- Returns, damage, and warranty can share one pipeline if claim type drives rules, evidence, and allowed resolutions.\n- Warranty needs registration, policy windows, and serials as first-class concepts, not a renamed return form.\n- Resolution means Shopify money or inventory moved, with an audit row, not a Slack emoji.\n- Auto-approval should clear the easy half of the queue; humans keep the edge cases.\n- Embed in Admin, keep order context live, treat state changes as events, store evidence outside the app process.\n\nIf your claims still live in a shared inbox, which claim type is creating the most silent backlog right now, and what would change if it had to move through one queue with an owner and a Shopify resolution?
\nMobile first made products work on a phone. MCP first makes them work for an AI agent.
\n\n- The hard constraint used to be a small screen. Now it is whether an authorized agent can finish the job.\n- Ship tools, permissions and skills first. The human UI becomes one client of the same capabilities.\n- Start small: one sprint and three outcomes your users already ask for.\n\nA decade ago the argument was settled in the hallway: mobile first. If the product did not work on a phone, it did not work.
\nTeams stopped designing for a 27-inch monitor and shrinking later. They started with the constrained surface and let the desktop inherit the discipline.
\nThat constraint was a screen.
\nToday the constraint is a capability contract. Users still want a UI, but more and more of the real work is asked in chat: Claude, Cursor, Grok Bot, an in-product agent.
\nThey do not want another settings page. They want to say what they need and have it done. Docs, support tickets, and click-paths become the fallback when the product cannot be operated by an agent.
\nSo the slogan flips: MCP first, or more precisely, agent first / skills first. Ship the tools, permissions, and judgment an agent needs before you polish the human interface that wraps them.
\nI am not arguing that MCP replaces your API. I already wrote API vs MCP for that fork.
\nMCP is how agents discover and call capabilities. Skills are how they apply judgment. The UI is how humans supervise. Get the order wrong and you bolt a chatbot onto a product that was never operable. Agents also need a map of what to read, and an llms.txt file at your site root is the simplest one to publish.
\nMCP first does not mean \"add a Model Context Protocol server and call it a day.\"
\nIt means you design the product as a set of outcomes an agent can complete, with:
\nMobile first forced responsive layout. MCP first forces operable capability.
\nIf an agent with the right credentials cannot complete the job end-to-end, the product is not finished, no matter how pretty the dashboard is.
\nEvery serious product now has two customers:
\n| Customer | \nWhat they need | \nFailure mode | \n
|---|---|---|
| Human | \nScreens, copy, empty states, accessibility | \nConfusing UI; they open a ticket | \n
| Agent | \nTools, schemas, errors they can recover from, audit trails | \nSilent failure; retries; invented actions | \n
Most teams still ship for the human and hope the agent can scrape the DOM or call undocumented REST. That is the 2012 equivalent of \"we'll make the mobile site later.\"
\nThe fix is one capability layer. Humans get forms and tables. Agents get MCP (or an equivalent tool surface). Both hit the same domain services.
\nWhen you add \"export CSV,\" you add the tool and the button, not only the button.
\n\nNext time you add a button, write the matching tool in the same pull request, calling the same domain service.\n\nConcrete example: an ops tool for delayed Shopify orders.
\nlist_delayed_orders, get_order, create_customer_message, issue_partial_refund (scoped).If you start with the queue page, the agent becomes a second-class citizen forever. If you start with the tools, the queue page is just one client.
\nPeople reach for chat because self-service still fails often enough to hurt. Industry surveys have long shown most customers try to solve a problem themselves before they contact a human, often cited around four in five attempts.
\nCompletion is the hard part. When the product cannot be operated by an agent with tools, chat becomes theater: a polite model that cannot refund, cannot look up the right tenant, and cannot tell you why the tool failed.
\nTreat chat as a front door:
\nNo more \"read the docs, click twelve pages, then open a ticket.\" The docs become training data for skills. The pages become optional. Support becomes the exception path.
\nA tool without a skill is a loaded gun with no safety briefing.
\nSkills (playbooks, runbooks, Cursor-style skill files, Claude Projects instructions; pick your stack) encode:
\nMCP first without skills first produces agents that call refund_order because the user sounded upset. Skills first without MCP first produces beautiful instructions with nothing to execute.
Ship both. In practice I write the skill outline in the same PR as the first tool handlers: same review, same tenant tests.
\nIf you already have a product, you do not need a rewrite. In one sprint:
\nnot_found, forbidden, validation_failed) instead of empty 500s.If the agent cannot finish the job with a human in the loop for money moves, you learned something cheap. If it can, you just shipped mobile-first for the agent decade.
\n\nOpen your last month of support tickets and pick the three requests that come up most. Those are your first three tools.\n\nMobile first was never really about phones. It was about designing for the hardest real constraint first.
\nThe hard constraint now is: can an authorized agent complete the work safely, with skills and tools, while a human supervises?
\nCall it MCP first, agent first, or skills first; same idea. Prepare the tools and access. Then build the product. Chat becomes the place people ask; capability is what answers.
\nIf you are wiring this into multi-tenant Laravel, start with auth and audit, idempotency, and structured tool errors.
\nIf you are still choosing the connector shape, read API vs MCP first.
\n\n- Every product now has two customers: a human and an agent.\n- Build in this order: outcomes, tools, permissions, skills, API, then UI.\n- Chat is a front door. Tools, approval gates and audit logs do the work behind it.\n- Ship skills with tools, so the agent knows when to call what.\n\nWhich three things do your users ask support for that an agent with the right tools could finish today?
\nWhen a tool fails, the model needs a sentence it can act on, not a crashed server.
\n\n- Business failures go back as a normal tool result with `isError: true`. JSON-RPC errors are for protocol problems only.\n- Catch once at the MCP adapter and map domain exceptions to a small, stable set of codes.\n- Each error needs a hint that names the next tool or field, and the same code goes into your audit log.\n\nAuth and audit logging keep a tool attributable. Idempotency keys keep a retry from charging twice. Neither helps if a failed tool call looks like a crashed server to the model.
\nI keep seeing the same bug in Laravel MCP servers. A domain failure throws, the transport turns it into a JSON-RPC error (or an empty 500), and Claude never gets a sentence it can use to self-correct.
\nThe agent retries the same bad invoice ID, invents a different tool, or apologizes to the user. The invoice was simply not in that tenant.
\nThis post is the third gate I ship with every mutating MCP surface: tool execution errors that stay inside the tool result, carry a stable code, and tell the model what to try next, without leaking stack traces or other tenants' IDs.
\nA support agent asks Claude to fetch invoice inv_9f2. Your tool does this:
public function handle(string $invoiceId, McpActor $actor): array\n{\n $invoice = Invoice::query()\n ->where('tenant_id', $actor->tenantId)\n ->where('id', $invoiceId)\n ->firstOrFail(); // ModelNotFoundException → 500 / JSON-RPC error\n\n return $this->present($invoice);\n}\nfirstOrFail() is correct for an HTTP controller that should return 404. Over MCP it is wrong.
The Model Context Protocol draws a hard line:
\nerror object. The client may surface these; models rarely recover well.tools/call result with isError: true and actionable text in content.That second path is how the MCP tools specification expects language models to self-correct. Hide the failure in a protocol error and you train the agent to flail.
\nEvery tool handler ends in one of three shapes:
\nfinal class McpToolResult\n{\n public static function ok(array $payload): array\n {\n $json = json_encode($payload, JSON_THROW_ON_ERROR);\n\n return [\n 'content' => [['type' => 'text', 'text' => $json]],\n 'structuredContent' => $payload, // optional; keep in sync with outputSchema if you declare one\n 'isError' => false,\n ];\n }\n\n public static function fail(\n string $code,\n string $message,\n bool $retryable = false,\n ?string $hint = null,\n ): array {\n $body = array_filter([\n 'error' => [\n 'code' => $code,\n 'message' => $message,\n 'retryable' => $retryable,\n 'hint' => $hint,\n ],\n ]);\n\n $text = $hint\n ? \"{$message} Hint: {$hint}\"\n : $message;\n\n return [\n 'content' => [['type' => 'text', 'text' => $text]],\n // Prefer omitting structuredContent on errors unless your clients\n // are known to skip outputSchema validation when isError is true.\n 'isError' => true,\n ];\n }\n}\nNotes that bite in production:
\nisError: true is still a successful RPC result. The call happened; the business outcome failed. That is the whole point.content. Models read the text blocks. A code alone is not enough.structuredContent on errors if the tool declares outputSchema. Some clients still validate error payloads against the success schema. The safe default today: actionable text in content, no structured error object, unless you have verified your client stack.Do not invent a new error dialect per tool. Use one mapper, shared with the auth gate denial reasons and the idempotency conflict codes:
\n| Code | \nWhen | \nRetry? | \nHint | \n
|---|---|---|---|
not_found | \nMissing in this tenant | \nno | \nList first; check the ID | \n
forbidden | \nActor lacks scope | \nno | \nElevate or pick another resource | \n
validation_failed | \nArgs fail domain rules | \nno | \nName the bad field and expected shape | \n
conflict | \nSame key, different args | \nno | \nNew key for a new intent | \n
in_flight | \nIdempotency claim held | \nyes | \nRetry the same key shortly | \n
rate_limited | \nTool budget exhausted | \nyes | \nBack off; include Retry-After | \n
upstream_unavailable | \nProvider blip | \nyes | \nRetry once; no second refund | \n
internal | \nTruly unexpected | \nno | \nStop; send the user to support | \n
final class McpErrorMapper\n{\n public function fromThrowable(\\Throwable $e, McpActor $actor): array\n {\n return match (true) {\n $e instanceof ModelNotFoundException => McpToolResult::fail(\n code: 'not_found',\n message: 'Resource not found in your workspace.',\n hint: 'List resources first, then retry with an ID from that list.',\n ),\n $e instanceof AuthorizationException => McpToolResult::fail(\n code: 'forbidden',\n message: 'You are not allowed to run this tool on that resource.',\n hint: 'Ask the account owner for the required permission.',\n ),\n $e instanceof ValidationException => McpToolResult::fail(\n code: 'validation_failed',\n message: $e->validator->errors()->first() ?: 'Invalid arguments.',\n hint: 'Fix the field named in the message and call again.',\n ),\n $e instanceof IdempotencyConflict => McpToolResult::fail(\n code: 'conflict',\n message: 'Idempotency key was reused with different arguments.',\n hint: 'Generate a new idempotency_key for a new intent.',\n ),\n $e instanceof IdempotencyInFlight => McpToolResult::fail(\n code: 'in_flight',\n message: 'A request with this idempotency key is still running.',\n retryable: true,\n hint: 'Wait a few seconds and retry with the same idempotency_key.',\n ),\n default => McpToolResult::fail(\n code: 'internal',\n message: 'The tool failed unexpectedly. Do not retry with new side effects.',\n hint: 'Tell the user support needs the session id '.$actor->sessionId.'.',\n ),\n };\n }\n}\nCatch at the MCP adapter boundary, once, not inside every tool with a different string. Tools throw domain exceptions. The adapter decides what the model is allowed to see.
\npublic function invoke(string $tool, array $args, McpActor $actor): array\n{\n try {\n $this->auth->assertAllowed($actor, $tool, $args);\n $result = $this->tools->run($tool, $args, $actor);\n $this->audit->allowed($tool, $args, $result);\n\n return McpToolResult::ok($result);\n } catch (\\Throwable $e) {\n $error = $this->errors->fromThrowable($e, $actor);\n $this->audit->denied($tool, $args, $this->errors->codeOf($e));\n\n return $error;\n }\n}\nnot_found must mean \"not in this tenant,\" the same rule as your global scopes from Laravel multi-tenancy. A row that exists for another customer is still not_found to this actor.
Never return forbidden with the other tenant's ID in the message. That is an enumeration leak dressed as honesty.
The first one is useless. The model will guess.
\nThe hint should name another tool the agent already has, or a field to fix, not a URL to your internal runbook.
\nYou are writing for an LLM tool loop, not a human SRE.
\nFor retryable: true, say what to reuse. Idempotency taught us that \"retry\" without \"same key\" creates double refunds. Encode that in the hint every time:
A refund with this idempotency key is still in flight.\nHint: wait ~30s and call refund_order again with the SAME idempotency_key and arguments.\nYour audit stream already records allowed | denied | replayed. Store the public error code on denials.
When an agent loops on not_found fifty times, you want a query, not a vibes-based log scroll.
McpAuditEvent::query()->create([\n 'tenant_id' => $actor->tenantId,\n 'user_id' => $actor->userId,\n 'session_id' => $actor->sessionId,\n 'tool' => $tool,\n 'decision' => 'denied',\n 'reason' => $code, // not_found | forbidden | …\n 'args_hash' => hash('sha256', json_encode($redactedArgs)),\n 'args' => $redactedArgs,\n 'result_summary' => ['isError' => true, 'code' => $code],\n 'idempotency_key' => $args['idempotency_key'] ?? null,\n 'duration_ms' => $ms,\n]);\nOps dashboard: top denied codes per tool, per tenant, last 24h. Product signal: a spike in validation_failed usually means your tool schema description is lying to the model.
firstOrFail() / uncaught exceptions escaping the MCP adaptercontent texterr vs error vs message only)forbidden messages that confirm another tenant's resource existsconflict / in_flight into a generic internalretryable: true without a keyIf a human API would return 404/409/422 with a problem+json body, the agent path needs isError with a code and a hint, not a thrown Exception.
This sits on top of MCP auth and audit logging and idempotency keys for mutations. Same McpActor, same audit table, same claim codes, now visible to the model instead of only to your pager.
Tenancy rules stay the Laravel multi-tenancy ones; do not invent a parallel \"agent ACL.\"
\nFor wallet-level token budgets at the LLM edge, that is a different layer. See rate limiting your AI wallet. Tool errors are what happen after the model is already allowed to call you.
\nIf your MCP tools still throw through to the client and you want the error surface reviewed before agents hold live keys, start with an AI Integration Sprint. We will inventory which failures are recoverable, which must stop the loop, and how the audit trail proves it after a bad night.
\n\n- **Use MCP tool-result errors (`isError: true`) for business failures.** Reserve JSON-RPC errors for protocol problems.\n- **Catch once at the adapter.** Map domain exceptions to a small, stable code set.\n- **Write for the model:** message + hint that names the next tool or field; set `retryable` honestly.\n- **Omit or carefully handle `structuredContent` on errors** if `outputSchema` is in play.\n- **Audit the same codes** the agent saw. Spikes are product bugs, not just noise.\n- **Never leak cross-tenant existence** through clever error text.\n\nWhen your agent hits a missing invoice today, does it get a hint it can act on, or a crash it can only apologize for?
\nGiving Claude a tool that can read tickets and billing is useful. Giving it the same credentials your admin panel uses is how you invent a new incident class.
\n\n- Bind one actor (tenant, user, session, scopes) at the edge. Tools read it and never accept a tenant from the model.\n- Keep MCP tools thin and reuse the same finders and policies as your admin UI.\n- Log every attempt, allowed or denied, with redacted args, a reason and the idempotency key.\n\nI have been wiring MCP servers into real Laravel SaaS surfaces, the same domain that powers the MCP SaaS case study.
\nThe hard part is not JSON-RPC. The hard part is making sure every tool call is tenant-scoped, attributable, and auditable, without teaching the model a parallel permission system.
\nThis post is the pattern I keep shipping: actor context on the request, tools that refuse to invent a tenant, and an audit log that survives a prompt injection attempt.
\nMost first MCP demos look like this:
\nlist_tickets, get_invoice, refund_order.That works in a sandbox. In production it fails the moment:
\ntool=refund_order ok=true with no actor, no tenant, and no arguments.MCP does not invent tenancy for you. Laravel already has it. The job is to force every tool through that gate.
\nTreat an MCP tool invocation like an HTTP request that happens to arrive over JSON-RPC. Bind an actor before any tool runs:
\nfinal class McpActor\n{\n public function __construct(\n public readonly string $tenantId,\n public readonly string $userId,\n public readonly string $sessionId,\n /** @var list<string> */\n public readonly array $scopes,\n ) {}\n}\nResolve it once at the transport edge (API token, signed agent session, or OAuth subject) and stash it on the container for the lifetime of that call:
\napp()->instance(McpActor::class, $actor);\nTools never accept tenant_id from the model. If the model passes one, drop it. The only tenant that exists is the one on McpActor.
That single rule kills the most common cross-tenant footgun: an LLM that \"found\" an ID in context and used it.
\n\nGrep your tool schemas for `tenant_id`. Remove it from every input and read it from `McpActor` instead.\n\nKeep MCP handlers boring. Authorize, call the same service the Filament admin would call, return a shaped DTO:
\nfinal class GetInvoiceTool\n{\n public function __construct(\n private readonly McpActor $actor,\n private readonly InvoiceFinder $invoices,\n private readonly McpAuditLogger $audit,\n ) {}\n\n public function __invoke(array $args): array\n {\n $invoiceId = (string) ($args['invoice_id'] ?? '');\n\n $invoice = $this->invoices->findForTenant(\n tenantId: $this->actor->tenantId,\n invoiceId: $invoiceId,\n );\n\n if ($invoice === null) {\n $this->audit->denied('get_invoice', $args, 'not_found_or_foreign');\n return ['error' => 'invoice_not_found'];\n }\n\n if (! in_array('billing:read', $this->actor->scopes, true)) {\n $this->audit->denied('get_invoice', $args, 'missing_scope');\n return ['error' => 'forbidden'];\n }\n\n $payload = $invoice->toMcpArray();\n $this->audit->allowed('get_invoice', $args, $payload);\n\n return $payload;\n }\n}\nNotice what is missing: a raw GraphQL passthrough, a DB::table query that forgets tenant_id, and any path where \"not found\" and \"wrong tenant\" diverge. Same response. Same audit reason code you can filter later.
If you already enforce tenancy in Eloquent global scopes, keep them. MCP is not a reason to bypass them \"just this once.\"
\nI log one row per tool attempt, allowed or denied, with enough to reconstruct the call without storing secrets:
\nMcpAuditEvent::query()->create([\n 'tenant_id' => $actor->tenantId,\n 'user_id' => $actor->userId,\n 'session_id' => $actor->sessionId,\n 'tool' => $tool,\n 'decision' => $decision, // allowed | denied\n 'reason' => $reason,\n 'args_hash' => hash('sha256', json_encode($redactedArgs)),\n 'args' => $redactedArgs,\n 'result_summary' => $summary, // counts / IDs, never card data\n 'idempotency_key' => $idempotencyKey,\n 'duration_ms' => $ms,\n]);\nRules that have paid for themselves:
\nargs.refund_order right before a successful one from a different session.This is the same discipline I want on agentic commerce checkouts. Money-moving tools are just the obvious version of the rule.
\n\nis_admin is a product concept. MCP wants capabilities:
tickets:read\ntickets:write\nbilling:read\nbilling:refund\ndocs:search\nMap your SaaS roles onto scopes at session mint time. The tool checks scopes.
\nWhen a customer upgrades from \"support viewer\" to \"billing ops,\" you change the session mint, not every tool.
\nAlso expire sessions aggressively. An MCP session that lives for a week is a stolen-laptop story waiting to happen.
\n\nWrite down the scopes each of your SaaS roles maps to, then check how long an MCP session lives before it expires.\n\nA short deny-list that has saved me more than any middleware:
\nIf a human support agent cannot do it in the admin UI, the model does not get a secret door.
\nThis sits next to Laravel multi-tenancy and Octane under load on the framework side, and next to MCP vs A2A on the protocol side.
\nThe tenancy model is Laravel. The tool surface is MCP. The trust boundary is yours.
\nIf you are putting Claude on top of a multi-tenant Laravel app and want a second pair of eyes on the boundary, start with an AI Integration Sprint. We will map the tools, the scopes, and the audit trail before any agent sees production data.
\n\n- **Never let the model choose the tenant.** Bind `McpActor` at the edge; tools read it, they do not accept it.\n- **Thin tools, thick domain.** Reuse the same finders and policies as your admin UI.\n- **Audit allows and denies.** Include tool, actor, redacted args, reason, and idempotency key.\n- **Scopes over roles** at the MCP boundary. Mint short-lived sessions.\n- **No parallel API.** If it is dangerous in HTTP, it is dangerous in JSON-RPC.\n\nIf you got paged tonight about a strange refund, could your audit log tell you which actor, tenant and session asked for it?
\nAgents retry. Transports retry. Humans mash \"send\" when the spinner hangs. Your mutation tools have to survive all three.
\n\n- Every MCP mutation needs a client-supplied idempotency key. Reads stay free.\n- The server short-circuits: replay the first result, reject a reused key with new args, and refuse duplicates still in flight.\n- A unique database index per tenant, tool and key is the real lock under Octane and Horizon.\n\nAuth and audit logging keep an MCP tool attributable. They do not keep a retry from charging the card twice.
\nIf your mutation tools treat every call as a new intent, you will invent double refunds, duplicate password emails, and seat resets that fire three times for one user request.
\nThis post is the follow-on pattern: client-supplied idempotency keys, a server short-circuit that returns the first result, keys stored on the same audit rows you already trust at 3am, and the race conditions that show up the moment you put Octane and Horizon in the path.
\nA support agent asks Claude to refund order ord_9f2. The tool runs. The HTTP connection from the MCP client drops at 30s.
The client retries with the same arguments. Your handler has no memory of the first call, so Stripe sees two refunds.
\nSame story for:
\nreset_seat: second call wipes a freshly restored seat.send_password_reset: user gets three emails in ninety seconds.apply_credit: ledger entry lands twice because Horizon re-dispatched after a worker restart.None of these are exotic. They are the default behavior of any system that treats \"call arrived\" as \"intent is new.\"
\nModels and clients will retry. Make the server safe when they do.
\nThe fix is not \"tell the model not to retry.\"
\nEvery mutating tool requires an idempotency_key in the arguments. The client generates it (UUID v4 is fine) and reuses it for retries of the same intent.
final class RefundOrderTool\n{\n public function __construct(\n private readonly McpActor $actor,\n private readonly IdempotencyGate $gate,\n private readonly OrderRefunder $refunder,\n private readonly McpAuditLogger $audit,\n ) {}\n\n public function __invoke(array $args): array\n {\n $key = trim((string) ($args['idempotency_key'] ?? ''));\n if ($key === '' || strlen($key) > 128) {\n $this->audit->denied('refund_order', $args, 'missing_idempotency_key');\n return ['error' => 'idempotency_key_required'];\n }\n\n if (! in_array('billing:refund', $this->actor->scopes, true)) {\n $this->audit->denied('refund_order', $args, 'missing_scope');\n return ['error' => 'forbidden'];\n }\n\n return $this->gate->run(\n actor: $this->actor,\n tool: 'refund_order',\n key: $key,\n args: $args,\n execute: fn () => $this->refunder->refund(\n tenantId: $this->actor->tenantId,\n orderId: (string) $args['order_id'],\n amountCents: (int) $args['amount_cents'],\n ),\n );\n }\n}\nRules I enforce at the edge:
\nget_invoice does not need one. refund_order does not run without one.The model is allowed to pass the key. It is not allowed to invent tenancy, scopes, or \"just this once\" bypasses.
\nSame discipline as the auth post: tools read McpActor, they do not accept tenant_id from the payload.
The gate is the whole product. First call executes and records. Later calls with the same key return the stored outcome without touching Stripe, the mailer, or the seat service.
\nfinal class IdempotencyGate\n{\n public function __construct(\n private readonly IdempotencyStore $store,\n private readonly McpAuditLogger $audit,\n ) {}\n\n /**\n * @param callable(): array $execute\n * @return array<string, mixed>\n */\n public function run(\n McpActor $actor,\n string $tool,\n string $key,\n array $args,\n callable $execute,\n ): array {\n $fingerprint = $this->fingerprint($args);\n $claim = $this->store->claim(\n tenantId: $actor->tenantId,\n tool: $tool,\n key: $key,\n argsHash: $fingerprint,\n ttlSeconds: 86_400,\n );\n\n if ($claim->status === ClaimStatus::Replay) {\n $this->audit->replayed($tool, $args, $key, $claim->result);\n return $claim->result;\n }\n\n if ($claim->status === ClaimStatus::Conflict) {\n $this->audit->denied($tool, $args, 'idempotency_payload_mismatch');\n return ['error' => 'idempotency_key_reuse_with_different_args'];\n }\n\n if ($claim->status === ClaimStatus::InFlight) {\n $this->audit->denied($tool, $args, 'idempotency_in_flight');\n return ['error' => 'request_in_flight'];\n }\n\n try {\n $result = $execute();\n $this->store->complete($actor->tenantId, $tool, $key, $result);\n $this->audit->allowed($tool, $args, $result, $key);\n return $result;\n } catch (\\Throwable $e) {\n $this->store->fail($actor->tenantId, $tool, $key, $e);\n $this->audit->denied($tool, $args, 'execution_failed', $key);\n throw $e;\n }\n }\n\n private function fingerprint(array $args): string\n {\n $copy = $args;\n unset($copy['idempotency_key']);\n ksort($copy);\n\n return hash('sha256', json_encode($copy, JSON_THROW_ON_ERROR));\n }\n}\nThat conflict case is non-negotiable. Idempotency is \"same intent, same result,\" not \"this UUID is a free pass for whatever args you send next.\"
\nYou already write one audit event per tool attempt. Put the key there.
\nWhen finance asks \"did we double-refund ord_9f2?\", you join on idempotency_key instead of guessing from timestamps.
McpAuditEvent::query()->create([\n 'tenant_id' => $actor->tenantId,\n 'user_id' => $actor->userId,\n 'session_id' => $actor->sessionId,\n 'tool' => $tool,\n 'decision' => $decision, // allowed | denied | replayed\n 'reason' => $reason,\n 'args_hash' => hash('sha256', json_encode($redactedArgs)),\n 'args' => $redactedArgs,\n 'result_summary' => $summary,\n 'idempotency_key' => $idempotencyKey,\n 'duration_ms' => $ms,\n]);\nreplayed is a first-class decision. It is not a silent cache hit. Ops needs to see that the client retried and the server refused to mutate again.
Index what you query:
\nSchema::table('mcp_audit_events', function (Blueprint $table) {\n $table->string('idempotency_key', 128)->nullable();\n $table->string('args_hash', 64)->nullable();\n $table->index(['tenant_id', 'tool', 'idempotency_key']);\n});\nThe audit table is the narrative. The idempotency store is the lock.
\nDo not merge them into one table and hope. Audit wants append-only history; the gate wants a unique claim row you can update from in_flight to completed.
Separate three things people keep collapsing:
\n| Field | \nPurpose | \nStore raw? | \n
|---|---|---|
idempotency_key | \nClient retry identity | \nYes, you need it for lookups and support | \n
args_hash | \nDetect key reuse with different payloads | \nHash only of redacted, sorted args | \n
args / result_summary | \nIncident reconstruction | \nRedacted IDs and counts, never secrets | \n
Hash the fingerprint of the business args (order ID, amount, seat ID) after stripping the key itself. Store the raw key. Store redacted args on the audit row the same way you already do for reads.
\nDo not hash the idempotency key into oblivion \"for privacy.\" You will need the original string when a client asks why their retry returned request_in_flight.
You will also need it when you correlate MCP logs with Stripe's Idempotency-Key header if you forward the same value downstream.
Forwarding is optional but clean: if Stripe already keys refunds, pass the MCP key through. One identity across your audit log and the payment provider.
\nA global unique index on idempotency_key alone is wrong in a multi-tenant SaaS. Two tenants can generate the same UUID. Two tools can legitimately reuse a key namespace if you are not careful.
The claim identity is:
\n(tenant_id, tool, idempotency_key)\nThat matches how Laravel multi-tenancy already scopes every other sensitive row. MCP is not special.
\nSchema::create('mcp_idempotency_claims', function (Blueprint $table) {\n $table->id();\n $table->string('tenant_id');\n $table->string('tool', 64);\n $table->string('idempotency_key', 128);\n $table->string('args_hash', 64);\n $table->string('status', 16); // in_flight | completed | failed\n $table->json('result')->nullable();\n $table->timestamp('expires_at');\n $table->timestamps();\n\n $table->unique(['tenant_id', 'tool', 'idempotency_key']);\n});\nPrune expired rows on a schedule. Do not rely on \"we will never get collisions\" as a retention policy.
\nThis is where the naive SELECT then INSERT dies.
Octane keeps workers warm. Two concurrent MCP calls for the same key can hit two workers in the same second.
\nHorizon can re-run a job after a SIGKILL mid-refund. Cache locks help for the in-process case; they are not enough alone if the mutation is async.
\n(tenant_id, tool, idempotency_key). The database is the source of truth for \"who won the claim.\"INSERT … ON CONFLICT / firstOrCreate + catch unique violations. The loser reads the winner's row and follows Replay / InFlight / Conflict logic. Never \"just run it anyway.\"complete, the row sits in in_flight. Expire it (e.g. 60 to 120s) so a legitimate retry can reclaim, but only after you are sure the first side effect did not commit. Prefer making the downstream call itself idempotent (Stripe key, mail message-id) so reclaiming is safe.public function claim(\n string $tenantId,\n string $tool,\n string $key,\n string $argsHash,\n int $ttlSeconds,\n): ClaimResult {\n try {\n $row = McpIdempotencyClaim::query()->create([\n 'tenant_id' => $tenantId,\n 'tool' => $tool,\n 'idempotency_key' => $key,\n 'args_hash' => $argsHash,\n 'status' => 'in_flight',\n 'expires_at' => now()->addSeconds($ttlSeconds),\n ]);\n\n return ClaimResult::fresh($row);\n } catch (UniqueConstraintViolationException) {\n $existing = McpIdempotencyClaim::query()\n ->where('tenant_id', $tenantId)\n ->where('tool', $tool)\n ->where('idempotency_key', $key)\n ->firstOrFail();\n\n return $this->interpretExisting($existing, $argsHash);\n }\n}\nIf you only remember one sentence: the unique index is the lock; application locks are a courtesy.
\n\nFire the same `refund_order` call twice in parallel against staging with one key. Exactly one refund should reach the payment provider.\n\nresult JSONIf a human admin UI would show a confirmation dialog before doing it twice, the agent path needs a key.
\nThis sits on top of MCP auth and audit logging: same McpActor, same audit stream, stricter rules for anything that moves money or mail.
Tenancy remains the Laravel model from multi-tenancy; concurrency reality is the Octane one.
\nIf you are exposing refund or billing tools over MCP and want the gate reviewed before an agent holds a live key, start with an AI Integration Sprint. We will map which tools mutate, how keys flow, and what the audit trail must prove after a retry storm.
\n\n- **Require client idempotency keys on every MCP mutation.** Reads stay free; refunds do not.\n- **Short-circuit on the server.** Replay the first result; conflict on arg mismatch; reject in-flight duplicates.\n- **Store the key on the audit row** and keep a separate claims table with `(tenant_id, tool, key)` uniqueness.\n- **Hash args for comparison; store keys raw; redact everything else.**\n- **TTL per tool class.** 24h for money, shorter for mail. Prune expired claims.\n- **Unique DB constraint beats Redis hope** under Octane workers and Horizon restarts.\n- **Downstream idempotency** (Stripe, mail providers) makes reclaim-after-crash safe.\n\nIf your MCP client dropped the connection mid-refund right now and retried, how many refunds would your customer see?
\nTrust is not a QA strategy.
\n\n- Clean formatting on AI code says nothing about whether it is secure or correct.\n- Half of agentic PRs change tested code without touching tests, and most agent-written tests assert almost nothing.\n- Gate on the diff: static analysis, diff coverage and mutation testing, with the spec written first.\n\nIt does not matter whether you are reviewing a pull request from a junior developer or accepting an entire feature an autonomous agent produced in ninety seconds. The engineering question is identical: does this code actually work, and how do you know?
\nSomewhere in the shift to agentic workflows, a lot of teams quietly stopped asking the second half of that question.
\nLLMs emit code that compiles, lints clean, and reads better than most of what humans write under deadline. So the code looks reviewed. Teams merge it, ship it, and find out what it does in production.
\nThis is the part that catches experienced engineers off guard, because our instincts were trained on human output.
\nHuman code that looks sloppy usually is rough, and we review it accordingly. Human code that looks polished usually got there because someone iterated on it, which means someone thought about it. Polish was a proxy for scrutiny.
\nLLMs break that correlation completely. Veracode's Spring 2026 GenAI Code Security update measured a syntax correctness rate above 95% across the models it tested, against a security pass rate of 55%.
\nThe code is almost always well-formed. It is secure roughly half the time.
\nFormatting stopped carrying any signal about correctness, and most review habits have not caught up.
\n// Generated in one shot. Clean names, correct types, zero lint warnings.\nfunction calculateDiscount(user: User, cartTotal: number): number {\n if (user.isVip) {\n return cartTotal * 0.2;\n }\n return cartTotal * 0.05;\n}\nNothing here is wrong, exactly. It is just unfinished in ways that only show up under real traffic.
\nThere is no guard against a negative cartTotal from a refund flow, no clamp on the maximum discount, no decision about what happens when user arrives undefined from a cache miss.
The type signature says those cases cannot happen. The type signature is a claim about the code, not about production.
\nVeracode's Spring 2026 report evaluated 80 coding tasks across Java, JavaScript, C#, and Python.
\nThe aggregate numbers are worth internalizing, because the failures are not evenly spread. They cluster hard around exactly the categories where a missing sanitizer is invisible in review.
\n| CWE category | \nSecurity pass rate | \n
|---|---|
| Insecure cryptography (CWE-327) | \n86% | \n
| SQL injection (CWE-89) | \n82% | \n
| Cross-site scripting (CWE-80) | \n15% | \n
| Log injection (CWE-117) | \n13% | \n
Parameterized queries and modern crypto defaults are baked into the training data. Models get those right most of the time because the safe pattern is the common pattern.
\nOutput encoding is contextual. Whether a string needs HTML-escaping, attribute-escaping, or JS-escaping depends on where it lands in the template, and the model cannot see the template from inside the function it is writing. So it guesses, and it guesses wrong about six times out of seven.
\nLanguage matters too. The same report puts Python at a 62% security pass rate and Java at 29%. If your stack is JVM-heavy, the baseline you are working from is worse than the headline number suggests.
\nOne genuinely encouraging data point: GPT-5 with extended reasoning hit 70 to 72%, the best pass rate Veracode has observed. That is real improvement. It is still a coin flip you would never accept from a human contributor.
\n\nIn your next review of agent-written code, check output encoding before anything else. It is the category that fails most.\n\nChat completions are one thing. Autonomous agents opening PRs are another, and this is where the accounting gets uncomfortable.
\nA July 2026 empirical study accepted at ICSME, Test Coverage Analysis of Agentic Pull Requests, analysed 4,882 agent-generated pull requests from the AIDev dataset: 532 Java, 4,350 Python, across five coding agents. Two findings should reshape how you gate merges.
\nAgents included test changes in only 49.6% of the PRs that touched code under test. Not greenfield repos with no test infrastructure: code that already sat inside a test suite, changed without the suite being touched.
\nExisting test suites did not pick up the slack. They executed 61.5% of the agents' changed executable lines in Java and just 27.0% in Python. In 64.8% of Python PRs, not one changed line was executed by any existing test. The safety net everyone assumes is there is mostly holes.
\nAgents optimize for satisfying the prompt. Ask for a checkout feature and you get a controller, a migration, and a view, because that is what \"done\" looks like from the prompt's perspective.
\nBoundary conditions, negative assertions, and concurrency cases are not in the prompt, so they are not in the diff.
\nHere is the failure mode I see most often: a team wires up a coverage gate, sees 90%, and concludes the AI-authored code is verified.
\nLine coverage measures execution, not verification. It tells you a line ran during the test suite. It says nothing about whether anything checked the result.
\nA separate 2026 study, All Smoke, No Alarm, looked specifically at what agent-written tests assert. It covered 86,156 test-file patches from 33,596 agent-authored pull requests spanning 2,807 repositories (Codex, Copilot, Devin, Cursor, and Claude Code).
\n80.2% of those patches contained weak or no explicit oracle signals. An agent rewarded for a green build learns very quickly that the cheapest path to green is an assertion that cannot fail.
\n# 100% line coverage on process_payment. Verifies essentially nothing.\ndef test_process_payment():\n result = process_payment(100)\n assert result is not None\nThat test passes if process_payment charges the right amount. It also passes if it charges zero, double-charges, returns a stale object, or silently swallows a declined card. Coverage went up. Confidence should not have.
The fix is mutation testing, which measures the thing you actually care about: if I break the code, does a test go red?
\nStryker for JS/TS, Infection for PHP, mutmut for Python, PIT for Java. Mutation score is the only coverage-adjacent metric I have found that agents cannot game by writing more assertions of nothing.
\nBack in the ICSME coverage study, there is a number that gets less attention than it deserves. Existing tests miss 86.0% of the try/catch blocks agents touch in Java, and 81.0% of Python error-handling constructs. The recovery paths are the least-tested code in the diff.
\nThis is the worst possible failure shape. A crash is loud, has a stack trace, and pages someone.
\nA catch block that logs a generic string and returns a default value produces a system that keeps running while quietly returning wrong answers. You do not find those in staging. You find them in a reconciliation report three weeks later.
When you review agent output, read the catch blocks first. Ask what happens to the caller when the block executes, and whether the log line contains enough context to reconstruct the failure at 3am.
That question is also the difference between logging and monitoring: a swallowed exception is logged, and monitored by nobody.
\nThe 2025 DORA State of AI-assisted Software Development report is the macro version of everything above. Its central finding: higher AI adoption correlates with an increase in delivery throughput and an increase in delivery instability, at the same time.
\nThat is not an argument against AI. It is a statement about where the bottleneck moved.
\nWhen change volume goes up and the control systems (automated testing, version control discipline, fast feedback) stay where they were, instability is the arithmetic result. DORA's own ROI model puts a number on it: in their sample calculation, a change failure rate drifting from 5% to 6% after AI adoption costs $344,000 in downtime.
\nAI amplifies whatever your delivery process already was. That is the same trade-off I mapped out in AI vs traditional development, now with a large-scale dataset behind it instead of an argument.
\nCoverage percentage is a vanity metric here. These four are not:
\nManual review does not scale against agent output. A reviewer facing 800 lines of plausible-looking code per hour is not reviewing; they are skimming for style.
\nThe gate has to be automatic, and it has to run on the diff. I make the full case for giving up line-by-line reading separately; this is the tooling that has to exist first.
\nname: Agentic PR gate\non:\n pull_request:\n types: [opened, synchronize, reopened]\n\njobs:\n sast:\n runs-on: ubuntu-latest\n container:\n image: semgrep/semgrep\n steps:\n - uses: actions/checkout@v5\n - run: semgrep ci --config=p/owasp-top-ten\n\n diff-coverage:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n with:\n fetch-depth: 0\n - uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.4\"\n coverage: pcov\n - run: composer install --no-interaction --prefer-dist\n - run: vendor/bin/phpunit --coverage-cobertura=coverage.xml\n - run: pipx install diff-cover\n # Fail on new lines that no test executes — total coverage is irrelevant here.\n - run: diff-cover coverage.xml --compare-branch=origin/${{ github.base_ref }} --fail-under=90\n\n mutation:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n with:\n fetch-depth: 0\n - uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.4\"\n coverage: pcov\n - run: composer install --no-interaction --prefer-dist\n # Mutate only the changed files, and require tests to actually kill the mutants.\n - run: |\n vendor/bin/infection \\\n --git-diff-filter=AM \\\n --git-diff-base=origin/${{ github.base_ref }} \\\n --min-msi=70 \\\n --threads=max\nThree layers, each catching what the previous one cannot:
\ndiff-cover refuses the PR where new lines are unexecuted.These two are worth more than any of it:
\nWrite the spec first. Give the agent the test file, the OpenAPI schema, or the behavioral contract before it writes the implementation. It is much better at satisfying an explicit oracle than at inventing one, and you keep authorship of the definition of correct.
\nGive the agent real context. A large share of the XSS failures come from the model not being able to see the rendering context. Wiring your agent into the actual schema, actual codebase, and actual docs (the MCP tooling story) measurably reduces guessing. It reduces it. It does not remove the need to verify.
\nNone of this is an argument for writing code by hand. I ship agent-authored code every week.
\nThe argument is that verification is the part of the job that did not get automated, and pretending otherwise just moves the discovery of your bugs from CI to your customers.
\nIf the only thing between an agent-authored PR and production today is \"a human skimming the diff,\" that is worth fixing this quarter. Tell me what your stack looks like and I will tell you where I would put the first gate. 🤘
\n\n- **Polish stopped being a proxy for scrutiny.** 95% syntax correctness against a 55% security pass rate means clean formatting now tells you nothing about whether the logic holds.\n- **Failures cluster in context-dependent categories.** Crypto and SQL injection pass 82 to 86% of the time. XSS and log injection pass 13 to 15%. Review output encoding first.\n- **Half of agentic PRs touch code without touching tests**, and existing suites execute as little as 27% of the changed lines. Gate on test presence, not good intentions.\n- **Mutation score over line coverage.** 80.2% of agent-written test patches carry weak or absent oracles. Only mutation testing catches an assertion that cannot fail.\n- **Read the catch blocks first.** Test suites miss over 80% of the error-handling paths agents touch, which means the likely production symptom is a wrong answer, not a crash.\n\nWhat is currently standing between an agent-authored PR and production in your pipeline?
\nAn AI agent can discover a product, build a cart, and reach checkout faster than a human. It can also make an expensive mistake in seconds.
\n\n- Never hand an agent broad Shopify credentials. Put an MCP governance server between the agent and the store.\n- UCP profile negotiation, signed requests and idempotency keys are security controls, not formalities.\n- Human approval is a checkout state. Respect `requires_escalation` instead of routing around it.\n\nAgentic commerce changes the security boundary of a Shopify store. A storefront is no longer only serving pages to people.
\nIt exposes product data, pricing, inventory, fulfillment options, checkout actions, and order state to autonomous software.
\nThat creates a real problem. If an agent receives broad Shopify credentials, a prompt injection or an implementation bug can expose customer data or trigger an unauthorized transaction.
\nThe fix isn't to block agents. It's to give them a narrow, observable, policy-controlled path through the commerce stack. The principle is short:
\nLet the agent reason about commerce, but never let it define its own permissions.
\nTraditional Shopify integrations assume a known application, a fixed workflow, and a human reviewing the final checkout screen. Autonomous agents remove all three assumptions.
\nA buyer agent may:
\nEvery step carries model uncertainty. The agent may misread intent. Product data may be stale. A tool description may be too broad. An external page may carry malicious instructions. A retry may duplicate an action that moves money.
\nThis is why agentic commerce needs a different engineering model. The agent should never hold unrestricted access to Shopify Admin APIs. It should reach approved commerce capabilities through a controlled protocol layer.
\nThe Universal Commerce Protocol (UCP) gives agents and merchants a standard way to describe capabilities, negotiate compatibility, and execute commerce workflows.
\nShopify's UCP implementation covers the main buyer journey:
\nUCP is not a replacement for Shopify's commerce APIs. It's a standardized interaction layer so different agents and merchant systems can speak in shared commerce concepts.
\nIt supports multiple transports (REST, JSON-RPC, MCP, and A2A, plus the JSON-RPC 2.0 channel behind Shopify's Embedded Checkout Protocol) while the business logic stays the same.
\nThe mechanism that matters for security is profile negotiation. A merchant publishes a business profile, usually at:
\nhttps://store.example.com/.well-known/ucp\nOn Shopify, that profile is tied to the merchant storefront. It's a self-contained document declaring version, services, capabilities, payment_handlers, and (via supported_versions) the profile URIs for older protocol versions. (See the Shopify UCP quick-start for getting the manifest live.)
The agent hosts its own platform profile at an HTTPS URL and passes that URL on every request through the UCP-Agent header. The platform profile mirrors the business one and adds a signing_keys registry.
That single document is both the capability declaration and the key-resolution mechanism, which is why profile validation is a security control, not a formality.
\nShopify then computes the intersection:
\nmerchant capabilities ∩ agent capabilities = negotiated capabilities\nOnly compatible capabilities become active. Extensions (dev.ucp.shopping.discounts, dev.ucp.shopping.buyer_consent, dev.ucp.shopping.fulfillment) carry an extends field naming their parent capability, and they never auto-activate. If the parent isn't in the negotiated set, the extension drops out with it.
A trimmed platform profile, showing the shape rather than a complete document:
\n{\n \"version\": \"2026-04-08\",\n \"services\": {\n \"dev.ucp.shopping\": {}\n },\n \"capabilities\": {\n \"dev.ucp.shopping.cart\": {},\n \"dev.ucp.shopping.checkout\": {}\n },\n \"signing_keys\": [\n {\n \"kty\": \"EC\",\n \"crv\": \"P-256\",\n \"kid\": \"agent-2026-08\",\n \"x\": \"...\",\n \"y\": \"...\"\n }\n ]\n}\nProtocol versions are still moving through 2026. Check the UCP specification for the current registry shape.
\nPin the version your integration uses, validate it strictly, and test version negotiation before rollout. Never silently downgrade to an older capability set.
\nFor the wider shift from visual storefronts to programmable commerce, see why agentic commerce will change the way you build Shopify stores.
\nYou need an intermediate control plane between the agent and Shopify:
\nbuyer\n ↓\nAI agent\n ↓\nMCP client\n ↓\nintermediate MCP governance server\n ↓\nUCP commerce tools\n ↓\nShopify storefront and checkout services\nThat governance server is the whole point. It keeps the model off sensitive APIs and gives the team one place to enforce policy.
\nDon't expose a generic shopify_admin_request. Create narrow tools with explicit inputs and outputs:
search_products(query, filters)\nget_variant(variant_id)\ncreate_cart(lines, buyer_country)\nupdate_cart(cart_id, lines)\ncreate_checkout(cart_id)\nget_order(order_id)\nOne tool, one well-defined job, every parameter validated against a schema before any downstream request.
\nNever accept arbitrary GraphQL from an agent. Use fixed queries and allowlisted fields. That prevents data overexposure and shrinks the blast radius of prompt injection.
\n\nAdmin API tokens belong in a secret manager. Never in:
\nagents.mdThe governance server holds the credential and calls only the minimum Shopify operations required. For public product discovery, prefer Storefront API access with appropriate scopes. Reserve Admin API access for back-office operations that genuinely need it.
\nUse separate credentials for:
\nRevocation and incident analysis get much easier when the blast radius is already partitioned.
\n\nSearch your prompts, tool responses, logs and `agents.md` for any Admin API token. Move every one you find into a secret manager.\n\nAuthorization depends on both the tool and the current commerce state.
\n| Commerce state | \nAllowed actions | \nApproval requirement | \n
|---|---|---|
| Discovery | \nSearch products, read public availability | \nNone | \n
| Cart building | \nAdd or remove line items, estimate totals | \nOptional | \n
| Pre-checkout | \nSelect fulfillment, validate buyer data | \nBuyer confirmation | \n
| Checkout completion | \nSubmit payment mandate or complete checkout | \nStrong authentication | \n
| Post-purchase | \nRead order status, track fulfillment | \nScoped buyer access | \n
A signed agent is not automatically allowed to complete a purchase. High-risk actions need the right authorization tier, scope, buyer identity, and payment consent.
\nTreat every checkout completion as a state transition, not an ordinary tool call.
\nThe protocol has to authenticate more than the model. It authenticates the agent profile, request origin, capability set, and transaction context.
\nUCP signs every HTTP-based transport with RFC 9421 HTTP Message Signatures. Signing is asymmetric: ES256 is mandatory, ES384 optional.
\nPublic keys are JWKs (RFC 7517) published in the signing_keys array of the signer's profile, and the body is bound in through a Content-Digest header (RFC 9530) computed over the raw body bytes.
A request signature covers @method, @authority, @path, the query, ucp-agent, idempotency-key, and the content headers. Responses cover @status instead of the method.
That component list is the security property worth internalizing. The idempotency key is inside the signature, so a replayed request can't be re-pointed at a new operation without invalidating it.
\nUCP deliberately handles replay protection at the business layer through idempotency keys rather than at the signature layer; the RFC 9421 created parameter is optional. If your governance server treats idempotency as an optional convenience, you've removed the replay control.
At the governance layer, verify:
\nUCP-Agent uses HTTPS.extends dependencies are satisfied.signing_keys.Content-Digest matches the raw body you actually read.For custom endpoints, webhooks, or internal proxy routes, HMAC adds a shared-secret check. HMAC does not replace UCP's asymmetric message-signature model. It's an extra control where both parties already share a secret.
\nA webhook verifier should:
\nIn Laravel terms:
\n$expected = hash_hmac(\n 'sha256',\n $timestamp . '.' . $rawBody,\n $sharedSecret\n);\n\nif (!hash_equals($expected, $receivedSignature)) {\n abort(401, 'Invalid signature');\n}\nNever verify an HMAC over a parsed and re-serialized JSON object. Whitespace and key ordering change the bytes. Verify the original payload.
\nAutonomous doesn't mean every action is automatic.
\nUCP models graceful escalation through checkout states such as:
\nincomplete: required information is missingrequires_escalation: buyer input is neededready_for_complete: everything is collected and the agent may finalize programmaticallyThose three sit alongside complete_in_progress, completed, and canceled in the checkout state machine. A checkout can move back and forth between incomplete and requires_escalation, and it can reach canceled from anywhere.
An agent may select a product and calculate shipping but still need buyer input for a 3DS challenge, address validation, terms acceptance, age verification, or custom selling terms.
\nA requires_escalation response must carry a continue_url and at least one message at escalation severity. That's the merchant handing you a place to send the buyer, plus the reason. Send them there with the cart intact instead of starting over.
Your agent should not try to route around this state. It should surface the escalation messages verbatim and preserve the exact checkout context the merchant returned.
\nThe same applies to compliance disclosures. A warning message with presentation: \"disclosure\" (Prop 65, allergens, age restrictions) must be displayed next to the item it applies to, never collapsed or auto-dismissed. If your surface can't render it faithfully, escalate to continue_url rather than silently downgrading.
Add approval gates for:
\nUse idempotency keys on every money-moving request. A retry must not create a second checkout or a duplicate order.
\nAn agents.md file helps compatible tools discover how your application expects agents to behave. Treat it as documentation and policy guidance, never as proof of identity or authorization.
A useful file describes safe defaults, tool boundaries, and failure handling:
\n# Commerce agent guidance\n\n## Safe defaults\n\n- Use catalog tools for product discovery.\n- Never request or expose Shopify Admin tokens.\n- Ask for buyer confirmation before checkout completion.\n- Respect `requires_escalation` responses.\n\n## Tool boundaries\n\n- Do not call arbitrary GraphQL.\n- Do not infer inventory when the API returns unknown.\n- Do not modify orders without an authenticated buyer session.\n\n## Failure handling\n\n- Retry read operations with backoff.\n- Use idempotency keys for writes.\n- Log tool name, request ID, policy decision, and result status.\nKeep it version-controlled and review changes like code. Sign or hash the policy bundle if your architecture supports it. A compromised instruction file must never grant permissions the server itself doesn't enforce.
\nFor more on MCP tool boundaries and context control, see Claude MCP and real-time developer tools.
\nThe same least-privilege rules apply to commerce agents, and the agentic workflow architecture guide covers human approval and stateful tool loops.
\nSecure agentic commerce is an API and policy engineering problem, not a prompt design problem. Before launch:
\nagents.md as guidance, never as authorization.The opportunity in agentic commerce is large, but trust decides which implementations survive production. If you want a second pair of eyes on that boundary, here's how I work with teams.
\n\n- Keep the agent on narrow, schema-validated tools behind a governance server. Admin tokens stay server-side.\n- Validate UCP profiles, pin versions, and verify signatures and content digests on every request.\n- Idempotency keys are your replay control. Treat them as required.\n- Model checkout as states, and send the buyer to `continue_url` when escalation is needed.\n\nWhat authorization boundary stops your Shopify agent from turning a reasonable buyer request into an unauthorized transaction?
\nThe missing piece in most multi-agent systems is not model quality. It is a communication standard.
\n\n- MCP connects agents to tools and data. A2A connects agents to each other.\n- ACP, the third name you will see in older posts, has merged into A2A.\n- Most agents play both roles at once: they use tools over MCP and answer peers over A2A.\n\nYou are ready to move past single-prompt chatbots and build a real multi-agent system. Then you look at what you actually have: agents that cannot reach your Postgres database without a bespoke script, and cannot coordinate with the agent another team shipped last quarter without a shared Slack channel and a human in the middle.
\nWithout a shared standard, you spend most of your engineering time writing glue code for integrations and very little on the behavior that makes the agents worth running. Every schema change breaks a hand-rolled adapter somewhere.
\nThe good news is that the landscape has converged. Two protocols now cover the two axes that matter: the Model Context Protocol (MCP) and Agent2Agent (A2A). The Agent Communication Protocol (ACP) has been folded into A2A.
\nThis post breaks down what each layer does and how to wire them together.
\nIn early 2025 you had to pick a vendor SDK and live with it. Function calling looked different on every model, and agent frameworks each invented their own message envelope. Two things changed that.
\nFirst, Anthropic donated MCP to the Linux Foundation, where it now sits under the Agentic AI Foundation (AAIF) alongside other agentic projects. Second, Google donated A2A to the Linux Foundation in mid-2025.
\nBoth protocols are now governed in the open by the same umbrella, which is the main reason the stack finally stopped moving under everyone's feet.
\nThe second shift is architectural. You no longer need one protocol that does everything. You use a vertical protocol for reaching data and tools, and a horizontal protocol for agents talking to peers.
\n| Protocol | \nPrimary role | \nGovernance | \nStatus | \n
|---|---|---|---|
| MCP | \nAgent to tool/data | \nLinux Foundation, AAIF | \nDe-facto standard | \n
| A2A | \nAgent to agent | \nLinux Foundation | \nStable, actively spec'd | \n
| ACP | \nAgent messaging | \nLegacy (IBM / BeeAI) | \nMerged into A2A | \n
MCP is the vertical layer. It standardizes how an agent connects to everything that is not another agent: your Shopify store, your Postgres instance, a filesystem, an internal REST service.
\n\nAny MCP-compatible client (Claude, an IDE assistant, a local model runner) connects and discovers the same capabilities without you rewriting anything.
\nWhy MCP won the tool layer:
\nstdio for local servers, Streamable HTTP for remote ones. (The older HTTP+SSE transport is legacy; new remote servers should use Streamable HTTP.)If you are modernizing how your systems are consumed, exposing your data through MCP is the first move with the biggest payoff. It decouples your tools from your model choice, which is the coupling that hurts most when you swap models.
\nI went deeper on the mechanics in API vs MCP and MCP tool-use for context-aware agents.
\n\nPick one internal API your agents already call through a custom script and wrap it in an MCP server.\n\nMCP handles tools. A2A handles peers.
\nWhen your merchandising agent needs stock levels from an inventory agent owned by another team (one built on a different framework, deployed in a different cluster), that conversation is A2A. This is the horizontal layer.
\nA2A lets agents discover each other, delegate tasks, stream progress, and hand back results.
\nThe anchor of the protocol is the Agent Card: a JSON document, conventionally served at /.well-known/agent-card.json. It declares who the agent is, which skills it offers, which transports and auth schemes it supports, and what input and output types it accepts. Discovery is fetching that file.
What A2A gives you:
\nThis is what makes cross-team agents workable. Building agentic commerce on Shopify already means a checkout flow handing structured instructions to fulfillment logic; A2A is what stops that handoff from being a private contract nobody else can reuse.
\nYou will still hit references to the Agent Communication Protocol. ACP came out of IBM and the BeeAI team as a REST-native approach to agent messaging.
\nIt was good at exactly the thing A2A was weakest at early on: stateful, long-running, asynchronous work.
\nThe problem was arithmetic. Two competing agent-to-agent protocols meant every framework author had to pick one or implement both, which is how ecosystems stall. During 2025 the ACP maintainers moved to consolidate behind A2A under Linux Foundation governance.
\nThe practical consequence is short. ACP is no longer an architectural choice you need to evaluate. Its strong ideas around state management and durable messaging live on in A2A's task model.
\nIf you had ACP on a roadmap, substitute A2A. The concepts map closely and the community is now in one place.
\nThe mental model that keeps this straight:
\nMCP is the device driver, A2A is the network protocol.
\nOne lets a process talk to the hardware it is attached to; the other lets it talk to other machines. You need both, and they do not overlap.
\nAn agent is typically an A2A server to its peers and an MCP client to its tools, at the same time. That dual role is the whole architecture.
\nA workable implementation order:
\nA tool call and an agent delegation are not interchangeable, even when both are technically reachable.
\nExposing another team's agent as an MCP tool flattens away the task lifecycle and turns a long-running negotiation into a request that either returns or times out.
\nUse MCP for deterministic capabilities you own, and A2A when the other side needs to reason, ask you a follow-up question, or take twenty minutes.
\nMoving to a protocol-first architecture is mostly a change in what you consider the unit of work. Stop shipping \"an agent\" and start shipping services that speak these standards.
\nAudit your tools first. Pick the five tools your agents actually reach for and build or adopt MCP servers for them. Resist wrapping everything: an agent handed forty tools makes worse choices than one handed six.
\nWrite Agent Cards early. Even with two agents, defining the card forces you to name each agent's skills and boundaries. Most multi-agent designs fail because responsibilities overlap, and the card is where that shows up before it becomes a runtime problem.
\nStandardize transports. Streamable HTTP for remote MCP servers, HTTP with SSE for A2A streaming. Streaming matters more than it looks: without it you cannot see intermediate reasoning or partial tool output, and debugging a multi-agent failure becomes archaeology.
\nVersion the contracts, not the prompts. Agent Cards and tool schemas are public interfaces now. Treat a changed tool signature like a breaking API change, because for every agent calling it, that is exactly what it is.
\nInstrument at the boundary. Every MCP call and A2A task should emit a trace with a correlation ID that survives delegation. When a five-agent chain returns nonsense, the only cheap way to find the bad hop is a trace that spans all of them.
\n\nWrite an Agent Card for one agent you already run, even if nobody calls it yet. Naming its skills will show you where responsibilities overlap.\n\nIf you are wiring MCP and A2A into a real product, here's how I help teams ship it.
\n\n- **MCP is vertical.** It connects agents to tools, databases, and APIs, and it is the de-facto standard for that layer.\n- **A2A is horizontal.** It handles discovery, delegation, and task lifecycle between agents, across frameworks and teams.\n- **ACP is legacy.** It merged into A2A. Do not start new work on standalone ACP.\n- **Governance is settled.** Both protocols now sit under Linux Foundation stewardship, which is what makes them safe to build against.\n- **Each agent plays two roles.** A2A server to its peers, MCP client to its tools: that dual role is the stack in one sentence.\n- **The boundary matters.** Deterministic capability you own is a tool; another team's reasoning system is a peer. Mixing them up is how the architecture rots.\n\nHow are you handling coordination between agent frameworks in your production stack right now?
\nScaling a web app used to mean signing a blank check. Coolify is why it doesn't have to anymore.
\n\n- Coolify gives you a Heroku-style workflow on your own servers, built on plain Docker.\n- Dokploy is now a real competitor. Coolify's edge is time in production, not catalog size.\n- Coolify does not manage your firewall. Docker bypasses UFW, so filter at your provider instead.\n\nYou start on the free tier, traffic spikes, and suddenly the bill has a comma in it.
\nThe alternative (a raw VPS, SSH, and a prayer) costs you the weekend instead of the money. Nginx vhosts, certbot renewals that silently fail, a deploy script nobody wants to touch.
\nThat gap is the whole reason Coolify exists, and it's why the project has quietly turned into the default answer for \"I want Heroku, but on my server.\"
\nAs of today the repo sits at 60,298 GitHub stars and ships hundreds of one-click services. Version 4 has been the stable line for a while now.
\nIt is no longer a hobby project you're brave for running. It's infrastructure with a bus factor above one.
\nI run production workloads on it. This post is what I'd tell you over coffee: what the numbers mean, where Coolify genuinely wins, and which of its marketing-adjacent claims you should quietly ignore.
\nStars are a vanity metric right up until they aren't. What the curve tells you here is not \"Coolify is good software.\" It's that a critical mass of developers decided sovereign infrastructure was worth the tradeoff.
\nThe reasoning is boring and correct. A managed PaaS owns your build pipeline, your runtime, your egress pricing, and your migration path. When any one of those changes, you refactor on their timeline.
\nHeroku killing free dynos taught a generation of developers that \"managed\" means \"someone else decides.\"
\nSelf-hosting used to mean giving up the good parts: git-push deploys, automatic TLS, a dashboard your teammate can read. Coolify's actual contribution is that it kept those and dropped the rent.
\nPractically, a mature project means the things that bite you at 2am are handled by someone other than you: proxy config, certificate renewal, container lifecycle, backup scheduling. That's the product.
\nUnderneath the dashboard, Coolify is an orchestrator for Docker. Every app, database, and service you deploy is a container. Your local environment and your production server run the same image, which is the entire point.
\nWhen you push to your repo, Coolify pulls the code, builds an image, and starts the replacement container.
\nBuild sources are flexible: a Dockerfile if you have one, a docker-compose.yaml for multi-service stacks, static output for a plain frontend, or Nixpacks when you want zero config and can live with its opinions.
What containerization buys you here:
\nIf you're weighing this against a full orchestrator, my breakdown of Docker vs Kubernetes covers where the line actually sits, and it's further out than most teams assume.
\nKubernetes earns its complexity somewhere north of \"a dozen services and a platform team.\" Below that, Coolify's model is strictly less work for the same outcome.
\nFor the ground-floor version of this setup, I wrote it up in Coolify and Docker for SaaS hosting.
\nThis is the part people underrate. The service catalog is not a gimmick. It's the difference between \"I'll set up Plausible this weekend\" and \"Plausible is running, next task.\"
\nWorth splitting into two things that get conflated.
\nDatabases are first-class resources, not templates. There are eight of them: PostgreSQL, MySQL, MariaDB, MongoDB, Redis, DragonFly, KeyDB, and ClickHouse. They get their own connection UI, their own lifecycle, and (for the SQL and document stores) their own backup scheduling.
\nEverything else is a compose template, and that's where the breadth lives:
\nEach template ships with sane defaults, networking, volume mounts, and generated secrets. You get a running service and a URL.
\nThe \"280+\" figure is Coolify's own tagline and it undersells the current state: the repo's templates/compose directory holds 362 of them today.
Worth knowing, though, that catalog size is no longer a moat. Dokploy ships 514 blueprints and CapRover has 354 one-click apps. Everyone in this space has enough templates.
\nTwo caveats worth stating plainly. First, a one-click template is a starting point. It is not tuned for your load, and the defaults are chosen to boot cleanly, not to survive traffic.
\nSecond, templates are community-maintained; pin your image tags rather than riding latest into a breaking upgrade.
The self-hosted PaaS space stopped being a one-horse race. Here's where the four main projects actually stand today:
\n| Feature | \nCoolify | \nDokploy | \nCapRover | \nDokku | \n
|---|---|---|---|---|
| Interface | \nWeb dashboard | \nWeb dashboard | \nWeb UI | \nCLI-first | \n
| Multi-server | \nNative + Swarm | \nNative + Swarm | \nDocker Swarm | \nVia plugins | \n
| Service catalog | \n362 templates | \n514 blueprints | \n354 one-click apps | \nPlugin-based | \n
| Build sources | \nDockerfile / Compose / Nixpacks / static | \nDockerfile / Compose / Nixpacks / Railpack / Heroku + Paketo buildpacks | \nDockerfile / captain-definition | \nHerokuish buildpacks / Dockerfile | \n
| Backups | \nS3-compatible, scheduled | \nS3-compatible | \nManual / scripted | \nPlugin-based | \n
| GitHub stars | \n60.3k | \n36.5k | \n15.1k | \n32.1k | \n
A few honest reads on that table.
\nDokploy is the real competitor now, not a footnote. It has 36k stars and is climbing fast, with a template catalog that has quietly grown larger than Coolify's and a wider set of build sources (Railpack and Paketo on top of the usual Nixpacks and Dockerfile).
\nIt's leaner and the UI is arguably cleaner. If you're starting fresh in 2026 it deserves a genuine evaluation rather than a dismissal. Coolify's remaining edge is time in production: more people have hit more edge cases, and those fixes are already merged.
\nDokku has the smallest resource footprint of the four and the most Unix-native design. If you're comfortable in a terminal and want something that will still work identically in five years, it's a defensible choice. The CLI-first model is a real barrier for teams where not everyone deploys.
\nCapRover is the oldest of the four and still actively maintained; it committed as recently as this week. It's Swarm-native by design, which is a genuine advantage if clustering is your starting requirement rather than an afterthought. Its ceiling is the smaller community: fewer people hitting problems means fewer problems already solved for you.
\nCoolify supports three topologies: a single server, multiple standalone servers managed from one dashboard, and Docker Swarm clusters.
\nThe middle one is what most teams want, and it's worth being precise about what it is. Adding a second server means Coolify SSHes into it and manages Docker there. Each server runs its own containers and its own proxy.
\nIt is not a cluster. There's no automatic failover, no scheduler moving workloads between nodes. If a server dies, the apps on it are down until you redeploy elsewhere.
\nThat's a feature, not a gap. It's the model that fits the 95% case: put the database on a high-memory box, the app on a high-CPU box, and keep the Coolify instance itself on a small separate server so a runaway build can't take down your control plane.
\nIf you need true clustering, that's what the Swarm mode is for. If you need more than Swarm, you're in Kubernetes territory.
\nI go deeper on the topology decisions, dedicated build servers, and zero-downtime rollouts in scaling SaaS with Coolify.
\nThis is where I'll break with the usual Coolify writeup, because there's a claim floating around that Coolify handles your firewall for you. It does not, and believing it will hurt you.
\nHere's what Coolify actually gives you:
\nencrypted cast, so they're not sitting in the database as plaintext. That cast uses Laravel's standard encrypted payload, the same format my Laravel decrypt tool reads with an APP_KEY. Scopes go per-resource, with shared variables at the environment, project, and team level, and separate build-time and runtime visibility.owner, admin, and member per team, so a contractor doesn't get production database access.Here's the part nobody says out loud:
\nDocker publishes ports by writing NAT rules that bypass UFW.
\nIf you ufw deny 5432 and then map Postgres to the host, that port is open to the internet and ufw status will happily tell you it's blocked. This is Docker behavior, not a Coolify bug, but the consequence lands on you either way.
The fix is to filter at a layer Docker can't route around:
\nDo those three and your self-hosted setup is meaningfully harder to reach than most managed deployments, because there's less surface exposed to begin with.
\n\nFrom a machine outside your network, scan your server's public IP for open database ports. If 5432 answers, fix it at the provider firewall today.\n\nStrip away the features and the case for Coolify is one sentence: you stop renting your deployment pipeline.
\nThe math is not subtle. Ten small projects on a managed platform (a few hobby-tier databases, some build minutes, a bit of bandwidth) lands somewhere around $150 to $250/month once you're past free tiers.
\nThe same ten projects fit on a single mid-tier VPS with 8 vCPU and 16 GB of RAM, with headroom to spare. On Hetzner that's a low-double-digit euro bill on the ARM line, roughly double on x86. Either way it's an order of magnitude, not a percentage.
\nYou are now the on-call engineer. Kernel updates, disk space, a container that wedges at 3am: that's yours. Coolify reduces that work by maybe 90%, not 100%.
\nFor a solo developer or a small team, the trade is usually worth it. For a company where an hour of downtime costs more than a year of managed hosting, it isn't, and you should pay the tax.
\nI made this exact switch and wrote up the numbers in why I'm ditching expensive cloud providers. The runway extension was the point; the control turned out to matter more.
\nIf you're considering the move, the order matters more than the tooling:
\nCoolify's real achievement is that it made the PaaS experience a commodity. The dashboard, the git integration, the automatic TLS: these were competitive moats five years ago.
\nNow they're a curl | bash away, and the only thing you're paying a managed provider for is not thinking about servers. That's a legitimate thing to pay for. It just isn't worth a 10x premium anymore.
What's still keeping you on managed hosting: the operational load, or the fear of being the one holding the pager? Tell me which one it is.
\nAI coding is strangely addictive.
\n\n- AI coding grows in levels: gateway prompts, then codebase context and RAG, then multi-agent loops over MCP.\n- Context is what changes the output. Specs, tests and pgvector memory beat clever prompts.\n- Split the agents so the one writing the code is never the only one grading it.\n\nYou start with a single prompt to fix a regex or center a div. You tell yourself it's a one-time thing to save ten minutes.
\nThen you watch it refactor an entire Laravel controller in seconds. Suddenly you're asking it to write your unit tests. Before long you're building custom agents that manage a Shopify store over GraphQL.
\nThe shift happens slowly at first. Then all at once.
\nThe problem it solves for most of us isn't typing speed. It's blank-page fatigue. Staring at a legacy codebase or a fresh architecture doc feels like a mountain of manual labor.
\nYou burn hours on boilerplate instead of business logic. That friction is where motivation goes to die.
\nAnd the cost of not evolving compounds. Coding without AI in 2026 feels like writing a book with a quill while everyone else runs a word processor. It isn't only speed. It's cognitive load. Energy spent on syntax and API nuance is energy not spent on architecture and strategy.
\nThe answer isn't \"use AI.\" It's building a structured AI coding workflow: moving from passive user to engineer who orchestrates systems. Once you've felt a multi-agent loop close on its own, there's no going back.
\nAlmost everyone starts here. You treat the model like an interactive Stack Overflow. Paste a snippet, ask for a fix, move on.
\nAt Level 1 you're generating small functions and boilerplate. At Level 5 you're debugging. Instead of manually scanning logs, you drop a stack trace into a chat and get a diagnosis. That alone saves real hours.
\nBut you're still stuck in the copy-paste loop. You write the code. The AI patches the errors. It's a helpful assistant that has no idea what you're building.
\nThe trap at this level is letting the model freestyle without context. Escaping it means writing specs instead of wishes. You stop saying \"fix this\" and start saying something like the right side here:
\n\nSame model. Completely different output.
\nAt this stage you realize the model is only as good as the information it can see. Isolated chat windows stop being enough.
\nYou start using tools that read your whole codebase. This is the point where the AI vs traditional development question stops being philosophical and turns into a workflow decision.
\nLevel 10 is tests. Writing unit tests is usually the chore you postpone; AI makes it fast. A full Pest or PHPUnit suite for a Laravel feature drops in seconds.
\nYou're not just producing code anymore. You're producing a safety net that lets the agents at higher levels move fast without wrecking things.
\nLevel 20 is when the AI actually knows your codebase. Codebase indexing, embeddings, RAG. The model retrieves the three files that matter instead of guessing from the file tree.
\nMany teams go further and stand up a local pgvector database holding internal docs, ADRs, and runbooks. It's the same retrieval discipline behind the 7 mistakes wrecking your production RAG stack.
\nThe payoff is that you stop re-explaining your architecture in every session. Your event-driven layer, your custom service container bindings, your weird legacy billing table: the agent looks it up.
\nThis is where the addiction matures into an engineering practice. You're no longer chatting with a model. You're building systems that act on your behalf.
\nLevel 50 is the Model Context Protocol. MCP lets agents connect to real tools: the Shopify Admin GraphQL API, your database, your cloud infrastructure.
\nIt's the same plumbing behind agentic commerce on Shopify, pointed at your own workflow instead of at a shopper. An agent can watch incoming orders and adjust inventory according to rules you described in plain language, because it holds an actual connection instead of a description of one.
\nThe Level 50 loop I actually run looks like this:
\nYou become the conductor. You own the what and the why; the agents grind through the how.
\nThe separation in step 3 matters more than it looks.
\nAn agent that writes both the implementation and its tests will happily write tests that pass against its own bug.
\nTwo agents with different context catch what one agent rationalizes away.
\nThat is why I stopped reading diffs line by line and moved to a multi-agent review pipeline, and why AI-generated code still needs its own test discipline.
\nOnce you've run an agentic workflow, the friction of manual coding becomes physically annoying. Five-minute tasks feel like an eternity.
\nHunting the correct GraphQL mutation shape for a Shopify app feels absurd when an agent with schema access resolves it in two seconds.
\nThe addiction isn't to the tool. It's to flow. AI removes the micro-frictions that shatter concentration: the doc lookup, the typo, the forgotten import, the \"wait, what's this method called again.\" You stay in a high-level creative state for longer stretches, so you solve bigger problems.
\nIt also quietly redefines what senior means. Not knowing every syntax detail from memory. Knowing how to decompose a system, and how to steer an AI into building it correctly. Builder to architect.
\nYou don't drift into Level 50. You have to build the infrastructure for it.
\nStart with custom MCP servers for the tasks you repeat weekly. If you work on Shopify apps, expose a tool that queries your store's Admin GraphQL API directly, and another that validates a mutation against the current schema version.
\nThat single step kills most of the copy-paste in your day. Keep the surface small. Every extra tool is context tax and one more thing the agent can misuse.
\n\nPick one task you repeat every week and wrap it in a single small MCP tool. Use it for a week before adding a second one.\n\nDon't rely on general knowledge for project-specific questions. Index your project.
\nIf you're running a Laravel SaaS, make sure the agent can retrieve your service-layer conventions and your event contracts, not just guess from app/. Storing documentation embeddings in pgvector is the cheapest long-term memory you'll ever buy, and it lives in the Postgres you already run.
Stop thinking about one AI and start thinking about a team. Planner, implementer, reviewer.
\nSeparation of concerns produces better code and fewer hallucinations, for the same reason it does with humans: the reviewer isn't emotionally invested in the plan.
\nShopify has moved on from REST for new development. Make sure your agents target the current GraphQL Admin API version and don't reach for deprecated fields they memorized from a 2023 blog post.
\nPin the API version explicitly in your prompts and your client. Schema drift is the single most common source of confidently wrong Shopify code.
\n// Example of a Laravel service using a Shopify GraphQL mutation.\n// An agent writes the boilerplate; your spec dictates the contract.\npublic function updateProductInventory(string $inventoryItemId, int $newQuantity)\n{\n $mutation = <<<'GRAPHQL'\n mutation inventorySetQuantities(\n $input: InventorySetQuantitiesInput!\n $idempotencyKey: String!\n ) {\n inventorySetQuantities(input: $input) @idempotent(key: $idempotencyKey) {\n inventoryAdjustmentGroup {\n createdAt\n reason\n changes {\n name\n delta\n quantityAfterChange\n }\n }\n userErrors {\n code\n field\n message\n }\n }\n }\n GRAPHQL;\n\n $variables = [\n 'idempotencyKey' => (string) \\Illuminate\\Support\\Str::uuid(),\n 'input' => [\n 'name' => 'available',\n 'reason' => 'correction',\n // Required on current Admin API: compare-and-set. null skips the check.\n 'quantities' => [\n [\n 'inventoryItemId' => $inventoryItemId,\n 'locationId' => config('shopify.default_location_id'),\n 'quantity' => $newQuantity,\n 'changeFromQuantity' => null,\n ],\n ],\n ],\n ];\n\n return $this->shopifyClient->query($mutation, $variables);\n}\nTwo details in there are exactly the kind of thing schema drift breaks.
\nThe @idempotent directive arrived as optional in API version 2026-01 and became required in 2026-04. Any agent working from a 2024 blog post will emit this mutation without it and get rejected outright.
Worse, a model that half-remembers the shape will reach for inventorySetQuantity (singular), which isn't a mutation in the current schema at all.
And note userErrors. Shopify returns a 200 with a populated userErrors array for most business-rule failures, so an agent that only checks HTTP status will report success on a write that silently did nothing.
Bake both rules into your spec once and every generated mutation inherits them.
\n\nAdd two lines to your agent spec: \"always send `@idempotent` with a key\" and \"treat a non-empty `userErrors` as a failure.\" Then regenerate one mutation and compare.\n\n\n- **Start small, aim high.** Level 1 prompts are a doorway, not a destination. The value lives in the loops.\n- **Context is king.** RAG and pgvector give the model specific knowledge of your codebase and business rules instead of internet averages.\n- **Embrace MCP.** Real tool connections (Shopify GraphQL, your database, your infrastructure) beat pasted screenshots every time.\n- **Split the agents.** Whoever writes the code should not be the only one grading it.\n- **Standardize the loop.** Plan, execute, verify, deploy. The structure is what keeps quality from collapsing as speed goes up.\n- **Focus on architecture.** As implementation gets cheap, your value moves to design, constraints, and judgment.\n\nThe hardest part of AI coding isn't the learning curve. It's realizing you can't go back. You've tasted a different level of efficiency and the old way now looks like manual labor.
\nThis isn't about replacing yourself. It's about amplifying yourself until you're doing the work of a team.
\nIf you want to compare agent stacks, get in touch.
\nHow are you structuring your AI coding workflow beyond simple chat prompts?
\nNobody wants to read a 5,000-line diff.
\n\n- Human review works in chunks of a few hundred lines. Agent diffs blow past that, so line-by-line reading turns into theater.\n- Move the gate to specs, property tests, contract tests and machine-checked rules, then let narrow review agents run in CI.\n- I still open migrations, auth and permissions, money paths, new seams and anything nobody asked for.\n\nBut that is what lands in the pull request when an agent spends an afternoon refactoring a legacy service or scaffolding a new Laravel module end to end.
\nThe failure mode that follows is worth naming precisely, because it does not look like failure while it is happening.
\nAn agent scaffolds a multi-tenant module in one pass: migrations, models, policies, jobs, tests. The diff is correct, mostly. You spend two evenings reading it line by line and approve it.
\nWeeks later something breaks in production: a queued job that ran outside the tenant scope. You read that file. You read that exact method. Your eyes went over it and your brain filed it as \"looks like the other ones.\"
\nIt feels like diligence. It is theater.
\nCode review guidance has been consistent for two decades: reviewers find defects reliably in chunks of roughly 200 to 400 lines, and effectiveness drops sharply past that.
\nNothing about that number has changed because we bolted an LLM onto the front of the process. Human attention is the constant. The volume of code is the variable, and it just went up by an order of magnitude.
\nThree things go wrong when you push past that limit:
\nNone of this is new. What changed is that we used to write code slowly enough that review capacity roughly matched output.
\nThat coupling is gone. Generation scales with tokens; reading scales with eyeballs.
\nA review agent pointed at a repository does not read files the way you do. Given the right tooling it walks the AST, follows call graphs, and traces a value from a controller through a service into a model and out to an external API.
\nIt does that for file forty with the same attention it gave file one.
\nThat is the real advantage: breadth and consistency, not intelligence. An agent will happily check all 340 controller actions for a missing authorization call. You will check the twelve you remember.
\nI want to be equally clear about the other side, because \"agents review better than humans\" is a bad summary of what I actually believe:
\nSo the move is not \"trust the agent instead of your eyes.\" It is: give the agent the work that scales with volume, and keep for yourself the work that scales with judgment.
\nThat split is the same one I drew in AI vs traditional development. The boundary moved; it did not disappear.
\n| Dimension | \nHuman line-by-line review | \nAgentic verification | \n
|---|---|---|
| Effective scope | \nA few hundred lines before quality drops | \nWhole repository, via AST and call-graph traversal | \n
| Consistency | \nDegrades with fatigue and diff size | \nSame checks applied to file 1 and file 400 | \n
| Typical focus | \nStyle, naming, local logic | \nInvariants, contracts, types, test outcomes | \n
| Failure mode | \nMisses cross-file interactions, rubber-stamps | \nConfident false positives, misses product intent | \n
| Cycle time | \nHours to days | \nMinutes, on every push | \n
If you are not reading the code, something else has to hold the line. For me that is the specification layer, and writing it now pays off more than anything else I do on an agent-assisted project.
\nConcretely, that means:
\nThat last one is the real trick. Every review comment I would type twice becomes a rule instead.
\nIf the agent can violate an invariant and the pipeline stays green, that is my bug, not the agent's.
\nThe numbers behind why this layer is non-negotiable, and the CI configuration I run, are in test AI code too.
\nWhen a check fails, I do not hand-patch the file. I fix the prompt, tighten the spec, or add the missing rule, then re-run.
\nThat loop is the practical core of agentic workflows, and it only works if the feedback the agent receives is precise. Vague context produces vague code, the point I made in prompt engineering vs context engineering.
\n\nScroll through your last ten review comments. Any comment you wrote twice is a candidate for a test, a static-analysis level or an architecture test.\n\nOne giant \"review this PR\" prompt produces mush. Narrow agents with narrow briefs produce findings you can act on.
\nThis is the verify stage of the plan, execute, verify, deploy loop, and it deserves its own agents for the same reason it deserves its own step. The setup I run now has three, and they all execute inside CI rather than in my editor:
\nEach agent needs real access to be useful: the schema, the logs, the actual test output, not a pasted snippet. That is exactly the problem MCP servers solve, and it is the difference between a reviewer that guesses and one that checks.
\nFirst, findings are severity-ranked and anything below \"would break in production\" goes to a comment, never a block.
\nSecond, a second agent tries to refute each finding before I see it. A large share of the initial findings die there, and they are exactly the ones that would otherwise burn an afternoon.
\nThe pipeline is only worth building if it closes without me in the middle of it:
\n[Spec + tests] --> [Agent generates] --> [Static analysis + type checks]\n ^ |\n | v\n [Self-correction] <-- [Test suite + review agents]\n |\n v (all green)\n [Human: seams review] --> [Merge / deploy]\nFailures feed back into the agent's context automatically. I only see the branch once it is green.
\nThat is the same discipline that makes CI and CD work in the first place: the pipeline is the gate, not the reviewer's inbox.
\n\"Stop reading code\" is a slogan, and taken literally it is wrong. Here is my actual list of things I open every time, no matter how green the pipeline is:
\nThat list is maybe 3% of a large diff. Reading 3% carefully beats skimming 100%, and it is the part where my decade of production scars actually transfers.
\nThe rest (the 340 controller actions, the DTO fields, the test scaffolding) is machine work, and I stopped pretending otherwise.
\n\nOn your next large agent PR, open only the files on that list. Note how much of the diff that covers.\n\n\n- **Line-by-line review does not scale to agent-sized diffs.** Past a few hundred lines you are performing diligence, not doing it.\n- **Use agents for breadth, not judgment.** Cross-file tracing and repeated checks are their edge; product intent and architectural taste are not.\n- **Convert review comments into rules.** Anything you would say twice belongs in a test, a static-analysis level, or an architecture test.\n- **Split the review agent into narrow briefs** (correctness, security, regression) and make a second pass try to refute each finding before it reaches you.\n- **Keep a short human read list:** migrations, auth, money, new seams, unrequested changes.\n\nThe uncomfortable question is what your team's review process is actually for. If it exists to catch defects, most of it should be automated by now. If it exists to spread knowledge, say that out loud and redesign it around that goal instead.
\nWhere does your review process still depend on someone reading every line, and what would it take to turn that into a check? Tell me via contact, I collect these. 🤘
\nThe era of manual syntax is over.
\n\n- Vibe coding is for discovery: fast prototypes and spikes whose code you throw away.\n- Agentic engineering is for production: a written spec, tests and agents reviewing each other's diffs before anything merges.\n- Split the day between the two, and never let the morning branch merge.\n\nIn 2026 the question is no longer whether AI writes your code. It's how you govern the AI that does.
\nMost developers I talk to are caught between two modes: the high-speed thrill of vibe coding and the slower discipline of agentic engineering.
\nBoth are real. Both work. But they don't work for the same job.
\nVibe coding will conjure a working application in an afternoon. Agentic engineering is what makes that application survive contact with production. Shipping in 2026 means knowing which one you're in at any given moment, and stopping the vibes before they reach main.
Vibe coding was the breakout habit of 2025. It's a conversational approach where the feel of the interaction takes precedence over the underlying architecture.
\nYou describe a feature, the model returns a large block of code, you run it to see whether it works. It breaks, you paste the stack trace back, you ask for a fix. Repeat.
\nThat cycle is fast. It's addictive. It feels like magic right up until you try to scale a system built entirely on vibes.
\nThe real problem isn't the code quality; models are good now. The problem is that vibe coding is development without verification.
\nWhen the model hands you 500 lines and you ship because it looks right, you're operating on faith.
\nYou've moved the bottleneck from typing to reviewing, and then you skipped the reviewing.
\nNone of that makes it useless. The first pass of a Shopify storefront section or a Laravel reporting dashboard can come together in an afternoon this way.
\nThe trouble shows up six weeks later: no tests, three competing patterns for the same problem, and a Controller doing the job of a service, a job, and a repository.
That's not a code problem, it's a process problem. It's the same failure mode I wrote about in why projects need more than just logic.
\nAgentic engineering is the professional evolution of the same tooling. It isn't chatting with a single model. It's running a system of agents that plan, implement, test, and review under your supervision. Your role changes from coder to architect of intent.
\nIn an agentic workflow you supply a specification, not a vibe: the requirement, the architectural constraints, the acceptance criteria.
\nThe system decomposes it. One agent researches the API surface. One writes the implementation. One writes the tests. One reads the diff looking for reasons to reject it.
\nThe spec is the whole trick, and it's usually shorter than people expect:
\n## Task: subscription pause endpoint\n\nConstraints\n\n- Laravel 12, existing `SubscriptionService`, no new packages\n- Idempotent: repeat calls with the same `Idempotency-Key` return the first result\n- Authorization via existing `SubscriptionPolicy@pause`\n\nDone when\n\n- Feature test: pause, double-pause, unauthorized, already-cancelled\n- No changes to `routes/api.php` ordering\n- `php artisan test` and `vendor/bin/pint --test` both green\nYou're no longer reviewing lines, a habit that stops scaling at agent-sized diffs. You're reviewing a pull request that other agents already argued about.
\nThe human decision shrinks to go/no-go, which is exactly where a human decision is worth the most.
\nThat's the architecture side of it, and I go deeper into the MCP and loop mechanics in the shift to agentic workflows.
\nMost AI-assisted projects that stall share one root cause: vibe coding applied to a problem that needed agentic engineering.
\n| Dimension | \nVibe coding | \nAgentic engineering | \n
|---|---|---|
| Primary goal | \nSpeed and exploration | \nCorrectness and scale | \n
| Your role | \nConversational partner | \nArchitect, mission control | \n
| Verification | \n\"Does it seem to work?\" | \nAutomated tests plus peer-agent diff review | \n
| Structure | \nLoose, improvised | \nWritten specs, repeatable SOPs | \n
| Best for | \nPrototypes, spikes, one-offs | \nProduction SaaS, Shopify Plus apps | \n
| Tech debt | \nHigh and invisible | \nManaged and visible | \n
| Failure mode | \nSilent wrongness | \nSlow starts, over-specification | \n
Note the last row. Agentic engineering has its own failure mode: you can spend an hour writing a spec for something you could have vibed in ten minutes and thrown away.
\nDiscipline applied to a throwaway prototype is just waste with better manners.
\nThe people shipping fastest in 2026 don't pick a side. They split the day. Creative exploration in the high-energy hours, engineering rigor in the disciplined ones.
\nIf you want the rung-by-rung version of how a workflow gets there, I mapped the levels from gateway prompts to agentic systems separately.
\nStart with exploration. If I'm adding a RAG-backed feature, I don't open a spec document. I open the editor and vibe out the retrieval logic.
\nTry pgvector, try a hosted index, see how the chunking feels against real documents. Wire Claude and MCP into the local environment and let it poke at the database directly.
\nThe output of the morning is not code you keep. It's a decision you keep. Nothing from this session is allowed to merge.
\nOnce the shape is proven, the vibes end. The prototype becomes the input to a spec: schema, API contract, error semantics, test coverage.
\nThen the agents refactor that morning code into something documented and tested, and a review agent tries to break it.
\nThe hard rule that makes this work: the morning branch never merges. It gets read, summarized into a spec, and deleted.
\nIf you let vibe-coded code merge \"just this once,\" you've quietly reverted to mode one, and today's speed becomes next month's outage.
\n\nAgents need predictable ground to stand on. My stack is Laravel for application logic and Shopify Plus when the surface is commerce. That's not nostalgia. It's because both are legible to a model.
\nLaravel's conventions are the point. Service providers, policies, form requests, and a canonical directory layout mean an agent can answer \"where does this belong?\" without guessing.
\nShopify's GraphQL Admin API is similarly self-describing: typed, introspectable, versioned. An agent can discover the mutation it needs instead of hallucinating a REST route that was deprecated two versions ago.
\nThat predictability is most of what makes agentic commerce on Shopify tractable.
\nThe other half is memory. Your agents need your codebase, your ADRs, and your incident history. Otherwise every session starts from zero and you re-litigate decisions you settled in March.
\nA vector store over your own repository and docs is the cheapest win in the stack, provided the retrieval layer is honest.
\nGarbage context produces confident garbage code, which is exactly the failure pattern behind most RAG mistakes in production.
\nThe last place vibes leak into production is the deploy. If a human is manually clicking ship after an agent wrote the change, the human is the weakest link. They're approving a diff they didn't write against tests they didn't read.
\nPush the verification into the pipeline instead: migrations checked, feature tests green, a smoke pass against a preview environment, error rate watched for a window after rollout, automatic rollback if it moves.
\nDocker plus Coolify gets you this on your own boxes without a platform bill, which is the whole premise of self-hosting your SaaS.
\nThe goal isn't zero human involvement. It's that a bad vibe can't reach customers without something automated objecting first.
\n\nList every manual step between \"agent opened a PR\" and \"customers see it.\" Each one is a candidate for a pipeline check.\n\n\n- Vibe coding is a discovery tool. Use it for prototypes, UI spikes, and messy ideas, then throw the code away and keep the decision.\n- Agentic engineering is a production tool. Specs, tests, and multi-agent review are what make AI-written code safe to charge money for.\n- The skill that matters in 2026 is writing unambiguous specs. It's the new senior-engineer bottleneck, and it's a writing skill more than a coding one.\n- Give your agents memory. Vector-indexed code, docs, and incident history stop every session from restarting at zero.\n- Split your day, and enforce the boundary. The morning branch does not merge.\n- Watch the opposite failure too: over-specifying a throwaway spike is waste, not discipline.\n\nIf you're trying to get an AI-assisted team from prototypes to production, here's how I help.
\nWhich part of your workflow is still running on vibes alone, and what would an agent find if it audited that code tomorrow?
\nDistributed systems are the most expensive way to solve a problem you do not have yet.
\n\n- Start with one deployable app split into strict modules with contracts, private data and internal events.\n- Enforce module boundaries with architecture tests in CI, and never let modules share tables or JOINs.\n- Split only on real friction: team bottlenecks, divergent scaling needs or repeat fault cascades. Extract workers first.\n\nEvery year a new batch of SaaS founders jumps straight to microservices because they want to \"build for scale.\"
\nSix months later they are debugging network latency and distributed transactions instead of shipping features. The product has not moved. The infrastructure bill has.
\nThe modular monolith has quietly won this argument. It gives you the logical separation you actually need without the operational tax of a distributed environment.
\nThis post covers why you should start there, how to structure it so it stays modular, and the specific signals that tell you it is finally time to split.
\nIf you are still weighing the two patterns at a high level, read monolith vs microservices first. This post assumes you have picked the monolith and want to build one that does not rot.
\nMost startups choose microservices for the wrong reasons. They look at Netflix or Uber and assume the infrastructure is the blueprint for success.
\nWhat they miss is that those companies adopted microservices to solve organizational problems (hundreds of engineers who could not coordinate a single deploy), not purely technical ones.
\nSplit prematurely and you pay a heavy tax:
\nFor a small team this overhead kills velocity. You spend 40% of your time on infrastructure and 60% on product. In a modular monolith that ratio flips back toward the thing customers pay for.
\nA modular monolith is not a big ball of mud with nicer folder names. It is a single deployable unit where code is strictly organized into independent modules with well-defined interfaces between them.
\nIn a Laravel codebase that means each domain gets its own namespace, its own service provider, its own routes, and its own migrations.
\nA Billing module never reaches into the Users tables directly. It talks through a contract.
// app/Modules/Billing/Contracts/CustomerDirectory.php\nnamespace App\\Modules\\Billing\\Contracts;\n\ninterface CustomerDirectory\n{\n public function find(int $customerId): ?CustomerSummary;\n}\n// app/Modules/Users/Adapters/EloquentCustomerDirectory.php\nnamespace App\\Modules\\Users\\Adapters;\n\nuse App\\Modules\\Billing\\Contracts\\CustomerDirectory;\nuse App\\Modules\\Billing\\Contracts\\CustomerSummary;\nuse App\\Modules\\Users\\Models\\User;\n\nfinal class EloquentCustomerDirectory implements CustomerDirectory\n{\n public function find(int $customerId): ?CustomerSummary\n {\n $user = User::query()->find($customerId);\n\n return $user\n ? new CustomerSummary($user->id, $user->email, $user->billing_country)\n : null;\n }\n}\nBilling depends on an interface it owns and a DTO it controls. Users supplies the implementation. Swap that adapter for an HTTP client tomorrow and Billing does not change a single line.
The seam already exists. You just move what is behind it.
\nThe goal is logical separation with physical unity. You get a clean, decoupled architecture without managing a fleet of containers on Coolify or Kubernetes before you need to.
\nThe concept doing the real work here is the bounded context, a term from Domain-Driven Design that marks the boundary where one model of a thing applies.
\nA Product in the Inventory module is not the same object as a Product in the Marketing module. One cares about stock levels, SKUs, and warehouse locations. The other cares about SEO titles, hero images, and merchandising rules.
Forcing them into one bloated Product model is how modular monoliths turn into mud.
Three rules keep boundaries intact:
\nModules\\Billing imports Modules\\Inventory\\Models. In PHP, Deptrac or a Pest arch test does this in a few lines. Any language has an equivalent.Common package that every module depends on is a monolith wearing a costume; you can never split anything out of it.Orders needs Shipping to react, emit a domain event instead of calling a method. Replacing an in-process listener with a message queue later becomes a config change, not a rewrite.That third rule pays off the most. Internal events are the cheapest dress rehearsal for asynchronous communication you will ever get, and they cost nothing while everything still runs in one process.
\n\nAdd one architecture test today that fails when any module imports another module's `Models` namespace. Run it and count the failures.\n\nThe database is where most monolith-to-microservice migrations die.
\nIf your modules share tables or run JOINs across domains, you are coupled at the data layer, the hardest coupling to break. No amount of tidy namespaces saves you when Reporting joins six tables owned by four other modules.
So aim for schema per module. Use one physical Postgres or MySQL instance for simplicity, but give each module its own tables and treat them as private. In practice:
\nBilling needs a customer email, it calls the contract, not the users table.Reporting needs a wide view, have modules publish into a reporting table they explicitly own.Here is the test: if you cannot imagine running your Orders module against a separate database server tomorrow, your monolith is not modular yet. It is a monolith with folders.
You rarely outgrow a monolith because of traffic. A single well-tuned Laravel box with Octane and a queue handles more load than most SaaS companies ever see.
\nYou outgrow it because of friction. Three signals matter:
\nAre developers constantly blocking each other? Twenty engineers merging into one codebase with a deployment queue is a real organizational bottleneck, and that alone can justify a split.
\nFive engineers? A monolith is almost always faster. Conway's Law cuts both ways: do not buy a distributed architecture for a co-located team.
\nDoes one part of the app have radically different resource needs? An AI inference module that wants GPUs while your dashboard is plain CRUD is a real case for extraction.
\nYou stop paying for GPU-sized instances to serve settings pages, and your DevOps spend tracks actual demand.
\nIf a bug in non-critical reporting takes down checkout, you have a resilience problem. First try to solve it in-process: better error handling, background jobs, timeouts, a bulkhead around the risky call.
\nIf it keeps happening, a separate service acts as a circuit breaker around your core revenue path.
\n| Metric | \nModular monolith | \nMicroservices | \n
|---|---|---|
| Deployment | \nSimple (one pipeline) | \nComplex (many pipelines) | \n
| Testing | \nEasy (in-process) | \nHard (end-to-end focused) | \n
| Data integrity | \nStrong (ACID) | \nEventual consistency | \n
| Debugging | \nStack trace | \nDistributed trace | \n
| Operational cost | \nLow | \nHigh | \n
| Team scaling | \nBottlenecks past ~15 to 20 | \nScales with squads | \n
Notice what is missing: raw performance. It is almost never the deciding factor, and when it is, a bigger box or a read replica usually beats a rewrite.
\nWhen the split finally makes sense, your first move should not be a full service.
\nStart with an extracted background worker. Move heavy, long-running tasks (PDF generation, CSV exports, video transcoding, embedding jobs) into their own process consuming from a queue.
\nYou get isolation, independent scaling, and blast-radius reduction without the pain of synchronous API contracts, retries, and versioning.
\nRabbitMQ or a Pub/Sub-style event backbone covers most of this, and the same pattern powers queue-based document processing.
\nOnly after that should you consider carving out a synchronous service. When you do, the strangler fig migration is the safe path: route one endpoint at a time and keep the monolith as your fallback.
\nOne more constraint: wait until your domain is stable. Early-stage products change their data models weekly.
\nHard service boundaries during that discovery phase turn every small feature into a multi-service orchestration project. Boundaries are cheap to move inside a monolith and expensive to move across a network.
\n\n- **Start modular.** Separate business domains with namespaces, modules, and contracts from day one. This costs nothing early and is nearly impossible to retrofit later.\n- **Enforce boundaries with tooling.** Architecture tests in CI, not code-review vigilance.\n- **Own your data per module.** No cross-module foreign keys, no cross-domain JOINs. Data coupling is the coupling that traps you.\n- **Emit events internally.** In-process listeners today become queue consumers tomorrow with a config change.\n- **Split on friction, not fashion.** Deployment bottlenecks, divergent scaling profiles, and repeat fault cascades are reasons. A conference talk is not.\n- **Extract workers first.** Async background processes give you most of the isolation benefits at a fraction of the operational cost.\n\nIf your architecture lets you change your mind later without a rewrite, you have already won. That is the whole point of deferring the decision.
\nHow often are you actually checking for cross-module coupling in your codebase? If the answer is \"never,\" that is the cheapest audit you can run this week, and I am happy to look at it with you.
\nScaling a web application often feels like trying to upgrade a jet engine while the plane is mid-flight.
\n\n- Stateful servers remember users in local memory, which forces sticky sessions and breaks when you add nodes.\n- Stateless servers forget every request, so any node can serve anyone. That is what makes horizontal scaling easy.\n- Shared storage like Redis lets an app feel stateful to users while each server stays stateless.\n\nYou start with a single server that handles every request perfectly. But as your traffic grows, you add more servers, and suddenly everything breaks.
\nUsers are getting logged out randomly. Shopping carts are disappearing into thin air. The database is screaming under the pressure of redundant session checks.
\nThis chaos usually stems from a fundamental architectural decision made in the early stages of development. You are likely struggling with the friction between stateless and stateful design patterns.
\nIf your application relies on the server remembering who a user is based on local memory, you are building a stateful system. This works great for small projects but becomes a nightmare when you need to scale horizontally.
\nOn the other hand, moving to a fully stateless model requires a complete rethink of how you handle authentication and data persistence.
\nYou need to understand the trade-offs between stateless and stateful apps before you can build reliable, fast software in the cloud.
\nBefore comparing the two architectures, we must define what \"state\" actually means in a web context.
\nState is any data that the application needs to remember between different interactions. This could be a user's login status, the items in a Shopify cart, or the progress of a multi-step form.
\nIn a traditional web application, the server manages this state by creating a unique session for every visitor. The server stores a small file or a piece of memory linked to a specific session ID.
\nWhen the user moves from one page to another, the server looks up that ID and retrieves the state. This makes the application \"stateful\" because the server's response to a request depends on what happened in previous requests.
\nA stateless application treats every single HTTP request as a completely new event. The server does not remember anything about the client once the request is finished.
\nA user clicks a button, the server processes the data, sends a response, and immediately forgets that user ever existed.
\nFor this to work, the client must provide all the necessary information with every single request. This is why stateless apps often use JSON Web Tokens (JWT) or API keys.
\nThe token contains the user's identity and permissions. When the server receives the token, it validates it, performs the requested action, and sends back the result.
\nThe primary advantage of this approach is horizontal scalability. Since the server does not store any local session data, any server in your cluster can handle any request.
\nYou can add ten more servers to your stack using a tool like Coolify or Docker and the load balancer can distribute traffic without worrying about where the user's session is located.
\nThis same property is what lets a stateless tier sit cleanly behind an API gateway.
\nStateful applications are built on the premise of continuity. The server maintains a persistent connection or a session record for every active user.
\nThis was the default way of building the web for decades. If you are using Laravel with standard Blade templates and the built-in authentication guard, you are likely running a stateful application.
\nIn a stateful system, the server's internal memory or local disk stores the user's context. This makes development faster and easier for certain features.
\nFor example, \"flashing\" a success message to the next page is trivial in a stateful app. The server just puts the message in the session and reads it on the next load.
\nHowever, statefulness introduces a \"sticky session\" problem. If you have three servers (A, B, and C) and a user's session is stored on Server A, the load balancer must ensure that every subsequent request from that user goes back to Server A.
\nIf the user hits Server B, that server will have no record of them, and they will appear logged out.
\nSticky sessions make scaling significantly more complex and create a single point of failure.
\nIf Server A crashes, all users \"stuck\" to it lose their session data.
\nChoosing between these two paths depends on your specific technical requirements. Below is a breakdown of how they compare across critical engineering metrics.
\n| Feature | \nStateless Apps | \nStateful Apps | \n
|---|---|---|
| Data storage | \nState is stored on the client or a shared database/cache. | \nState is stored in the server's memory or local storage. | \n
| Scalability | \nHigh. Any server can process any request. | \nLow. Requires sticky sessions or complex synchronization. | \n
| Resilience | \nHigh. If a server dies, another one takes over instantly. | \nModerate. Server failure often leads to session loss. | \n
| Complexity | \nHigher initial setup for token management and external caching. | \nLower initial setup for small-scale projects. | \n
| Performance | \nPotential overhead from passing tokens or fetching data from Redis. | \nFaster local memory access but slower overall scaling. | \n
In the world of PHP development, Laravel gives you the flexibility to choose either path. By default, the web middleware group uses stateful sessions.
It stores a session cookie on the browser and matches it to a server-side entry, which since Laravel 11 lands in the database driver out of the box. This is perfect for traditional dashboards where you need simple state management and cookie-based CSRF protection.
To make your Laravel app stateless, you move toward token-based authentication using Laravel Sanctum or Passport.
\nYou skip session cookies on your API routes and let every request carry its own bearer token, validating the user against a shared store like Redis or the database. This shift is essential when building backends for mobile apps or complex React/Vue frontends.
\nThis is also where the stateless model pays off as you grow. Because no node owns a session, you can lean on horizontal scaling and add capacity behind the load balancer without any session-affinity gymnastics.
\n\nOpen your production `.env` and check `SESSION_DRIVER`. If it says `file` and you run more than one node, that is your random-logout bug.\n\nShopify provides an interesting look at how these architectures live together. The Shopify platform itself is a massive distributed system.
\nWhile the \"state\" of a store (orders, inventory, customers) is preserved in giant databases, the web tier that handles incoming traffic is designed to be stateless. This allows Shopify to handle massive spikes during Black Friday by spinning up thousands of web workers that don't need to share local memory.
\nIf you are building agentic commerce solutions on Shopify, your app architecture should lean toward statelessness. Your app will receive webhooks and API calls from various Shopify nodes.
\nIf your app is stateful and relies on local memory to track a multi-step checkout process, you will likely encounter race conditions or lost data as Shopify's infrastructure scales.
\nUsing a shared Redis instance to store temporary job state is the professional way to handle this.
\nYour choice between stateless vs stateful apps will dictate your DevOps strategy. Stateless apps are the \"gold standard\" for containerization.
\nSince they have no local dependencies, you can run them in Docker containers and deploy them to any cloud provider without friction. You don't need to worry about preserving the \"health\" of a specific server instance.
\nStateful apps require much more care. If you are deploying a stateful Laravel app, you must ensure that your session driver is set to something central like Redis or a database.
\nLaravel 11 and later default to the database driver, but if you switch it back to file, each node writes sessions to its own disk and your horizontal scaling will fail.
You also need to consider how you handle deployments. A \"zero-downtime\" deployment is harder when you have thousands of active sessions living in server memory. You often have to wait for sessions to drain before taking a node offline.
\n\nBefore your next deploy, take one node out of the load balancer and watch whether any users get logged out. That tells you how stateful you really are.\n\n\n- **Stateless apps** treat every request as a new event, requiring the client to provide context (tokens) every time.\n- **Stateful apps** remember user interactions through local server memory or session files, making them easier to build but harder to scale.\n- **Horizontal scaling** is the primary driver for choosing a stateless architecture. It allows you to add servers without session conflicts.\n- **Laravel** supports both. Use sessions for simple web apps and tokens (Sanctum) for scalable APIs.\n- **Shared storage** (like Redis) is the bridge that allows an application to feel stateful to the user while remaining stateless at the server level.\n- **Shopify developers** should prioritize stateless designs to handle the platform's distributed nature and high-volume webhooks.\n\nHow are you managing session persistence in your current production environment to ensure horizontal scaling?
\nShipping software is hard.
\n\n- Docker packages your app into a portable image and runs containers on one host.\n- Kubernetes orchestrates many containers across a cluster: scheduling, scaling, self-healing and rolling deploys.\n- Small teams can stay on Docker or Coolify. Reach for Kubernetes when availability and scale demand it.\n\nYou build an app that works perfectly on your machine, but the moment it hits production, everything breaks.
\nThis \"it works on my machine\" syndrome is exactly why understanding the difference between Docker vs Kubernetes is essential for modern engineers.
\nEnvironment inconsistencies create friction, delay releases, and lead to late-night debugging sessions that could have been avoided with the right tooling.
\nThe gap between local development and production scale is wide. One tool packages your application into a portable box; the other manages a fleet of those boxes across a cluster of servers.
\nIf you are shipping modern web or e-commerce applications, you eventually need both, and knowing where each one stops saves you a lot of wasted effort.
\nDocker changed how we think about software delivery by introducing the concept of containerization.
\nA container is a lightweight, standalone package that includes everything needed to run an application. This includes the code, runtime, system tools, libraries, and settings.
\nWhen you use Docker, you are essentially creating an immutable image of your software. This image behaves exactly the same whether it is running on your laptop, a colleague's machine, or a staging server.
\nIt eliminates the need to manually configure environments, which is a major win for developer productivity.
\nDocker operates on a single-host level. It manages the lifecycle of individual containers. You use the Docker CLI to build, run, and stop these containers.
\nFor many small-scale projects or early-stage startups, Docker on a single VPS is often enough. In fact, tools like Coolify for Docker SaaS hosting have made it easy to manage these deployments without needing a massive orchestration layer.
\nIf Docker is about the container, Kubernetes is about the cluster. Kubernetes, often abbreviated as K8s, is an open-source platform designed to automate the deployment, scaling, and management of containerized applications.
\nIt does not compete with Docker. Docker builds the image; Kubernetes schedules and runs it at scale through whatever container runtime sits on each node.
\nImagine you have a high-traffic Shopify app that suddenly experiences a surge in users. Managing fifty individual Docker containers across five different servers manually would be a nightmare.
\nYou would have to track which server has space, handle load balancing, and manually restart any container that crashes.
\nKubernetes handles this automatically. It uses a declarative approach where you define the \"desired state\" of your system in YAML files.
\nThe Kubernetes control plane then keeps the actual state matching what you declared. If a container dies, Kubernetes restarts it. If a node fails, it reschedules the work elsewhere.
\nComparing Docker vs Kubernetes is not about choosing one over the other. They operate at different layers of the stack.
\nDocker is the runtime that builds and runs the container. Kubernetes is the orchestration layer that schedules those containers across a network.
\n| Feature | \nDocker | \nKubernetes | \n
|---|---|---|
| Primary goal | \nContainerizing an application | \nManaging a cluster of containers | \n
| Scaling | \nManual scaling on a single host | \nAutomated horizontal scaling | \n
| Self-healing | \nRestart policies, single host | \nProbe-based restarts and rescheduling | \n
| Networking | \nBasic host-level networking | \nAdvanced cluster-wide networking | \n
| Complexity | \nLow to medium | \nHigh | \n
Docker focuses on the \"how\" of running an app. Kubernetes focuses on the \"where\" and \"how many\" of running those apps across a distributed system.
\nIf the unit of scheduling is new to you, it helps to understand the difference between a container and a pod before you reach for orchestration.
\nIn a professional DevOps workflow, Docker and Kubernetes are teammates. The process usually starts with a Dockerfile.
\nYou build a Docker image and push it to a private container registry. This image is your \"source of truth\" for the application.
\nNext, you create Kubernetes manifests. These files tell the cluster which image to use, how many replicas are needed, and what environment variables to inject.
\nWhen you apply these manifests, the kubelet on each node calls the container runtime to pull the image and start the container.
\nOne caveat that trips people up: since version 1.24, Kubernetes removed dockershim, so it no longer talks to Docker Engine directly. Nodes now run a CRI-compliant runtime such as containerd or CRI-O.
\nThe image you built with Docker still runs fine because it follows the OCI image spec, but Docker Engine itself is no longer the thing executing it.
\nTogether, the two make updates smooth. You can perform rolling deployments where Kubernetes replaces old pods with new ones in a controlled rollout. This keeps the service available throughout.
\nIf the new version has a bug, Kubernetes stops the rollout when health checks fail, and one command (kubectl rollout undo) takes you back to the previous stable image.
Not every project needs the overhead of Kubernetes. If you are an entrepreneur or a small team building a simple web application, starting with Docker Compose or a self-hosted SaaS platform might be more efficient.
\nIt allows you to ship fast without managing a complex cluster.
\nHowever, if your business requires high availability, handles massive amounts of data, or uses a microservices architecture, Kubernetes is the industry standard.
\nIt provides the horizontal scaling needed to grow from a few hundred users to millions without changing your core infrastructure.
\nShopify itself is a good example of the payoff: its platform runs on Kubernetes across hundreds of clusters on Google Kubernetes Engine, autoscaling stateless workloads to absorb the brutal traffic spikes of Black Friday and Cyber Monday.
\nMost merchants never touch that layer because Shopify owns it. But if you are building custom apps or your own infrastructure at that scale, the ability to declare capacity and let the cluster reconcile it is the competitive advantage that outweighs the setup complexity.
\n\nIf you're weighing that jump right now, here's how I help teams get the call right.
\nAt what stage of growth did your infrastructure move from a single server to a distributed cluster?
\nMost developers interact with proxies every single day without realizing which direction the traffic is actually flowing.
\n\n- A forward proxy speaks for the client and hides its IP. It handles outbound traffic, filtering and anonymity.\n- A reverse proxy speaks for the server and hides the backend. It handles load balancing, SSL termination, WAF and caching.\n- In Docker and Laravel setups, Nginx as a reverse proxy with the right forwarded headers is the pattern you will touch most.\n\nThe choice between a forward proxy vs reverse proxy comes down to one question: whose side is the middleman on?
\nYou set up Nginx for a Laravel app, and it works. You route outbound traffic through a corporate proxy, and it works.
\nBut when the site goes down under heavy load or your IP gets blacklisted, that distinction becomes the difference between a quick fix and hours of infrastructure debt.
\nIn modern web architecture, a proxy is essentially a \"middleman\" server. It sits between two parties to facilitate communication.
\nHowever, the side it represents determines its name, its function, and its security benefits. This guide breaks down the core technical differences to help you choose the right pattern for your next deployment.
\nA forward proxy sits in front of one or more client machines and acts as their representative to the open internet.
\nWhen a user makes a request to a website, the request first hits the forward proxy. The proxy then evaluates the request, sends it to the web server, and passes the response back to the user.
\nFrom the perspective of the internet, the forward proxy is the one making the request. The original client's IP address remains hidden behind the proxy's IP. This is a common setup in corporate environments or for users seeking anonymity.
\nForward proxies are primarily used to manage and secure outbound traffic. Here are the most common engineering use cases:
\nWhile a forward proxy represents the client, a reverse proxy represents the server. It sits at the edge of your infrastructure and intercepts all incoming requests from the internet before they reach your backend application.
\nThe client believes they are communicating directly with the origin server. In reality, they are talking to the reverse proxy. The proxy then decides which backend server should handle the request.
\nThis architecture is a cornerstone of modern cloud infrastructure and high-availability systems.
\nReverse proxies are essential for scaling and securing web applications. They handle several \"heavy lifting\" tasks that your application code shouldn't have to manage.
\nproxy_pass https:// and proxy_ssl_verify on instead.The easiest way to remember the difference is to look at who is being \"hidden\" from whom.
\n| Feature | \nForward Proxy | \nReverse Proxy | \n
|---|---|---|
| Who it protects | \nThe client | \nThe server | \n
| IP hiding | \nHides client IP from server | \nHides server IP from client | \n
| Placement | \nInternal network (client-side) | \nEdge of the network (server-side) | \n
| Primary goal | \nPrivacy, filtering, egress control | \nLoad balancing, security, performance | \n
| Example tool | \nSquid, Privoxy | \nNginx, HAProxy, Cloudflare | \n
In a forward proxy setup, the origin server has no idea who the actual client is. In a reverse proxy setup, the client has no idea which backend server actually processed their request.
\nBoth are essential for clean architecture but solve entirely different problems.
\nWhen building modern web applications, you are most likely to interact with reverse proxies. If you are using a tool like Coolify or Docker, the reverse proxy is often the component that manages your containers.
\nFor instance, an Nginx reverse proxy configuration for a Laravel application might look like this:
\nserver {\n listen 80;\n server_name myapp.com;\n\n location / {\n proxy_pass http://app_container:8000;\n proxy_set_header Host $host;\n proxy_set_header X-Real-IP $remote_addr;\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n proxy_set_header X-Forwarded-Proto $scheme;\n }\n}\nIn this setup, Nginx acts as the reverse proxy. It receives the request from the user and forwards it to the app_container.
It also passes along the X-Forwarded-For header so your application can still know the original user's IP address if needed.
This is a critical pattern when building an API gateway for an AI stack or managing Shopify app extensions.
\n\nLog `$request->ip()` in your Laravel app behind the proxy. If it shows the proxy's address instead of the user's, your forwarded headers or trusted proxies need fixing.\n\nIn complex microservices environments, the lines blur. Service meshes like Istio and Linkerd typically deploy a \"sidecar\" proxy next to every service.
\nEach sidecar acts as both a forward and a reverse proxy depending on whether the traffic is ingress (incoming) or egress (outgoing).
\nThis gives you fine-grained control over how services talk to each other. You can implement automatic retries, circuit breaking, and mutual TLS without writing a single line of application code.
\nWhile this adds complexity, it provides the level of observability and reliability required for enterprise-grade systems.
\n\n- **Forward proxies protect the user.** Use them when you want to control how your internal network accesses the internet or to maintain user anonymity.\n- **Reverse proxies protect the infrastructure.** Use them to scale your applications through load balancing, improve performance via caching, and secure your backend.\n- **Direction matters.** A forward proxy handles outbound requests. A reverse proxy handles inbound requests.\n- **Nginx is the industry standard.** It is the most common tool for implementing reverse proxies in [Laravel and Docker environments](/blog/category/laravel/).\n- **Security is centralized.** Both types of proxies allow you to centralize security policies rather than implementing them on every individual device or server.\n\nUnderstanding which direction traffic flows is the first step toward building resilient systems.
\nIf you are managing a growing web application, you should focus on your reverse proxy configuration. It is one of the most useful tools you have for uptime and performance.
\nHave you ever found yourself troubleshooting an \"IP mismatch\" error caused by a misconfigured proxy header?
\nDNS was built for servers that live for years. Your containers live for minutes.
\n\n- DNS maps a name to an IP and leans on caching, so changes are slow to reach clients and it has no idea whether an instance is healthy.\n- Service discovery keeps a live catalog of running instances, runs health checks and pushes changes to clients almost instantly.\n- The best setup for most teams is hybrid: a dynamic registry (Kubernetes or Consul) that still answers plain DNS queries.\n\nYou deploy a new version of your microservice and the old containers are killed instantly. You update your internal DNS records to point to the new IP addresses.
\nHowever, for the next five minutes, your API gateway continues to send 40 percent of your traffic to the dead containers. Your logs fill with connection timeouts and your error rate spikes.
\nThis is the classic DNS staleness trap that plagues modern distributed systems.
\nThe problem lies in how we locate resources. In a modern cloud environment, servers might only live for minutes. Relying on static naming for high-churn infrastructure creates a gap between where your traffic goes and where your healthy services actually live.
\nThis article explores the technical differences between DNS vs service discovery, why the distinction matters for your uptime, and how to choose the right strategy for your DevOps architecture.
\nAt its core, DNS is a distributed, hierarchical naming system. It acts like the Yellow Pages for the internet. You give it a hostname, and it returns an IP address.
\nIt is very efficient and available everywhere. Every programming language and operating system knows how to talk to it.
\nService discovery is a more specialized control plane. Instead of a static map, it acts like a live radar. It tracks the current state of every service instance in your cluster.
\nDNS tells you where a service should be. Service discovery tells you where it is right now.
\nA service discovery system like Consul, or the internal registry in Kubernetes, maintains a \"service catalog.\" This catalog is a live database of every running instance.
\nWhen a new container starts, it registers itself with the catalog. When it shuts down, it is removed. This happens in milliseconds, far faster than traditional DNS propagation.
\nThe biggest hurdle in using DNS for microservices is Time-To-Live (TTL). DNS relies on caching so that every request does not hammer the name servers.
\nYour operating system caches the result. Your browser caches the result. Your API gateway caches the result.
\nIf you set a high TTL, your service changes are slow to propagate. If you set a very low TTL (like 0 or 1 second), you might overwhelm your DNS server with lookups.
\nEven worse, some runtimes ignore the TTL entirely. The JVM, for example, caches a successful lookup for 30 seconds by default and forever when a security manager is set, so it can keep a stale IP until the process restarts.
\n| Feature | \nDNS | \nService Discovery | \n
|---|---|---|
| Primary Goal | \nHuman-readable name to IP | \nDynamic instance tracking | \n
| Propagation | \nSlow (TTL dependent) | \nNear-instant (Gossip/Streaming) | \n
| Metadata | \nVery limited (TXT records) | \nRich (tags, version, region) | \n
| Client Support | \nNative in all OS | \nOften requires API/Agent | \n
| Health Awareness | \nNone (Static) | \nActive monitoring | \n
Service discovery solves this by moving away from deep caching. Clients or load balancers often keep a streaming connection to the registry.
\nWhen a service instance moves, the registry pushes a notification to all subscribers. There is no waiting for a cache to expire. The system converges on the new state almost instantly.
\n\nCheck the DNS cache settings of your runtime. On the JVM, look at `networkaddress.cache.ttl` in the `java.security` file and set it to a short value if your services resolve internal names.\n\nA standard DNS server does not know whether the IP addresses it returns are healthy. If a server rack loses power, the DNS record still points to those IPs.
\nYou have to update the record by hand or run a custom script to change it.
\n\nService discovery systems run active health checks on every registered instance. A check can be as simple as a TCP probe or as complex as a custom HTTP endpoint that checks database connectivity.
\nIf a health check fails, the instance is marked \"unhealthy\" in the registry. The discovery system then stops returning that IP address to clients.
\nThis gives you a level of self-healing that plain DNS cannot, and it pairs naturally with the signals you already collect through logging and monitoring.
\nIf you are hosting a SaaS on Docker, automated health-aware routing is the difference between a minor blip and a major outage.
\nDNS is built to return one type of data: an IP address. You can use SRV records for port numbers or TXT records for strings, but the interface is clunky. It does not support complex queries.
\nService discovery allows metadata-driven routing. You can ask the registry for \"all healthy instances of the 'orders' service running version 2.1 in the 'us-east-1' region.\"
\nThis level of detail is essential for scaling complex systems where simple round-robin load balancing is not enough.
\nIn Kubernetes, you still use DNS names to talk to services. You might call http://order-service.default.svc.cluster.local. Does this mean Kubernetes just uses DNS?
Not exactly. Kubernetes uses a hybrid approach. It runs CoreDNS, but CoreDNS is not backed by static zone files. It is plugged directly into the Kubernetes API.
\nWhen a Pod starts or dies, the control plane updates the Service and EndpointSlice objects. CoreDNS watches those objects through the API and updates its answers in real time.
\nFor a normal ClusterIP Service, the name resolves to one stable virtual IP, and kube-proxy updates the routing behind it as Pods come and go. For a headless Service, the name resolves straight to the current Pod IPs.
\nIn this model, DNS is just the \"wire protocol.\" It is the interface your application uses because it is easy and standard. The backend is a fully dynamic service discovery engine.
\nThat gives you the simplicity of DNS and the speed of service discovery.
\n\nRun `kubectl get endpointslices -l kubernetes.io/service-name=` while you scale a Deployment up and down. Watch the addresses change without touching any DNS record.\n\nChoosing between DNS vs service discovery depends on your infrastructure scale and churn rate.
\nIf you run a monolithic application on a few virtual machines that rarely change, DNS is perfectly fine. You can manage your records with simple automation or even by hand. A dedicated discovery system would cost more than it gives back.
\nIf you use containers, serverless functions, or microservices, you need a dynamic solution. If your IPs change every time you deploy or scale, DNS will eventually break your traffic flow.
\nFor many developers, the best path is a tool that provides a DNS interface on top of a dynamic registry. HashiCorp Consul is a popular choice for this.
\nIt lets you query the registry with a standard DNS lookup, while it handles the health checking and registration logic behind the scenes.
\n\nCount how often your instance IPs changed last week. A few times a month means DNS is fine. Several times a day means you want a registry.\n\n\n1. **DNS is for stability**: Use it for external traffic, long-lived resources, and cross-company communication where caching is an advantage.\n2. **Service Discovery is for churn**: Use it for internal microservices, containers, and autoscaling groups where instances appear and disappear frequently.\n3. **Mind the Cache**: Never rely on plain DNS for sub-minute failover because client-side caching will almost always outlive your records.\n4. **Health is Binary**: Without active health checks, your discovery system is just a list of guesses. Always integrate automated liveness and readiness probes.\n5. **Hybrid is Best**: Use a discovery engine (like Kubernetes or Consul) that exposes a DNS interface. This simplifies your application code while keeping your infrastructure dynamic.\n\nIf you're untangling service routing for a high-churn stack, here's how I help teams ship it.
\nWhen was the last time a stale DNS record caused a production incident in your environment?
\nYou pay for a 1 Gbps fiber line, yet your API calls take 500ms and your backups crawl. You have a throughput problem, not a bandwidth problem.
\n\n- Bandwidth is the capacity of the link. Throughput is the data that actually arrives. Throughput is almost always lower.\n- Latency caps throughput: every round trip your app waits on is time the pipe sits idle, no matter how wide it is.\n- Smaller payloads, fewer round trips, reused connections and modern protocols raise throughput more than a bigger plan does.\n\nMost engineering teams mistake capacity for performance. They assume that widening the pipe automatically solves the lag.
\nThis misunderstanding leads to over-provisioned cloud bills and under-performing applications. You are throwing money at a bandwidth problem when you actually have a throughput bottleneck.
\nBandwidth vs throughput is the difference between a theoretical promise and a practical delivery. In high-performance web development and Shopify ecosystems, knowing which one limits you is what lets you scale without overpaying.
\nBandwidth is the maximum theoretical amount of data that can pass through a network path in a given time. Think of it as the number of lanes on a highway.
\nA twelve-lane highway has massive bandwidth. You have the potential to move thousands of cars at once.
\nIn technical terms, bandwidth is a measure of capacity. It is usually expressed in bits per second (bps), Megabits per second (Mbps), or Gigabits per second (Gbps).
\nWhen a cloud provider like Google Cloud or AWS quotes you a network speed, they are selling you bandwidth. It is a hard ceiling on how much data can move.
\nBandwidth does not account for the speed of the individual cars or a traffic jam at the exit ramp.
\nYou can have a 100 Gbps connection, but if the protocol or the destination server cannot keep up, most of that bandwidth stays unused.
\nThroughput is the actual amount of data that successfully travels from point A to point B in a specific timeframe. If bandwidth is the number of lanes, throughput is the number of cars that arrive every minute.
\n\nThroughput is almost always lower than bandwidth. Protocol overhead, network congestion, hardware limits, and packet loss all cut into the actual flow of data.
\nYou might have a 1 Gbps link while your application only reaches 200 Mbps because of internal processing delays or inefficient code. To see what that gap means for a real file, try my transfer time calculator.
\nMeasuring throughput gives you the real picture of your system health. It tells you how your Shopify store or your Laravel backend actually performs under load.
\nIf your throughput is far lower than your bandwidth, you have a bottleneck that more \"lanes\" will not fix.
\nYou cannot discuss bandwidth and throughput without latency. Latency is the time it takes for data to travel from the source to the destination; the full trip there and back is the round-trip time (RTT).
\nIt is the \"lag\" you feel in a video call or a gaming session.
\nHigh bandwidth does not mean low latency. Consider a geostationary satellite link. It may have massive bandwidth (great for downloading large files), but the round trip is roughly 500ms because the signal has to climb to about 35,786 km and back.
\nFor a \"chatty\" application that fires many sequential calls, like a RAG-based AI system chaining retrieval and generation, high latency kills throughput.
\nEven if you have infinite bandwidth, your throughput is capped by how many round trips your data has to make.
\nEvery time your application waits for an acknowledgment from the server, it sits idle. This is why optimizing for latency often matters more than upgrading your network plan.
\n\nCount the sequential network calls in your slowest request (database, cache, external APIs). Multiply by the round-trip time to each. That number is a floor no bandwidth upgrade can lower.\n\nIn a Shopify Plus environment, the gap between bandwidth and throughput directly affects conversion rates. When a customer hits your store, their browser makes dozens of requests for images, scripts, and API data.
\nIf your images are unoptimized, they eat up bandwidth. If your Shopify apps are poorly coded, they add latency. The result is lower throughput: the data does not reach the customer fast enough.
\nYou can have the fastest hosting in the world, but if your liquid templates do heavy lifting on every page load, the actual delivery speed stalls.
\nFor businesses scaling their digital presence, focusing on throughput means focusing on the user experience. It means smaller payloads and fewer round trips.
\nIt is about making sure the \"cars\" on your highway move at top speed and arrive without delay.
\nIn a typical Laravel environment managed with tools like Docker or Coolify, throughput bottlenecks often hide in the database or the cache layer.
\nYou might have a 10 Gbps internal network between your app server and your database server. On paper, your bandwidth is huge.
\nIf your queries are not indexed, or you fetch thousands of unnecessary rows, your throughput will tank. The network is fast, but the application is slow to process and deliver the data.
\nThis is where clean architecture and efficient query design become network optimization tools.
\nDevOps engineers also face throughput issues during CI/CD deployments. Moving a large Docker image across a network takes bandwidth. The time it takes to unpack, layer, and verify that image is a throughput concern.
\nOptimizing your Dockerfiles to reduce layer size is a direct way to improve the \"actual flow\" of your deployment pipeline.
\n\nRun `docker history ` on your largest image and look for the biggest layers. Moving one heavy install into a multi-stage build often shrinks the image more than any network change.\n\nImproving network performance needs two things: manage your capacity and optimize your flow. Here are the technical levers you can pull:
\nStop treating your ISP's speed test as the only measure of success. To understand your system, monitor both bandwidth utilization and application throughput. Tools like Google Cloud Monitoring or Datadog give detailed insight into both.
\nLook for places where available bandwidth is high but delivered throughput is low. These gaps are where your technical debt lives. They point to inefficient protocols, unoptimized assets, or server-side delays.
\nClosing this gap is how you get a \"snappy\" feel for your web applications.
\nA well-designed system on a moderate 100 Mbps link will often outperform a bloated system on a 1 Gbps link. Focus on the efficiency of the data flow rather than the width of the pipe.
\nIf you're chasing a throughput bottleneck in production, here's how I help teams ship the fix.
\nIf you had to choose between doubling your bandwidth or halving your latency for a real-time AI application, which would give the better return for your infrastructure?
\nA container is the package. A Pod is the place it runs, and the difference decides whether your sidecars can even reach your app.
\n\n- A container is one isolated process with its own filesystem. A Pod is the smallest unit Kubernetes deploys, and it can hold one or more containers.\n- Containers in the same Pod share one IP address, talk over `localhost` and can share Volumes.\n- The sidecar pattern uses that sharing to add logging, proxies or config reloads without touching your app code.\n\nWhen you move from a simple Docker setup to an orchestrated environment like Kubernetes, the terminology starts to blur.
\nYou wonder why you can't just deploy a container directly, and why Kubernetes insists on wrapping your perfectly good Docker image in something called a Pod. The container vs Pod distinction is the first thing that trips people up.
\nTreat a Pod as merely a \"fancy container\" and you miss the shared networking and storage that make Kubernetes work.
\nYou end up with monolithic containers that are hard to scale, or sidecars that can't actually reach the main application. Understanding how a container relates to a Pod is the difference between a brittle deployment and a scalable, resilient system.
\nA container is a lightweight, standalone executable package. It includes everything needed to run an application: code, runtime, system tools, system libraries, and settings.
\nAt its core, a container is an isolated process running on a host operating system. It shares the host's kernel but keeps its own filesystem and environment variables.
\nIn the world of DevOps and Docker, containers solved the \"it works on my machine\" problem. They give you a consistent environment from development to production.
\nWhether you are running a Laravel backend or a Shopify app extension, the container stays the same.
\nHowever, a container is solitary by nature. By default, it has its own IP address and its own isolated filesystem. Communication between two containers usually requires explicit networking configuration or external links.
\nKubernetes does not manage containers directly. This is a common point of friction for beginners. Instead, it manages Pods.
\nA Pod is the smallest deployable unit in Kubernetes. Think of a Pod as a \"logical host\" for one or more containers.
\nThe Pod acts as a wrapper. It provides a shared execution environment for the containers inside it.
\nMost Pods contain a single container, but the ability to group several is a core feature. These containers are always co-located and co-scheduled. They run on the same physical or virtual machine within the cluster.
\nThis coupling lets them behave as if they were processes on the same server, while each stays isolated in its own container.
\nThe biggest technical advantage of a Pod is the shared network namespace. Every Pod gets a unique IP address within the Kubernetes cluster. Every container inside that Pod shares this IP address.
\nThis changes how services communicate. In a standard Docker setup, Container A needs to know the IP of Container B to send a request. Inside a Kubernetes Pod, Container A can talk to Container B simply by using localhost.
localhost is extremely fast. There is no need for complex routing or service discovery between the containers within the same Pod.Just as they share a network, containers in a Pod can share storage. A container's internal filesystem is ephemeral and isolated, but a Pod can define shared Volumes.
\nThese volumes are mounted into the filesystem of every container that needs them.
\nThis is essential for applications that pass data between processes. For example, a main application might write log files, while a separate log-processing container reads those files and ships them to a central server.
\nWithout the Pod abstraction, you would have to deal with complex network mounts or external storage providers. In a Pod, it is as simple as mounting a shared directory.
\nThe \"sidecar pattern\" is the main reason we use multi-container Pods.
\nThe \"sidecar\" container sits next to the \"main\" container and performs a supporting task.
\nA sidecar lets you add functionality to an application without changing the application code.
\nBy separating these concerns into different containers within the same Pod, you keep your main application container clean and focused. This modularity is a pillar of containers versus orchestration thinking.
\nHistorically you added a sidecar by listing a second entry under spec.containers, which gave you no control over startup or shutdown order.
Kubernetes now has first-class sidecar support. An init container with restartPolicy: Always starts before the main containers, stays running for the Pod's lifetime, and shuts down after them.
This feature reached stable in Kubernetes v1.33, so reach for native sidecars over plain co-containers when ordering matters.
\n| Feature | \nStandalone Container | \nContainers in a Pod | \n
|---|---|---|
| Unit of Deployment | \nSingle process image | \nKubernetes Pod object | \n
| Networking | \nUnique IP per container | \nUnique IP per Pod (shared) | \n
| Localhost Access | \nIsolated to the container | \nShared across all containers | \n
| Storage | \nIsolated filesystem | \nShared Volumes available | \n
| Scheduling | \nManual or via runtime | \nAutomatic by K8s scheduler | \n
| Lifecycle | \nTied to the process | \nTied to the Pod's status | \n
| Scaling | \nScale individual instances | \nScale the entire Pod unit | \n
When you move your workloads to managed cloud providers like Google Cloud (GKE) or AWS (EKS), the container vs Pod distinction becomes operational. These platforms manage the infrastructure, but you still define the Pod specifications.
\nEKS on Fargate still runs real Pods, so multi-container Pods and sidecars work exactly as they do on a self-managed node. Fargate just provisions the compute per Pod instead of per node.
\nCloud Run started as single-container only. Since May 2023 (first in preview, now generally available) it supports multi-container deployments where sidecars share the network namespace and talk over localhost, just like a Pod.
So you no longer have to jump to full Kubernetes the moment you need a sidecar. You do once you need richer orchestration like DaemonSets, complex scheduling, or fine-grained networking.
\n\nWrite down the one sidecar you need today. If it is a single proxy or log agent, check whether Cloud Run or Fargate already covers it before you stand up a cluster.\n\n\n- A container is the packaging format, while a Pod is the execution environment.\n- Containers in a Pod share the same IP address and can communicate via `localhost`.\n- Shared Volumes allow containers within a Pod to exchange data through the filesystem.\n- The sidecar pattern uses multiple containers in one Pod to separate concerns like logging and security.\n- Kubernetes schedules and scales Pods as a single unit, ensuring all containers land on the same node.\n- Understanding Pods matters when you manage complex deployments on GCP, AWS, or [self-hosted tools like Coolify](/blog/coolify-self-hosted-saas/).\n\nIf you're architecting workloads that have outgrown plain Docker, here's how I help teams ship it.
\nAt what point does the overhead of managing a multi-container Pod outweigh the benefits of process separation in your current infrastructure?
\nRules you write by hand can be AI. Patterns a model learns from data are machine learning. Mix them up and you build the wrong system.
\n\n- AI is the umbrella: any system that reasons, plans or decides, including plain hardcoded rules. Machine learning is one way to get there, by training a model on data.\n- RAG pipelines and agents are hybrids: ML models (embeddings, LLMs) inside an AI system you design with code and prompts.\n- The split shapes your stack: orchestration code runs on normal servers, while ML models need data pipelines, GPUs and drift monitoring.\n\nThe tech industry often treats \"AI\" and \"machine learning\" as interchangeable. This confusion leads to real architectural debt when teams build intelligent features without understanding the underlying mechanics.
\nYou end up trying to build a predictive engine out of static rule-based systems or, the other way around, over-engineering a simple decision tree with a neural network.
\nPicking the wrong tool produces bloated cloud bills, slow performance, and systems that fail to generalize to real-world data. When you mix up the umbrella field of artificial intelligence with the specific mechanics of machine learning, you lose the ability to choose the most efficient path for your product.
\nThis guide draws the line between the two concepts and shows how they show up in modern stacks like Laravel and Shopify, with a focus on RAG and agentic systems. The goal: a clear framework for deciding when to reach for rule-based logic and when to deploy a learned model.
\nArtificial Intelligence (AI) is the broad field of creating systems that can perform tasks that usually require human intelligence. This is the \"umbrella\" term.
\nIt covers everything from simple \"if-then\" logic to the most complex large language models (LLMs).
\nIn engineering terms, AI is about the system's behavior. If a software system can reason, plan, or solve problems, it is an AI system. This does not mean it has to \"learn.\"
\nEarly AI systems, often called symbolic AI or expert systems, relied on hardcoded rules and logic. These systems are very predictable and easy to debug because every decision path is written out explicitly.
\nTake a sophisticated shipping calculator in a Shopify Plus store that adjusts prices based on complex regional taxes and carrier API responses. It mimics human decision-making from a set of logical parameters, which puts it at the simple, rule-based end of AI.
\nIt is not machine learning, because it does not improve its performance based on past data unless a developer updates the code.
\nMachine Learning (ML) is a subset of AI focused on algorithms that let computers learn from data and make predictions. Instead of being explicitly programmed for a task, an ML model is \"trained\" on a dataset to find patterns and hidden structures.
\nWhen your product needs to handle high-dimensional data where rules are too complex to write by hand, ML is the solution. Think about image recognition or fraud detection.
\nYou cannot write enough \"if\" statements to account for every possible pixel arrangement in a photo of a cat. Instead, you feed an ML model thousands of images, and it learns the statistical representation of a \"cat.\"
\n\nIn production, ML involves a lifecycle that is distinct from traditional software development. It requires data collection, cleaning, feature engineering, model training, and evaluation.
\nYou can read more about how this differs from standard workflows in our post on AI vs traditional development. For the deeper discriminative-vs-generative comparison inside ML itself, see machine learning vs generative AI.
\nThe distinction becomes critical when you move into modern AI engineering, particularly with Retrieval-Augmented Generation (RAG). A RAG system is a clear example of an AI system that uses several ML components to reach a goal.
\nIn a RAG pipeline, the system behavior is the AI part. How a user query is received, how a search runs against a vector database, and how the results go into a prompt is a design problem. The components doing the heavy lifting, however, are ML models.
\nThe \"retrieval\" part of RAG relies on embedding models. These are ML models (specifically deep learning models) that turn text into high-dimensional vectors.
\nWhen you store these vectors in a tool like pgvector or a dedicated vector database, you are using the output of a machine learning process.
\nThe \"generation\" part is handled by a large language model. This is a massive ML model trained on trillions of tokens of text.
\nWhether you call it through an API or a local deployment, the model itself is a static file of weights learned through intensive training.
\nWhen you refine your RAG system by changing the chunking strategy or the system prompt, you are doing AI engineering.
\nYou are only doing ML engineering if you decide to fine-tune the embedding model or the LLM itself on your domain data, a trade-off covered in RAG vs fine-tuning. Understanding these layers helps prevent common RAG mistakes in production.
\nYou are only doing ML engineering if you decide to fine-tune the embedding model or the LLM itself.
\n\nLook at the last three changes you made to your RAG pipeline. Label each one \"AI engineering\" (chunking, prompts, routing) or \"ML engineering\" (training, fine-tuning). Most teams find nearly all of them are the first kind.\n\nAgentic systems take the AI vs machine learning comparison a step further. An \"agent\" is an AI system that uses an ML model as its \"brain\" to decide a sequence of actions toward a goal. The jump from a static model to an autonomous one is the focus of LLM vs AI agent.
\nIn an agentic workflow, the ML model (the LLM) is just one tool in the agent's belt. The agent might also have access to a search engine, a calculator, or a database. The logic that handles the \"loop\" (Observe -> Think -> Act) is the AI system design.
\nFor developers working with Shopify, agentic commerce allows for autonomous customer service or inventory management.
\nThe agent uses machine learning to understand the customer's intent (NLP) but uses traditional software APIs to execute a refund or check stock levels.
\n// A conceptual example of an AI Agent 'thought' process in a Laravel controller\npublic function handleAgentRequest(Request $request)\n{\n $userInput = $request->input('query');\n\n // ML Component: Identify intent using an LLM\n $intent = $this->llmService->identifyIntent($userInput);\n\n // AI System Logic: Choose the tool based on intent\n if ($intent === 'check_order_status') {\n return $this->orderService->getStatus($request->user());\n }\n\n // AI System Logic: Fallback to RAG if intent is informational\n return $this->ragService->queryKnowledgeBase($userInput);\n}\nIntegrating AI and ML into existing frameworks like Laravel or Shopify requires a clear understanding of where the data lives and how the models are served.
\nLaravel is an excellent framework for building the \"AI system\" around an ML model. You can use Laravel's queue system to handle long-running ML tasks, or its HTTP client to talk to Python-based ML microservices.
\nMany developers use Laravel to manage the \"human-in-the-loop\" side of AI, such as reviewing model outputs before they are published.
\nShopify has built AI features directly into its platform through Shopify Magic. For custom Shopify Plus builds, you might implement your own ML-driven product recommendation engine.
\nThis involves capturing user behavior data, training a model (likely off-platform on Google Cloud or AWS), and serving those recommendations through a custom app or a Liquid block.
\n\nPut a queue between your Laravel app and any model call that can take more than a second. Dispatch a job, store the result, and let the UI poll or listen for it.\n\nDeploying these systems requires different infrastructure strategies. AI systems (the orchestration code) can usually run on standard web servers or serverless functions.
\nMachine learning models, especially for training or high-throughput inference, often require GPUs or specialized hardware.
\nDocker and tools like Coolify make it simpler to deploy these modular systems. You might host your Laravel application on a standard VPS while your vector database and ML inference server run in separate containers. For more on this, check out our guide on Coolify and Docker for SaaS hosting.
\nMonitoring is also different. For an AI system, you monitor latency and error rates. For an ML model, you monitor \"drift\" (when the real-world data starts to look different from the training data) and accuracy.
\n| What you run | \nWhere it runs | \nWhat you monitor | \n
|---|---|---|
| AI system (orchestration code) | \nStandard web servers or serverless | \nLatency and error rates | \n
| ML model | \nGPUs or specialized hardware | \nDrift and accuracy | \n
Are you building a system that needs to follow strict regulatory rules, or one that needs to adapt to unpredictable user behavior?
\nTraining is the expensive class you take once. Inference is the bill you pay on every single request.
\n\n- Training adjusts a model's weights over a huge dataset. It is compute-heavy, runs on GPU clusters and happens once or now and then.\n- Inference is a forward pass with frozen weights. It is cheaper per request, but it runs every time a user asks, so it usually becomes the bigger cost over time.\n- Size each phase on its own: throughput and big interconnects for training, latency, quantization and smaller hardware for inference, plus RAG to add new knowledge without retraining.\n\nMost engineering teams treat AI models as a single black box. They allocate a massive GPU budget and hope for the best.
\nBut when the application hits production, latency spikes, costs spiral, and the \"intelligent\" features start timing out.
\nThis happens because developers fail to separate the two phases of an AI lifecycle: training and inference. Without a clear picture of how each stage consumes resources, your AI strategy is a shot in the dark.
\nTraining is the \"learning\" phase of a machine learning model. During this stage, you feed an algorithm a massive dataset.
\nThe model looks at the data, makes a guess, compares its guess to the actual answer, and adjusts its internal parameters (weights) to get closer to the truth next time. This process is repeated millions or even billions of times.
\nThe main goal of training is to minimize error. Because this requires massive matrix multiplications and constant backpropagation, it is extremely compute-intensive.
\nYou are asking a computer to solve a giant calculus problem over and over again. This is why training usually happens on large clusters of high-end GPUs or TPUs.
\nTraining is usually a burst-heavy, high-upfront-cost activity. You might spend two weeks and $50,000 to train a custom model for your specific industry.
\nOnce the model reaches a satisfactory level of accuracy, the training phase ends. The model's weights are \"frozen,\" and it is ready to be used in the real world.
\nInference is the \"execution\" phase. This is what happens when a user interacts with your application.
\nA customer types a query into your Shopify store's AI assistant, the model processes that query using its frozen weights, and it returns an answer. No learning happens during inference. The model simply applies what it already knows.
\nIn technical terms, inference is a \"forward pass.\" You provide an input, the data flows through the layers of the model, and an output is generated.
\nThere is no backpropagation and no weight updates. This makes inference much faster and less compute-intensive than training on a per-request basis.
\nHowever, inference is where the \"unbounded cost\" problem lives. Training is a one-time or periodic expense. Inference happens every time a user makes a request.
\nIf you have a million users making ten queries a day, you are running ten million inferences. Over the lifetime of a successful product, inference costs often end up far larger than training costs.
\n\nEstimate your monthly inference volume: daily active users, times queries per user, times 30. Multiply by your cost per request. Compare that with what the model cost to train or fine-tune.\n\nTo make these ideas simpler, think of a student in medical school.
\n\nThe years of study. The student reads thousands of textbooks, attends lectures, and takes practice exams. This process is slow, expensive, and requires intense focus. The \"parameters\" of the student's brain are being adjusted as they learn how to diagnose diseases.
\nThe doctor in the clinic. A patient walks in with symptoms. The doctor uses existing knowledge to give a diagnosis and does not go back to medical school for every patient. The diagnosis happens in minutes, not years.
\nIf you are building an AI vs traditional development strategy, you need to decide if you are training a new doctor (custom training) or simply hiring one that already exists (using a pre-trained model via API).
\nThe hardware requirements for these two phases are very different. Understanding this can save you thousands in infrastructure costs.
\n| Feature | \nTraining | \nInference | \n
|---|---|---|
| Primary Goal | \nHigh Throughput | \nLow Latency | \n
| Compute Pattern | \nBurst-heavy / Parallel | \nSteady / Sequential | \n
| Hardware | \nMulti-GPU Clusters (H100, A100) | \nSingle GPU, CPU, or Edge (T4, L4, Apple Silicon) | \n
| Optimization | \nGradient Descent / Backprop | \nQuantization / Model Compression | \n
| Cost Type | \nCAPEX (Upfront) | \nOPEX (Recurring) | \n
For training, you need massive VRAM and high-speed interconnects (like NVLink) between GPUs. For inference, you often prioritize energy efficiency and \"cost per token.\"
\nIn many cases, a well-optimized model can run inference on standard CPUs or specialized \"edge\" chips, which is much cheaper than keeping a fleet of high-end GPUs.
\nIf you manage your own servers, you can split the workloads: put training tasks on high-performance clusters and move inference to smaller, distributed nodes closer to your users.
\n\nTry a quantized version (8-bit or 4-bit) of the model you serve on your smallest available GPU or a CPU. Measure latency and answer quality on 20 real prompts before paying for bigger hardware.\n\nWhen building custom web solutions, you usually deal with the inference side of the equation. You are not training a frontier model from scratch.
\nYou are calling an API or a self-hosted model to perform a task. Here is a typical pattern for handling AI inference inside a Laravel controller.
\nnamespace App\\Http\\Controllers;\n\nuse Illuminate\\Http\\Request;\nuse Illuminate\\Support\\Facades\\Http;\n\nclass AIInferenceController extends Controller\n{\n /**\n * Handle an AI inference request.\n */\n public function generate(Request $request)\n {\n $prompt = $request->input('prompt');\n\n // We use a high-performance inference endpoint\n // This could be OpenAI, Anthropic, or a self-hosted vLLM server\n $response = Http::withHeaders([\n 'Authorization' => 'Bearer ' . config('services.ai.key'),\n ])->post('https://api.inference-provider.com/v1/completions', [\n 'model' => 'llama-3-70b',\n 'prompt' => $prompt,\n 'max_tokens' => 150,\n 'temperature' => 0.7,\n ]);\n\n if ($response->successful()) {\n return response()->json([\n 'result' => $response->json('choices.0.text'),\n 'latency' => $response->header('X-Inference-Time'),\n ]);\n }\n\n return response()->json(['error' => 'Inference failed'], 500);\n }\n}\nThis simple setup highlights a key architectural point: the API Gateway for your AI stack must handle the latency needs of inference.
\nUsers expect an answer in seconds, not minutes, and the first token well under that. A slow gateway in front of a fast model still feels slow.
\n\nLog the time to first token and the total response time for every inference call, next to the model name. Those two numbers tell you whether to fix the model, the network, or the prompt size.\n\nOne way to bridge the gap between training and inference is retrieval-augmented generation (RAG).
\nInstead of re-training a model every time your data changes (which is expensive and slow), you give the model \"context\" during the inference phase. That tradeoff is its own decision, covered in RAG vs fine-tuning.
\nIn a RAG system, you search a vector database for relevant information and \"stuff\" it into the prompt. The model then uses its pre-trained reasoning to answer based on that new data. This is inference acting like it has a temporary memory.
\nBe careful, though. If you don't optimize your vector search, your inference latency will explode. You can read more about avoiding RAG mistakes in production to keep your systems lean.
\nIf you're architecting an AI stack that has to stay fast and affordable at scale, here's how I help teams ship it.
\nAt what point in your product's growth do you expect inference costs to exceed your initial development budget?
\nVector search finds things that are similar. Graph search finds things that are connected. Multi-hop questions need the second.
\n\n- Vector search (for example pgvector) turns text into embeddings and finds the nearest ones. It is easy to start with and great for fuzzy, semantic matching.\n- Graph search stores entities and their relationships, so it can follow chains of facts that vector search misses.\n- GraphRAG combines them: vectors find the starting points, graph traversal pulls in the connected context.\n\nRetrieval-Augmented Generation (RAG) forces a question most teams put off: how is data actually retrieved?
\nMost production systems start with vector search because it is accessible and effective for basic similarity. But as queries get more complex, the limits of semantic similarity show.
\nA system that finds \"similar\" things often fails when asked to explain \"connected\" things. That gap is where the choice between vector search and graph search decides whether your RAG strategy holds up.
\nRelying only on vector embeddings can lead to what is known as the semantic trap. You might retrieve five document chunks that all mention a topic, but if those chunks do not link to each other, the LLM will struggle to put together a multi-hop answer.
\nThis often results in hallucinations or incomplete reasoning. If your RAG pipeline cannot traverse relationships, it is working with a deck of shuffled cards rather than an organized map.
\nThe fix starts with understanding the trade-offs between vector-based and relationship-based retrieval. Vector search excels at high-dimensional similarity; graph search offers structured traversals that preserve the logic of your data.
\nCombining the two into a hybrid architecture is usually the safest path forward.
\nVector search works by converting unstructured data into dense numerical arrays called embeddings. These embeddings represent the semantic meaning of the text in a multi-dimensional space.
\nWhen a user submits a query, that query is also converted into a vector. The system then calculates the distance between the query vector and the stored vectors using measures like cosine similarity or Euclidean distance.
\nFor developers working within the Laravel or Node.js ecosystems, tools like pgvector have made this very accessible. By adding a vector column to a standard Postgres database, you can run similarity searches right next to your relational data.
\nThis approach is very efficient for \"fuzzy\" matching. It can find a product description for a \"crimson summer dress\" even if the query only mentions a \"red lightweight gown.\"
\nHowever, vector search is limited by its lack of structural awareness. It treats every chunk of data as an independent point in space.
\nIt has no built-in understanding that \"User A\" is the \"CEO\" of \"Company B\" unless those words sit together in a single text chunk.
\nGraph search takes a different approach by modeling data as nodes (entities) and edges (relationships). Instead of looking for similarity in a coordinate system, graph search follows defined paths.
\nA node might represent a person, a place, or a concept, while an edge defines how they relate. For example, an edge labeled \"works_at\" might connect a person node to a company node.
\nThis structure allows multi-hop queries that are nearly impossible for pure vector search. You can ask the system to \"find all employees at companies that use Laravel and were founded after 2015.\"
\nA graph database like Neo4j or a graph-capable extension can follow these links precisely. In RAG, this means the retrieval engine can pull in a coherent chain of facts rather than a disjointed set of similar-sounding paragraphs.
\nThe challenge with graph search is the ingestion process. Vector search only requires an embedding model. Graph search requires entity extraction and schema mapping.
\nYou must identify which entities exist in your text and how they relate before you can store them in the graph. This adds complexity to your data pipeline but pays off in reasoning accuracy.
\nA system that finds \"similar\" things often fails when asked to explain \"connected\" things.
\nChoosing the right approach depends on the nature of your data and the questions your users ask. The table below outlines the main technical differences.
\n| Feature | \nVector Search (pgvector) | \nGraph Search (Knowledge Graph) | \n
|---|---|---|
| Data Model | \nHigh-dimensional dense vectors | \nNodes, edges, and properties | \n
| Query Type | \nK-Nearest Neighbors (k-NN) | \nTraversals and pattern matching | \n
| Strengths | \nSemantic similarity, fuzzy match | \nComplex relationships, multi-hop logic | \n
| Weaknesses | \nStruggles with structural reasoning | \nHigh ingestion complexity | \n
| Ideal For | \nFAQ bots, general search, broad Q&A | \nFraud detection, supply chain, reasoning | \n
| Scaling | \nEfficient with ANN indexing (HNSW) | \nSensitive to graph density and depth | \n
For a deeper look at avoiding common pitfalls in this space, see our guide on 7 RAG mistakes in production. Many teams start with vectors and only introduce graphs when they hit a \"reasoning wall.\"
\nThe most advanced RAG systems are moving toward a hybrid model often called GraphRAG. This architecture does not choose one over the other. It uses both to give the LLM richer context.
\nIn a GraphRAG setup, the system first runs a vector search to find relevant starting points in the knowledge base. From those starting points, it uses graph traversals to pull in related entities and their relationships.
\n\nImagine a medical research application. A vector search might find a paper about a specific drug.
\nThe graph search then identifies the chemical compounds in that drug, the clinical trials associated with it, and the known side effects reported in other related papers.
\nThe context given to the LLM is a structured \"subgraph\" of knowledge. This reduces the risk of hallucinations because the LLM is working with explicitly linked facts.
\nThis hybrid approach needs a solid DevOps foundation. Tools like Docker and Coolify can help manage the services involved, including your vector store, graph database, and the extraction services that keep them in sync.
\nIf you are exploring how to host these stacks, our article on Coolify and self-hosted SaaS is a good starting point for infrastructure management.
\nFor teams already using Postgres, pgvector is the logical starting point for adding vector capabilities. It fits into existing SQL workflows.
\nYou can store your embeddings in a vector column and use the <-> (Euclidean) or <=> (cosine distance) operators to query them.
-- Example: Finding similar document chunks in Postgres\nSELECT content, 1 - (embedding <=> '[0.1, 0.2, 0.3, ...]') AS similarity\nFROM document_chunks\nORDER BY embedding <=> '[0.1, 0.2, 0.3, ...]'\nLIMIT 5;\nOrder by the distance expression itself and keep the LIMIT. That is the shape pgvector's HNSW and IVFFlat indexes can use. To see how big those indexes get and how much RAM they need, try my pgvector size calculator.
A WHERE filter on the similarity score (for example > 0.8) cannot use the index and forces a full scan, so apply a score threshold to the five rows you get back instead.
To approximate a graph without a dedicated graph database, you can model simple one-to-many or many-to-many relationships with relational joins.
\nBut as the number of hops grows, recursive joins and WITH RECURSIVE queries get expensive fast, and the planner struggles to keep them efficient.
That is the point to introduce a specialized graph layer such as Neo4j or the Apache AGE extension, which adds openCypher graph queries directly to Postgres. For the broader question of which retrieval components to assemble, see picking the right RAG stack.
\n\nRun `EXPLAIN ANALYZE` on your similarity query. If you do not see an index scan on your HNSW or IVFFlat index, rewrite it to `ORDER BY embedding <=> ... LIMIT k`.\n\nThe decision between vector and graph search is not binary. It is a spectrum of technical complexity versus reasoning capability.
\nFor simple semantic search over a collection of PDFs, vector search with pgvector is almost always the right answer. It is fast, easy to implement, and needs little maintenance.
\nIf your application needs high precision, multi-step logic, or the ability to explain \"why\" a result was chosen, the investment in a knowledge graph is necessary.
\nThe overhead of building an extraction pipeline is the price you pay for a system that understands the relationships within your data.
\nMost production AI systems will eventually use vectors for broad discovery and graphs for deep reasoning. If you design your system to be modular today, your infrastructure is ready to evolve as your AI needs grow.
\n\n- Vector search is best for semantic similarity and is easy to implement using tools like pgvector.\n- Graph search excels at multi-hop reasoning and mapping complex relationships between entities.\n- Vector search treats data points as isolated, while graph search treats them as connected nodes.\n- GraphRAG is a hybrid approach that uses vector search for discovery and graph search for context expansion.\n- Start with vector search for most projects, but prepare for graph search if your queries require complex structural reasoning.\n- Monitor retrieval metrics like recall and precision to spot when a vector-only strategy starts failing.\n\nIf you're building a RAG pipeline for production, here's how I help teams ship it.
\nHow is your current RAG pipeline handling multi-hop queries that connect data points across documents?
\nTerraform builds the servers. Ansible sets up what runs on them. Most pipeline pain comes from asking one to do the other's job.
\n\n- Terraform is a declarative provisioning tool: you describe the cloud resources you want and it uses a state file to create, change or destroy them.\n- Ansible is a configuration management tool: agentless playbooks over SSH install and configure software on servers that already exist.\n- The strongest setup uses both: Terraform for the cloud fabric, Ansible for the software layer, wired into one CI/CD pipeline.\n\nYou spend six hours clicking through the AWS console to set up a VPC, three subnets, and a load balancer. Then you realize you missed a single security group rule that breaks the entire staging environment.
\nThen you have to do it all over again for production. This manual toil is a ticking time bomb for technical debt and human error.
\nInfrastructure as Code (IaC) is the industry standard solution, but the market is split between two titans: Terraform and Ansible. Choosing the wrong one for the task leads to fragile pipelines and \"snowflake\" servers that no one dares to touch.
\nThe most common misunderstanding in DevOps is treating Terraform and Ansible as interchangeable tools. They are not. At their core, they solve two distinct stages of the infrastructure lifecycle.
\nTerraform is a provisioning tool. Its main job is to talk to cloud providers like AWS, Google Cloud, or Azure to create the virtualized \"hardware\" of your stack: VPCs, subnets, IAM roles, and managed databases.
\nIt builds the foundation your software will live on.
\nAnsible is a configuration management tool. Once the server exists, Ansible takes over to install Nginx, configure the PHP-FPM pool, or deploy your latest Laravel build.
\nIt is designed to manage the internal state of the operating system and the applications running on it.
\n| Feature | \nTerraform | \nAnsible | \n
|---|---|---|
| Primary Goal | \nProvisioning infrastructure | \nConfiguration management | \n
| Language | \nHCL (declarative) | \nYAML (procedural/imperative) | \n
| State | \nManaged (state file) | \nStateless | \n
| Architecture | \nClient-only (API-based) | \nAgentless (SSH/WinRM) | \n
| Best For | \nVPCs, clusters, databases | \nInstalling apps, OS hardening | \n
The way you talk to these tools defines how you manage your stack. Terraform is strictly declarative. You describe the \"end state\" you want.
\nIf you tell Terraform you want five EC2 instances, it compares that with the instances it already manages. If it manages three, it adds two. If it manages seven, it destroys two.
\nYou don't tell it how to do it; you tell it what you want the result to be. Instances created by hand outside Terraform are invisible to it until you import them.
\nAnsible is mainly procedural (or imperative), though it uses declarative modules. You write playbooks that list a series of steps:
\nAnsible modules are idempotent (they won't re-install Docker if it's already there), but the workflow follows a specific sequence of commands. This makes Ansible very good at complex application deployments where the order of operations matters.
\n\nThe choice between Terraform vs Ansible often shapes your architectural philosophy.
\nTerraform leans heavily toward immutable infrastructure. In an immutable world, you don't \"fix\" a server. If you need to update the base OS image, you destroy the old server and provision a new one from a fresh image.
\nThis is the same philosophy behind containers and orchestration. It removes \"configuration drift,\" where servers that started identical become different over time because of manual patches.
\nAnsible is the king of mutable infrastructure. It is designed to go into existing, long-lived servers and modify them.
\nThis is often necessary for legacy systems or complex environments where spinning up a fresh cluster for every minor config change is too slow or expensive.
\nFor teams using self-hosted solutions like Coolify for SaaS hosting, Ansible is a good fit for managing the underlying VPS environment.
\n\nPick one server you patch by hand today. Write the patch as a small Ansible playbook and run it with `--check --diff` first, so you see what would change before it changes.\n\nTerraform's superpower (and its greatest complexity) is the state file. This JSON file is a map of your real-world infrastructure.
\nWhen you run a command, Terraform compares your HCL code against this state file and the real resources to work out what needs to change.
\nThis lets Terraform detect \"drift\": when someone manually changes a setting in the AWS console, Terraform sees the difference and offers to revert it.
\nAnsible is stateless. It doesn't keep a record of what it did yesterday. It simply connects to the IP addresses in your inventory and runs the playbook.
\nIf a server is down, Ansible reports a failure, but it has no \"global view\" of your infrastructure the way Terraform does. This makes Ansible easier to start with but harder to use for tracking the full lifecycle of high-level cloud resources.
\nIn a professional DevOps pipeline, you rarely choose just one. Most strong teams use both.
\nBy separating the cloud fabric (Terraform) from the software layer (Ansible), you create a modular system that is easy to debug and scale.
\nIf you move from AWS to Google Cloud, you rewrite your Terraform code for the new provider's resources but keep your Ansible playbooks largely the same.
\nWiring both stages into a single CI/CD pipeline turns this into a one-command, reproducible build.
\nOne 2026 caveat worth flagging: HashiCorp moved Terraform to the Business Source License in August 2023, and the community forked it into OpenTofu (now a CNCF project under the Linux Foundation).
\nOpenTofu is a near drop-in replacement using the same HCL syntax and state format, so \"Terraform\" in this article applies equally to it. IBM completed its acquisition of HashiCorp in February 2025, which has only sharpened that licensing question for teams.
\n\nAdd a `terraform output -json` step to your pipeline and feed it into an Ansible dynamic inventory, so new servers get configured without anyone copying IPs.\n\n\n- **Terraform** is for building the house (infrastructure). It is declarative, immutable, and stateful.\n- **Ansible** is for decorating and maintaining the house (software configuration). It is procedural, mutable, and stateless.\n- Use **Terraform** to manage resources with a clear lifecycle, like databases and load balancers.\n- Use **Ansible** to manage the \"inside\" of a VM, such as package updates and application deployments.\n- Avoid using Ansible to provision cloud resources; while possible, it lacks the state management and dependency graphing of Terraform.\n- Combine both tools in a CI/CD pipeline for a fully automated, reproducible environment.\n\nIf you had to rebuild your entire production environment from scratch in 30 minutes, would your current automation tools be enough to restore both the infrastructure and the application state?
\nAn SRE keeps production up for your users. A platform engineer makes shipping easy for your developers. Hire for the bottleneck you have today.
\n\n- SREs own production reliability: SLIs, SLOs, error budgets, on-call, incident response and automating away toil.\n- Platform engineers own developer experience: an internal developer platform with self-service \"Golden Paths\" from laptop to production.\n- The two feed each other: SREs find recurring failures, platform engineers bake the fix into the defaults so every new service gets it.\n\nYour production environment is down and the error budget is burning. You have two choices.
\nYou can hire someone to fix the immediate fire and make sure it never happens again. Or you can hire someone to build the tools that keep your developers from starting that fire in the first place.
\nThis is the core tension between Site Reliability Engineering (SRE) and Platform Engineering.
\nFor years, \"DevOps\" was the catch-all term for anyone who touched a server. As systems grew more complex, the roles split into specialized disciplines. SRE and Platform Engineering are the two most visible results.
\nThey often share tools like Kubernetes, Terraform, and Docker, but their missions are worlds apart. One focuses on the runtime reality of the user. The other focuses on the internal experience of the developer.
\nSite Reliability Engineering is what happens when you ask a software engineer to design an operations function. Originally pioneered by Google, SRE treats operations as a software problem.
\nThe mission of an SRE is simple but difficult: make sure production services are reliable, scalable, and performant.
\nSREs live and breathe metrics. They define Service Level Indicators (SLIs) and Service Level Objectives (SLOs) to put a number on what \"up\" actually means.
\nIf a service has a 99.9% availability target, the SRE manages the \"error budget.\" This is the remaining 0.1% of downtime allowed before the team must stop shipping features and focus on stability.
\nThe daily life of an SRE involves on-call rotations, incident response, and post-mortem analysis. Beyond fixing bugs, they build automation to eliminate \"toil.\"
\nToil is the manual, repetitive work, like scaling clusters or rotating certificates, that grows linearly with the size of the system.
\nBy writing code to automate these tasks, an SRE lets the infrastructure grow without a matching increase in headcount.
\nPlatform Engineering is the practice of building and running internal developer platforms (IDPs). If the SRE focuses on the end user, the Platform Engineer focuses on the internal developer.
\nTheir goal is to improve Developer Experience (DX) by hiding the complexity of the underlying infrastructure.
\nA Platform Engineer builds \"Golden Paths.\" These are standardized, self-service workflows that let a developer go from \"code on my laptop\" to \"running in production\" without opening a ticket for the infrastructure team.
\nInstead of every developer learning the details of AWS IAM roles or Kubernetes ingress controllers, they use a platform that handles these details automatically.
\nPlatform Engineers are measured by lead time to production and developer satisfaction. They treat the platform as a product and run user research with their internal developers to find friction points.
\nThey build CI/CD pipelines, environment provisioning tools, and internal portals that make shipping software feel effortless. You can see similar patterns in how tools like Coolify simplify SaaS hosting by providing one management layer.
\nThe easiest way to tell these roles apart is to look at who they serve and how they are measured.
\n| Feature | \nSRE | \nPlatform Engineer | \n
|---|---|---|
| Primary Customer | \nEnd users & product teams | \nInternal developers | \n
| Success Metric | \nSLOs, latency, MTTR | \nLead time, deployment frequency | \n
| Key Focus | \nProduction reliability | \nDeveloper productivity | \n
| Artifacts | \nAlerting rules, runbooks, SLOs | \nIDPs, CLI tools, CI/CD templates | \n
An SRE cares about the \"now.\" Is the site up? Is the latency within limits? If a database query takes 500ms instead of the usual 50ms, the SRE is the first person to notice. They are the front line for the business's reputation.
\nA Platform Engineer cares about the \"how.\" How long does it take a new hire to ship their first line of code? How many manual steps are in the deployment process?
\nThey are the industrial engineers of the software factory. They optimize the assembly line so everyone else can move faster.
\nThis role matters more as we move toward agentic commerce and automated Shopify workflows, where the speed of iteration is a competitive advantage.
\nImagine a Formula 1 team.
\n\nWithout the factory engineer, the car wouldn't be fast. Without the race engineer, the car wouldn't finish the race.
\nBoth roles use similar technologies, but they apply them differently.
\nSRE tooling:
\nPlatform engineering tooling:
\nThe SRE uses observability tools to find \"unknown unknowns\" in production. The Platform Engineer uses those same tools to bake monitoring defaults into the platform.
\nIf you want to dig deeper into the technical stack for modern applications, check our guide on the API gateway for the AI stack.
\nIn a high-performing organization, these two roles form a tight feedback loop.
\nThe SRE discovers a recurring failure mode during an incident. Perhaps developers forget to set proper resource limits on their containers, and nodes run out of memory.
\nInstead of telling developers to \"be more careful,\" the SRE brings this feedback to the Platform Engineer. The Platform Engineer updates the \"Golden Path\" templates to include sensible default resource limits automatically.
\nNow every new service created on the platform is \"reliable by design.\"
\nThis loop reduces the SRE's operational load. They spend less time fighting fires and more time on high-level architecture.
\nAt the same time, developers are happier because the platform keeps them from making common mistakes. This is what mature DevOps looks like.
\nThe choice depends on your current pain points.
\nYour developers keep struggling to set up environments, fighting CI/CD pipelines, or waiting weeks for infrastructure tickets. You have a productivity bottleneck. You need to productize your infrastructure so your team can scale.
\nYour site is frequently down, your performance is unpredictable, and your developers are terrified of shipping on a Friday. You have a reliability bottleneck. You need someone to bring the discipline of SLOs and incident management into your culture.
\nIn many startups, a single \"DevOps Engineer\" handles both. As you grow, the split is inevitable, because specialized roles allow deeper expertise.
\nAn SRE who doesn't have to build CI/CD pipelines can focus on making the system hard to break. A Platform Engineer who isn't on call for every production blip can focus on making the developer experience excellent.
\n\nFor one week, tag every infrastructure complaint from your team as \"it broke\" or \"it's slow to ship.\" Whichever pile is bigger tells you which role to hire first.\n\n\n- SRE focuses on the reliability of production systems for end users.\n- Platform Engineering focuses on the developer experience and internal tooling.\n- SRE uses SLOs and error budgets to balance speed and stability.\n- Platform Engineering uses \"Golden Paths\" to provide self-service infrastructure.\n- The two roles are complementary. SRE provides the feedback, and Platform Engineering provides the abstractions.\n- Both roles are essential for scaling modern, complex software organizations.\n\nHow does your team handle the balance between shipping fast and staying up?
\nA slow API call inside a web request ties up a worker that every other user is waiting for.
\n\n- Synchronous calls block: the worker waits until the task finishes. Fine for fast reads and writes, dangerous for slow third-party APIs.\n- Asynchronous work goes on a queue and runs in a background worker, so the user gets an instant response and failed jobs can retry.\n- In Laravel and Shopify that means: dispatch jobs for slow work, and answer webhooks fast, then process them in a job.\n\nEvery developer has faced the dreaded \"spinning wheel of death.\" You click a button, the browser hangs, and you wait five seconds for a confirmation that never feels fast enough.
\nThis lag usually comes from a synchronous process holding the entire request-response cycle hostage while it waits for a third-party API or a heavy database query.
\nIn high-traffic environments, these blocking calls are more than a nuisance. They are a scalability killer that can bring down your entire infrastructure.
\nArchitecting for scale means knowing when to keep things synchronous and when to move them to the background. This decision affects everything from user experience to server costs.
\nBy shifting heavy lifting to asynchronous workers, you decouple your system. This lets your application handle thousands of concurrent users without breaking a sweat.
\nSynchronous communication is the traditional request-response model. When a client sends a request, it waits for the server to process the logic and return a response.
\nThis is a \"blocking\" operation. The execution thread is tied up until the task completes.
\nThink of it like a phone call. You dial a number, wait for the other person to pick up, and you cannot do anything else until the conversation is over.
\nIn web development, this is perfectly fine for simple operations like fetching a user profile or updating a single database row. The latency is minimal and the user needs the data immediately to continue.
\nSynchronous flows become dangerous when you add external dependencies. If your Laravel controller calls the Shopify Admin API to update 500 products during a standard HTTP request, your PHP worker is stuck.
\nIf Shopify takes three seconds to respond, that worker cannot serve any other users for those three seconds.
\nUnder heavy load, your worker pool will run out. This leads to 504 Gateway Timeout errors and a broken experience.
\n\nFind your three slowest controller actions in your APM or logs. For each one, note whether the time is spent waiting on an external API. Those are your first candidates for a queue.\n\nAsynchronous communication decouples the request from the processing. The client sends a message or triggers an event and immediately receives a response.
\nThe actual work happens \"out of band\" in a separate process or at a later time.
\nThis is more like sending an email. You hit send and go straight back to your day. The recipient reads and processes the message whenever they are available.
\nIn a technical stack, this is done with message queues like Redis or Amazon SQS, or a dedicated broker such as RabbitMQ.
\n\nWhen you move a task to an asynchronous worker, the user gets an instant \"Success\" message. The heavy processing happens in the background. This architecture is essential for:
\nLaravel gives you asynchronous processing through its Queue system. Instead of performing a slow task in your controller, you dispatch a \"Job.\"
\n// Synchronous (Bad for slow tasks)\npublic function store(Request $request) {\n $order = Order::create($request->all());\n Mail::to($request->user())->send(new OrderConfirmed($order)); // Waits for SMTP\n return response()->json($order);\n}\n\n// Asynchronous (Scalable)\npublic function store(Request $request) {\n $order = Order::create($request->all());\n ProcessOrderEmail::dispatch($order); // Returns immediately\n return response()->json($order);\n}\nBy using dispatch(), you push a payload into Redis. A separate process, managed by a tool like Laravel Horizon, picks up the job and runs it. This keeps your web-facing workers free to handle more incoming traffic.
If the email server is down and the job has retries configured ($tries and $backoff on the job, or --tries on the worker), the job goes back on the queue and tries again later.
This kind of resilience is impossible in a strictly synchronous world.
\n\nOpen one of your job classes and set `public $tries = 5;` and `public $backoff = [10, 60, 300];`. Laravel's default is a single attempt, so without this a failed job just fails.\n\nFor e-commerce developers, understanding asynchronous communication is mandatory when working with Shopify. Shopify uses webhooks to notify your application of events like orders/create or products/update.
Webhooks are asynchronous by design. Shopify does not wait for your app to process the entire order. It sends the POST request and expects a 2xx response within five seconds.
If your code tries to run complex inventory logic or sync data to an ERP synchronously within that webhook route, you risk timing out.
\nShopify then retries the failed delivery up to eight times over roughly four hours, which can mean duplicate processing if your handler is not idempotent.
\nWorse, if your endpoint keeps failing, Shopify removes the webhook subscription entirely, and you stop receiving events until you re-register it.
\nReceive the webhook, validate the signature, dispatch a Laravel Job, and return the response immediately.
\nThis is the best practice for agentic commerce on Shopify. It keeps your app responsive and keeps Shopify happy with your delivery rates.
\nMoving to an asynchronous architecture changes how you manage your infrastructure. In a synchronous world, you scale your web servers to handle peak traffic. In an asynchronous world, you scale your workers.
\nTools like Coolify and Docker make it easier to manage these separate services. You can run your web container on one set of resources and your worker containers on another.
\nIf you get a massive spike in background jobs, you can spin up more workers without affecting the performance of your main website.
\nFor cross-service communication, Google Cloud Tasks is a strong alternative to a self-hosted queue. It dispatches HTTP callbacks to any endpoint with built-in rate limiting, scheduling, and retries.
\nThis is particularly useful when building an API gateway for AI stacks, where different microservices need to talk to each other without blocking the main user flow.
\nChoosing the right pattern depends on the user's expectations and the reliability of the task.
\n| Feature | \nSynchronous | \nAsynchronous | \n
|---|---|---|
| User Experience | \nImmediate feedback required. | \nBackground processing is okay. | \n
| Data Integrity | \nResult needed for next step. | \nEventual consistency is okay. | \n
| Reliability | \nFails if the connection breaks. | \nCan retry on failure (set $tries). | \n
| Complexity | \nSimple code structure. | \nRequires queue management and workers. | \n
| Example Use Case | \nLogin validation, viewing a cart. | \nOrder fulfillment, image processing. | \n
How are you currently handling long-running API tasks in your Laravel or Shopify application?
\nA solutions architect owns the blueprint. A forward deployed engineer owns the outcome. When a project fails, it usually fails in the gap between them.
\n\n- Solutions architects design: they turn business requirements into a low-risk architecture, with diagrams and design docs, then hand off to delivery.\n- Forward deployed engineers (FDEs) build: they work inside the customer's environment, write the production code and own whether it works.\n- Pick by product: integration-heavy but repeatable products need architects; complex platforms that must be adapted to each customer's data need FDEs.\n\nWriting code is only part of the job. Where that code sits in the customer's value chain matters just as much, and the difference between a solutions architect and a forward deployed engineer decides who is accountable when it lands there.
\nMany organizations invest millions in complex software platforms like Shopify Plus or custom Laravel ecosystems, only to realize they cannot bridge the gap between a sales demo and a running production system.
\nThe implementation phase becomes a graveyard of missed requirements and technical debt. Sales cycles stall because the \"how\" is missing. This is exactly where the two roles diverge.
\nBoth roles sit where engineering meets customer success, but they solve different problems. One builds the blueprint so the house will stand. The other moves onto the construction site to make sure the pipes actually connect to the city main.
\nThe Solutions Architect is the strategic engine of a technical engagement. Their main goal is to design a scalable, low-risk implementation architecture that maps customer business requirements to technical capabilities.
\nThey operate in the space between a \"yes\" from the buyer and a \"go-live\" from the developers.
\nAn architect focuses on the big picture. They evaluate how a new Laravel development project will integrate with existing legacy databases, or how a Shopify app will interact with a third-party ERP.
\nTheir deliverables are not usually production pull requests. Instead, they produce sequence diagrams, entity-relationship diagrams (ERDs), and security compliance documents.
\nThey spend their days in design workshops and advisory conversations. They must convince a CTO that the proposed cloud infrastructure is secure, while proving to the marketing manager that the data flow will support their holiday campaigns.
\nThe Solutions Architect owns the design. They make sure that if the plan is followed, the system will be stable, secure, and performant.
\nOnce the blueprint is approved, though, they often move on to the next challenge and hand the execution to a delivery team.
\nThe Forward Deployed Engineer is a hybrid role born in high-stakes enterprise software. Pioneered by Palantir to embed engineers in messy, sensitive customer data environments, the FDE is a full-stack engineer who works directly inside the customer's environment.
\nThey write the production code that makes the system real.
\nIf a Solutions Architect is a designer, the FDE is a lead engineer who is also a specialized diplomat. They inherit the \"messy\" reality of customer data, undocumented APIs, and internal politics.
\nWhen a RAG pipeline fails because the customer's vector database has inconsistent metadata, the FDE is the one debugging the ingestion script at 2:00 AM.
\nThe FDE owns the outcome. They are measured by whether the system actually works in production and delivers business value.
\nThis role needs a high tolerance for ambiguity and the ability to pivot when a design meets the harsh reality of production.
\nThey are also the company's best feedback loop. They find the bugs and missing features in the core product and push those requirements back to the internal product teams.
\nThe divide between these roles comes down to where they sit in the software development lifecycle (SDLC) and who owns the risk.
\n| Feature | \nSolutions Architect | \nForward Deployed Engineer | \n
|---|---|---|
| Primary output | \nBlueprints, PoCs, design docs | \nProduction code, integrations, live systems | \n
| Lifecycle stage | \nPre-sales and discovery | \nImplementation and post-launch | \n
| Code ownership | \nReference implementations only | \nDirect commits to customer repos | \n
| Main stakeholder | \nCTOs and architects | \nEngineering managers and devs | \n
| Goal | \nRisk reduction and scalability | \nSuccessful deployment and value delivery | \n
A Solutions Architect might design a multi-region deployment strategy on Google Cloud Platform (GCP) for a global e-commerce brand. They specify the load balancers, the database replication strategy, and the caching layers.
\nA Forward Deployed Engineer then takes that design and writes the Terraform scripts, configures the Docker containers, and debugs the latency issues that appear when the first 10,000 users hit the site.
\nThe architect operates at a level of abstraction. The FDE operates at a level of execution.
\nThis is why FDEs are often found in companies selling complex, horizontal platforms that need significant customization to be useful.
\n\nLook at your last delayed project and find the moment the design doc was \"done.\" List every problem that showed up after that point. If most of them needed code, not a better diagram, you were missing an FDE.\n\nTo see how these roles interact, picture the deployment of an agentic AI system for a large retailer. The stack is complex: a Laravel backend, a Shopify frontend, and an AI orchestration layer using Claude and MCP.
\nThe Solutions Architect starts the project. They define how the Claude MCP dev tools will interface with the retailer's private product catalog.
\nThey map out the authentication flow so the AI agent cannot access sensitive customer financial data. They produce a high-level technical document that the retailer's security team signs off on.
\nOnce the \"paperwork\" is done, the Forward Deployed Engineer moves in. They realize the product catalog API is much slower than promised, so they write a custom Redis caching layer in the Laravel middleware.
\nThey spend three days pairing with the retailer's frontend team to integrate the AI chat widget into a bespoke Shopify theme.
\nThey are also the ones who notice the RAG system needs a different chunking strategy for the type of PDF manuals the retailer uses. They fix it in the code.
\n\nChoosing between a Solutions Architect and a Forward Deployed Engineer depends on your product's complexity and your business model.
\nIf you are a consultancy or software vendor with a product that is \"plug-and-play\" but needs significant integration planning, a Solutions Architect is your best bet.
\nThey can handle five or six customers at once, giving the high-level guidance that keeps projects moving without getting stuck in any single code repository.
\nIf you are building \"deep tech\" or complex platforms like AI-driven supply chain tools or custom enterprise ERPs, you need Forward Deployed Engineers.
\nThese products are never truly \"plug-and-play.\" They need an engineer to sit with the customer for months, adapting the product to the data.
\n\nAsk your last three customers how long it took from signed contract to first real value in production. If the answer is months and full of custom code, plan for FDEs in your next hire.\n\nFrom a candidate's point of view, the choice comes down to the \"blank page\" versus the \"broken system.\"
\nArchitects enjoy the creative challenge of designing from scratch and the variety of working across many industries. FDEs enjoy shipping code that solves a tangible, immediate problem for a person they work with every day.
\nIf you are weighing engineering tracks more broadly, the same blueprint-versus-operations split shows up in SRE vs platform engineer and in cloud engineer vs DevOps engineer.
\nA project rarely fails because of a bad design or a bad implementation alone. It fails in the gap between the two, so the best teams make these roles work in tandem.\n
\nIf you need a Forward Deployed Engineer to close that gap, here's how I help teams ship.
\nIf you are scaling a technical team right now, are you optimizing for the number of designs approved or the number of systems successfully deployed into production?
\nREST is built for humans and browsers. gRPC is built for machines talking to machines. Good architectures use each where it fits.
\n\n- gRPC sends compact binary Protocol Buffers over HTTP/2, with a contract-first `.proto` schema, generated stubs and native streaming.\n- REST sends human-readable JSON over plain HTTP that every browser and tool understands, which makes it the default for public APIs.\n- The common pattern: REST at the public edge, gRPC between internal services.\n\nChoosing between REST and gRPC is a choice of architectural philosophy as much as protocol, and it shapes how your services communicate.
\nPick the wrong one for a high-traffic microservice environment and you will spend your time fighting network latency and rising cloud costs as JSON payloads bloat under load.
\nForce a binary protocol onto a public-facing web client and you break the interoperability that makes the modern web work.
\nThe choice between Representational State Transfer (REST) and gRPC (a Google-built RPC framework whose name is a recursive joke: \"gRPC Remote Procedure Calls\") is a trade-off between human readability and machine efficiency.
\nREST has been the champion of the web for decades, while gRPC has quickly become the preferred choice for internal service-to-service communication. Understanding both is essential for any engineer building scalable systems in 2026.
\nAt its core, gRPC is built to be fast. It gets there by moving away from the text-based nature of REST and using a binary format called Protocol Buffers (Protobuf).
\nIn a typical REST interaction, your server takes an object, serializes it into a JSON string, and sends it over the wire. The receiving server then parses that string back into an object.
\nThis process is CPU-intensive and creates large payloads, because JSON includes every key name in every single message.
\nProtobuf is a strongly typed binary serialization format. Because the schema is defined in advance in a .proto file, the messages do not need to include field names. Instead, they use small numeric tags to identify each field.
The payload savings depend heavily on data shape. Numeric and structured data shrinks a lot, often to several times smaller than the equivalent JSON, while large free-text fields see less benefit.
\nEither way, binary Protobuf parsing is typically much faster than JSON parsing, which cuts serialization CPU in high-throughput environments.
\nAnother pillar of gRPC is HTTP/2. Many REST implementations still run on HTTP/1.1, but gRPC requires HTTP/2, which brings request multiplexing.
\nIn HTTP/1.1, a client effectively sends one request at a time over a connection, which leads to application-level head-of-line blocking. HTTP/2 multiplexes many requests and responses over a single TCP connection and removes that HTTP-layer blocking.
\n(TCP-level head-of-line blocking still exists under packet loss. That is the problem HTTP/3 and QUIC were designed to solve.) This efficiency matters for modern API gateway architectures that handle thousands of concurrent requests.
\nIf gRPC is so much faster, why hasn't it replaced REST? The answer lies in the browser.
\nBrowsers cannot speak native gRPC directly. gRPC-Web exists, but it needs a proxy in front of your services and does not support every streaming mode.
\nREST was born from the architecture of the web itself. It treats every piece of data as a resource that can be accessed with standard HTTP verbs like GET, POST, PUT, and DELETE. This resource-oriented approach makes it intuitive and easy to consume.
\nEvery web browser, every language, and every command-line tool like curl understands REST. If you are building a public API that third-party developers will use, REST is usually the right choice.
The barrier to entry is almost zero. A developer can open their browser console and immediately see what your API returns in a human-readable format. This \"inspectability\" is a big advantage for debugging and onboarding.
\nREST is also flexible. It does not require a strict contract to function. Tools like OpenAPI (Swagger) provide structure, but a REST API can still evolve loosely.
\nThis flexibility suits startups and small teams that need to iterate quickly. When you are deploying a new SaaS product on Coolify or Docker, the simplicity of a RESTful interface often outweighs the performance gains of gRPC in the early stages.
\nOne of the biggest differences between these two philosophies is how you actually write code. gRPC forces a \"contract-first\" approach.
\nYou must define your service and your message types in a .proto file before you write a single line of application logic.
// A typical gRPC service definition\nservice ProductService {\n rpc GetProduct (ProductRequest) returns (ProductResponse);\n}\n\nmessage ProductRequest {\n string id = 1;\n}\n\nmessage ProductResponse {\n string name = 1;\n double price = 2;\n}\nOnce this file is defined, gRPC tools generate client and server stubs in the language of your choice. This gives you strong typing across services.
\nIf you have a Go backend talking to a Python microservice, the Protobuf contract ensures both sides agree on the data structure. This removes a whole class of bugs caused by missing fields or incorrect types in JSON.
\nREST typically follows a \"resource-first\" or \"implementation-first\" approach. You define your routes and your controllers, and then you might generate documentation later.
\nThis allows faster prototyping, but it can lead to \"API drift,\" where the documentation and the actual implementation fall out of sync. In a large microservice ecosystem, this lack of strict contracts can become a maintenance nightmare.
\n\nAnother area where gRPC shines is native streaming. Because it is built on HTTP/2, gRPC supports four types of communication:
\nThis makes gRPC a strong candidate for real-time applications like chat services, stock tickers, or IoT telemetry feeds.
\nIn the world of REST, you would typically reach for WebSockets or Server-Sent Events (SSE) to get similar results. These are often treated as \"add-ons\" to the API rather than a core part of the protocol. With gRPC, streaming is built in.
\nImagine a system that processes a large dataset of vector embeddings for an AI application. A client could stream the data to the server, and the server could stream back the results as they are processed.
\nThis reduces the memory footprint on both ends, because the entire dataset never needs to be loaded into RAM at once.
\nBase the decision on your \"consumer.\"
\nIf your consumer is a human developer or a web browser, choose REST. The ecosystem of tools, from Postman to Chrome DevTools, is simply too mature to ignore. REST is the language of the public web.
\nIf your consumer is another server, or a performance-constrained device like a mobile phone or an IoT sensor, gRPC is usually the better choice. The gains in serialization and the reduced network overhead save you money on bandwidth and compute.
\nIn a modern architecture, it is common to see both living together. A RESTful API gateway handles incoming traffic from the internet, then talks to internal microservices using gRPC.
\nIf you are building for the web, start with REST. If you are building for the cloud at scale, master gRPC.
\nAs we move further into AI-integrated development, the efficiency of these data pipelines matters even more. Large language models and agentic systems need low-latency access to data, which makes gRPC a natural fit for the \"inner loop\" of AI infrastructure.
\nIf you're designing services like this for production, here's how I help teams ship it.
\nAre you prepared to handle the added complexity of a contract-first architecture in exchange for smaller payloads and faster serialization?
\nServerless wins when traffic is spiky and tasks are short. Containers win when traffic is steady and processes run long. Most 2026 stacks use both.
\n\n- Serverless (FaaS) hides the whole runtime: it scales per request and to zero, but you pay with cold starts, time limits and a stateless design.\n- Containers package the full runtime: portable, no execution limit and steady latency, at the cost of owning images, scanning and memory.\n- Serverless containers like Google Cloud Run blend the two, so your main app runs as a container while bursty side tasks use functions.\n\nDevelopers are drowning in YAML files and server patches while the business demands faster shipping. Managing infrastructure often feels like a full-time job that has nothing to do with writing actual code.
\nWhen your application hits a sudden traffic spike during a flash sale or a viral product launch, the \"504 Gateway Timeout\" becomes your worst enemy. If your infrastructure cannot scale in seconds, you lose revenue and customer trust.
\nThe choice between Serverless vs Containers is a strategic decision as much as a technical one. It sets your operational overhead and your ability to stay lean.
\nIn 2026, the lines between these technologies have blurred. We see serverless containers and event-driven architectures that mix both worlds. This guide breaks down the engineering substance of Serverless vs Containers to help you decide where to host your next Laravel application or Shopify integration.
\nThe debate over Serverless vs Containers is often simplified into \"no servers\" vs \"virtual servers.\" In reality, the difference lies in the abstraction layer.
\nServerless (often called Function-as-a-Service or FaaS) abstracts the entire execution environment. You upload code, and the provider (AWS Lambda, Google Cloud Functions) handles the trigger, the scaling, and the underlying OS.
\nIt is event-driven by design. Your code stays dormant until an HTTP request, a file upload, or a database change wakes it up.
\nContainers (Docker, Kubernetes) package the entire runtime environment: the application code, libraries, and system dependencies. You have full control over the operating system version and the configuration.
\nContainers run on infrastructure you manage (or semi-manage via Fargate or Cloud Run). They are typically \"always on\" or need specific scaling rules to spin up and down.
\n| Feature | \nServerless (FaaS) | \nContainers (Docker/K8s) | \n
|---|---|---|
| Control | \nMinimal; provider-managed | \nHigh; you define the image | \n
| Scaling | \nAutomatic per request | \nOrchestrator-managed | \n
| State | \nStrictly stateless | \nCan be stateful | \n
| Runtime | \nLimited to provider support | \nAnything that can be Dockerized | \n
| Timeout | \nUsually 15 minutes or less | \nNo inherent execution limit | \n
Serverless is the tool for developers who want to ignore infrastructure. The main advantage is the \"scale-to-zero\" model. When no one is using your app, you pay nothing.
\nWhen traffic surges, the provider spins up new instances of your function automatically, within account concurrency limits. AWS Lambda defaults to 1,000 concurrent executions per region (raisable on request) and scales out at roughly 1,000 new environments every 10 seconds.
\nThe catch is the \"cold start.\" When a function hasn't been used recently, the provider must provision a container and boot your runtime.
\nLightweight runtimes like Node.js and Go start fast natively, and provisioned concurrency keeps instances warm. Heavier runtimes feel it more.
\nThat is why AWS built SnapStart (first for Java, later .NET and Python) to snapshot an initialized runtime and cut cold starts sharply. For frameworks like PHP, cold starts remain a factor that needs careful planning.
\nServerless is perfect for \"glue code.\" If you are building a Shopify app architecture that needs to process webhooks or resize images on the fly, serverless functions are very efficient.
\nThey absorb the bursty, unpredictable nature of webhooks without a dedicated server sitting idle 99% of the time.
\nContainers offer a level of portability that serverless cannot match. A Docker image that runs on your local machine runs the same way on AWS ECS, Google Cloud Run, or a self-hosted Coolify instance.
\nFor long-running processes or heavy compute tasks, containers are the standard. If your application needs a persistent WebSocket server, heavy machine learning inference, or long-running cron jobs that exceed 15 minutes, containers are the natural fit.
\n(Managed add-ons like API Gateway WebSockets can bridge some of this for functions, but the long-lived process itself belongs in a container.)
\nThe main drawback is the \"operational tax.\" Even with managed services like AWS Fargate, you are still responsible for your Dockerfiles, image security scanning, and making sure your containers don't run out of memory.
\nYou are managing the environment, even if you aren't managing the physical hardware.
\n# A typical Laravel Dockerfile for a containerized approach\nFROM php:8.4-fpm-alpine\n\n# Install system dependencies\nRUN apk add --no-cache libpng-dev libzip-dev zip unzip git\n\n# Install PHP extensions\nRUN docker-php-ext-install pdo_mysql gd zip\n\n# Set working directory\nWORKDIR /var/www\n\n# Copy application code\nCOPY . .\n\n# Install composer dependencies\nRUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer\nRUN composer install --no-dev --optimize-autoloader\n\nCMD [\"php-fpm\"]\nWhen comparing Serverless vs Containers, cost is the most common point of confusion. Serverless looks cheaper because you only pay for what you use. This is true for low-traffic or spiky applications.
\nAt a certain level of sustained traffic, containers become more cost-effective. Once your application handles thousands of requests per minute consistently, the per-invocation cost of serverless starts to exceed the monthly cost of a reserved container instance.
\n\nLatency is the other big trade-off. Containers offer \"warm\" execution. Since the process is already running, the time-to-first-byte (TTFB) is consistent.
\nServerless performance can be \"jittery.\" One request might take 30ms while the next (a cold start) takes several hundred milliseconds.
\nFor an interactive admin dashboard, this inconsistency can frustrate users. It also ties directly into how you architect stateless vs stateful apps, since serverless forces a strictly stateless design.
\n\nTake last month's request count and average duration, and price them on your provider's function calculator. Then price one always-on container that can handle your peak. The cheaper line tells you which side of the crossover you are on.\n\nFor a modern Laravel application, the choice often comes down to Laravel Vapor (Serverless) vs Docker on Cloud Run/Coolify (Containers).
\nIf you are building a SaaS with unpredictable growth, Laravel Vapor is a strong option. It handles the complexity of deploying a full-stack PHP framework to AWS Lambda.
\nIt manages your assets on S3 and handles database scaling. It is the \"easy button\" for high-scale Laravel apps that don't want a dedicated DevOps engineer.
\nIf your application is \"always on\" and involves heavy background processing (like syncing thousands of products for a Shopify store), containers are the better fit.
\nDocker lets you run your HTTP layer, your queue workers, and your scheduler in one consistent environment. This setup matches your local development environment, which cuts down the \"it works on my machine\" bugs.
\nModern engineering has moved past the binary choice of Serverless vs Containers. We are now in the era of serverless containers.
\nServices like Google Cloud Run let you package your app as a container and still get the autoscaling and pay-per-use benefits of serverless.
\nAWS App Runner played that role on AWS, but it stopped accepting new customers on April 30, 2026. AWS now points new container apps to ECS Express Mode, which runs on Fargate and does not scale to zero.
\nYou can deploy your main Laravel API as a container on Cloud Run, which scales to zero when idle. At the same time, you can use pure serverless functions (like AWS Lambda) for specific, isolated tasks like PDF generation or sending transactional emails.
\nYour core business logic stays in a predictable container, while your bursty, auxiliary tasks scale independently.
\nThis hybrid approach puts each workload in the environment that suits it.
\nIf you're weighing serverless against containers, here's how I help teams make that call.
\nIf you had to migrate your core API today, would you prioritize the absolute control of a Dockerfile or the operational freedom of an event-driven function?
\nReplication copies your data everywhere in milliseconds, including the mistake that just deleted it. Only a backup lets you go back in time.
\n\n- Replication is for uptime: replicas take over when hardware fails and can serve reads, but they mirror every bad write instantly.\n- Backups are for recovery: point-in-time snapshots on separate storage are your only way back from bugs, bad migrations and human error.\n- In Laravel, use `read`/`write` connections with `sticky` for replicas, keep real backups beside any Shopify sync, and test your restores.\n\nYou deploy a breaking database migration that drops a critical column. Your production database replicates in real time to three nodes across two regions.
\nWithin milliseconds, that destructive change reaches every one of your \"safety\" copies. You have zero downtime, and that column is gone from every copy.
\nThis is the moment you realize that replication is not a backup.
\nEngineering teams often confuse high availability with data durability. Both matter, but they solve different problems.
\nReplication protects you from hardware failure and latency. Backups protect you from logic errors, corruption, and human mistakes.
\nIf you are building a Laravel application or a high-volume Shopify Plus integration, knowing where one ends and the other begins is the difference between a minor incident and a company-ending event.
\nReplication is about continuity. Its goal is that if your main database server vanishes, a secondary node is ready to take over immediately.
\nThis is measured in Recovery Time Objective (RTO). In a well-tuned system, failover happens in seconds. You are mirroring every INSERT, UPDATE, and DELETE as they happen.
Backups are about recovery. Their goal is a safe version of your data from a specific point in history.
\nIf a bug corrupts your inventory levels at 2:00 PM, a backup from 1:00 PM is your only lifeline. Backups are usually stored as compressed snapshots on detached storage like Amazon S3 or Google Cloud Storage.
\n| Feature | \nReplication | \nBackup | \n
|---|---|---|
| Primary Goal | \nHigh Availability (HA) | \nDisaster Recovery (DR) | \n
| Data State | \nReal-time / Live | \nHistorical / Snapshot | \n
| Propagation | \nImmediate (including errors) | \nNone (isolated from live changes) | \n
| Storage Cost | \nHigh (hot, identical hardware) | \nLow (cold, compressed snapshots) | \n
| Recovery Speed | \nSeconds to Minutes | \nMinutes to Hours | \n
The greatest strength of replication is also its greatest weakness. It is designed to be a mirror.
\nIf you run a truncate command on your primary database, the replication protocol assumes it was intentional. It will purge that data from your replicas before you can even hit \"Ctrl+C\" in your terminal.
I have seen developers rely solely on RDS Read Replicas as their \"safety net.\" When a rogue queue job began overwriting customer email addresses with null values, the replicas updated instantly.
\nWithout a point-in-time backup, those original emails were gone forever. Replication provides infrastructure resilience, not data integrity.
\nFor a production-grade Coolify self-hosted SaaS, you must separate your concerns.
\nUse replication to scale your reads and handle node failures. Use automated, encrypted backups so you can roll back to a known-good state.
\nLaravel handles read/write splitting through native configuration. By defining read and write connections in your config/database.php, the framework automatically routes SELECT statements to your replicas while sending INSERT and UPDATE statements to the primary.
This is the same connection layer you tune when designing a multi-tenant Laravel architecture.
\n'mysql' => [\n 'read' => [\n 'host' => [\n '192.168.1.10', // Replica 1\n '192.168.1.11', // Replica 2\n ],\n ],\n 'write' => [\n 'host' => [\n '192.168.1.1', // Primary\n ],\n ],\n 'sticky' => true,\n 'driver' => 'mysql',\n // ... other settings\n],\nsticky mattersThe sticky option is important here. If you write a record during a request, any later reads during that same request come from the primary.
This avoids \"read-after-write\" lag, where a replica is a few milliseconds behind the primary and your application looks as if the data vanished.
\n\nCheck your `config/database.php`. If you have `read` hosts but no `'sticky' => true`, add it, then test a \"create and redirect to the new record\" flow under load.\n\nIn the world of Shopify Plus, many developers build \"replication-style\" sync engines. They listen for orders/create or products/update webhooks and mirror that data into a local Laravel database.
This is a useful pattern for custom reporting or complex agentic commerce workflows.
\nHowever, this local database is often mistaken for a backup. If an admin deletes a collection of products in the Shopify admin panel, Shopify fires a webhook.
\nYour Laravel app receives that webhook and promptly deletes those products from your local DB to stay \"in sync.\"
\n\nIf you do not have a separate backup strategy that creates daily snapshots of your local DB, your local data is just as ephemeral as the live Shopify data.
\nTo build true resilience, store historical JSON payloads of those Shopify resources in a versioned storage system like S3. This lets you reconstruct your state even if the live sync deletes everything.
\nManaging replication by hand is a DevOps nightmare. For most Laravel applications, managed services like AWS RDS or Google Cloud SQL are the correct move. They handle the binary log replication, failover orchestration, and monitoring out of the box.
\nWhen configuring your cloud DB, you will see \"Multi-AZ\" (AWS) or \"High Availability\" (GCP) settings.
\nA single-standby Multi-AZ deployment uses synchronous replication. It roughly doubles your database cost, because you run a standby instance in a second Availability Zone that does nothing but wait for the primary to die.
\nThis standby cannot serve read traffic. To scale reads you need separate read replicas (or RDS Multi-AZ DB clusters), not the HA standby.
\nBackups, by contrast, are very cheap. Storing 500GB of compressed dumps in S3 Standard costs roughly $11 to $12 a month. Archival tiers like Glacier Deep Archive drop that to around 50 cents, at the cost of multi-hour retrieval.
\nThe cost of a backup is not in the storage; it is in the testing. A backup that has never been restored is just a collection of random bits. You must automate your restore tests.
\nTime the download too: my transfer time calculator shows how long 500GB takes on your link.
\nI recommend using Coolify and Docker to spin up ephemeral environments where you can test your backup restoration process regularly without touching production.
\n\nSchedule a monthly job that restores last night's backup into a throwaway container, runs `php artisan migrate:status` and a row count on your key tables, then deletes the container. Alert if any step fails.\n\n\n- **Replication is for uptime.** It allows your app to stay online during hardware failures or network partitions.\n- **Backups are for survival.** They are your only defense against data corruption, malicious attacks, and developer errors.\n- **Replication propagates bugs.** If your code breaks the data, the replica will break just as fast.\n- **Laravel supports read/write splitting.** Use the `sticky` configuration to prevent consistency issues during web requests.\n- **Shopify webhooks are not backups.** A synchronized local database is a replica, not a historical record.\n- **Test your restores.** Automate a process to verify that your backup snapshots actually work.\n\nIf you're architecting that resilience layer for production, here's how I help teams ship it.
\nIf you had to choose between a system with 99.99% uptime but no backups, and a system with 95% uptime but hourly backups, which one would keep you from losing your business?
\nRAG gives your model an open book at runtime. Fine-tuning changes how the model behaves. Use RAG for facts and fine-tuning for style.
\n\n- RAG retrieves relevant chunks from your own data at request time, so answers stay fresh and access control stays in your database.\n- Fine-tuning retrains the model's weights on a focused dataset to lock in tone, terminology and response patterns, with lower latency per request.\n- Many production systems combine them: a fine-tuned model for behavior, RAG for live facts.\n\nLarge language models are only as good as the information they can access. You build a custom application and realize the base model knows nothing about your internal documentation or yesterday's product launch. It starts making things up.
\nThis hallucination problem is the biggest hurdle for developers moving from a prototype to production, and it is exactly where the RAG vs fine-tuning decision gets made.
\nYou can either give the model a search engine to find the right information at runtime, or retrain the model to bake that information into its weights. These two methods are Retrieval-Augmented Generation (RAG) and fine-tuning.
\nChoosing the wrong one wastes engineering hours and runs up infrastructure bills. A RAG system might be too slow for a high-traffic API. A fine-tuned model might become obsolete the moment you update your pricing page.
\nRetrieval-Augmented Generation is like giving an LLM an open-book exam. Instead of relying on the model to remember everything from its original training, you provide relevant documents along with the user prompt.
\nThe model then uses this context to generate an accurate answer.
\nThe workflow has a few moving parts. First, you convert your documents into numerical representations called embeddings and store them in a vector database like pgvector or Pinecone.
\nWhen a user asks a question, your system runs a semantic search to find the most relevant \"chunks\" of text. These chunks are then injected into the prompt window of the LLM.
\nRAG is the standard choice for applications where data changes frequently. If you are building a support bot for a Shopify store, it needs to know about current inventory levels.
\nRetraining a model every time a product sells out is not practical. With RAG, you just update the vector index. You can read more about avoiding common pitfalls in our guide on 7 RAG mistakes in production.
\nFine-tuning takes a pre-trained model and continues its training on a smaller, specialized dataset. You are changing the \"brain\" of the model by adjusting its internal weights.
\nThe model then picks up the patterns, tone, and terminology of your domain without needing external documents for every query.
\nThis approach is less about adding new facts and more about teaching the model how to act. It shines when you need a consistent style, tone, or response pattern that prompting alone cannot pin down reliably.
\n(For strict JSON shape, reach for structured outputs or constrained decoding first. Every major provider now guarantees schema-valid output at inference time, no training run required.)
\nFine-tuning is also common in specialized industries like legal or medical tech, where the model learns the specific \"language\" of the field.
\nThe major drawback is that it creates a static snapshot of knowledge. If the underlying facts change, the model stays stuck with its training data until you run another expensive training cycle.
\n\nPerformance is where these two architectures split sharply. RAG adds several steps to every single request.
\nYour application must embed the query, search the vector database, and then send a much larger prompt (with the retrieved text) to the LLM. Each step adds milliseconds or even seconds to the total response time.
\nFor high-traffic applications, this latency can be a deal-breaker. The retrieval step also means standing up and querying a vector store alongside your app. See picking the right RAG stack for how that choice shapes both speed and cost.
\nFine-tuning offers much lower latency at runtime. There is no external retrieval step, and the prompt stays short because the model already \"knows\" the context.
\nThis makes fine-tuned models a good fit for real-time applications or high-volume background tasks like sentiment analysis or lead scoring.
\nThe cost of RAG is mostly operational. You pay for storing and querying the vector database.
\nYou also pay more for LLM tokens because your prompts are much longer. Over millions of requests, these \"context tokens\" add up quickly. RAG is cheaper to set up but can become more expensive at extreme scale.
\nFine-tuning has a high upfront cost. You need to curate a high-quality dataset, which often requires human labeling, and significant GPU power to run the training.
\nOnce the model is trained, your per-query costs are lower. The prompts are shorter and you do not need to maintain a complex retrieval pipeline.
\nThe deeper split is operational. Fine-tuning lives in the training world (GPUs, datasets, MLOps), while RAG lives in the inference-time data-pipeline world.
\nDecide which world your team is actually equipped to run.
\n\nMeasure the average prompt size of your RAG calls in tokens, multiply by your monthly request count and your model's input price. That is your monthly \"context tax,\" the number to compare against a fine-tuning run.\n\nFor enterprise clients, data privacy is often the deciding factor. RAG offers better data governance.
\nSince the information is retrieved from your own database at runtime, you can apply standard access controls and make sure a user only retrieves documents they are authorized to see. The LLM never \"learns\" the data permanently.
\nFine-tuning is riskier here. If you train a model on sensitive customer data, that information is baked into the model weights.
\nMembership inference and training-data extraction attacks are well-documented ways to pull memorized data back out.
\nThis makes it hard to \"delete\" a user's data from a fine-tuned model short of retraining, which can lead to compliance issues under regulations like GDPR.
\nIn 2026, many advanced systems do not choose one over the other. They combine both.
\nYou might fine-tune a smaller, cheaper model (like a Llama variant) to learn your company's specific API schemas and brand voice. Then you use RAG to give that model access to live data.
\nThis combination lets the model be both fast and accurate. Fine-tuning handles the \"how\" (formatting and style) while RAG handles the \"what\" (current facts and data).
\nThis works especially well with Claude and the Model Context Protocol (MCP), where the model can use specific tools to fetch data while keeping a trained understanding of the environment.
\n\nBefore you fine-tune anything, write a strong system prompt plus five example answers in your target style and run it through your RAG pipeline. If that closes the gap, you may not need a training run at all.\n\n\nChoosing between RAG and fine-tuning depends on your goals for data freshness, latency, and budget.\nIf you're deciding between these paths for a real product, here's how I help teams ship it.
\nIs your current AI architecture bottlenecked by the speed of retrieval or the accuracy of the model's internal knowledge?
\nRate limiting says \"no\" once you go over the line. Throttling says \"slow down.\" Pick the wrong one and your users get errors they didn't need, or your database melts anyway.
\n\n- Rate limiting is a policy: X requests per Y time, then an instant HTTP 429. Use it for fairness, quotas and blocking abuse.\n- Throttling shapes traffic: it delays or queues requests so a spike drains at a pace your backend can handle.\n- Use both: rate limit at the edge, throttle inside your app around databases and third-party APIs, and track the counters in Redis.\n\nYour server is screaming, not because of a bug, but because of a success. A viral tweet or a massive batch of automated requests has flooded your API.
\nWithout rate limiting or throttling in place, your database locks up, your CPU pegs at 100%, and the site goes down for everyone.
\nThis is the nightmare scenario for any engineering team. To prevent it, you need to control the flow of traffic.
\nEngineers often use \"rate limiting\" and \"throttling\" interchangeably. Both deal with traffic management, but they solve the problem with different philosophies.
\nChoosing the wrong one can lead to a degraded user experience or, worse, a complete system failure under pressure. This guide breaks down both strategies and how to implement them in stacks like Laravel and Shopify.
\nThe problem starts with resources. Every server has a finite amount of memory, compute, and bandwidth.
\nWhen requests come in faster than your system can process them, a queue builds up. Once that queue exceeds the system's capacity, the service fails.
\nThis is not only about malicious actors or DDoS attacks. It is often about \"noisy neighbors\" in a multi-tenant environment, or a poorly optimized loop in a client-side application.
\nI have seen production environments crash simply because a mobile app's \"retry\" logic was too aggressive during a minor network hiccup.
\nRate limiting and throttling are your tools to enforce discipline. They make sure no single user or service can monopolize your infrastructure. They are the gatekeepers of your cloud infrastructure.
\nRate limiting is a policy-based approach. It defines a strict contract: \"You are allowed exactly X requests per Y amount of time.\" Once a user reaches that limit, the gate shuts.
\nWhen a request exceeds the limit, the server rejects it immediately, typically with an HTTP 429 \"Too Many Requests\" status code.
\nThe server spends no more resources on that request. It does not look up data in the database. It just says \"no.\"
\nRate limiting is mainly about fairness and security. It protects your API gateway from abuse.
\nIf you run a SaaS, you might have different tiers. A free user gets 60 requests per minute, while a premium user gets 1,000.
\nRate limiting is easy to communicate to users. They can check their response headers (like X-RateLimit-Limit and X-RateLimit-Remaining) to see exactly where they stand.
It is a binary state: you are either within your limit or you are blocked.
\nThrottling is a runtime behavior designed to \"shape\" traffic. Instead of a hard rejection, throttling slows down the processing of requests.
\nThink of it like a funnel. You can pour a bucket of water into it all at once, but it only drips out at a controlled, steady speed.
\nIn a throttled system, if you send too many requests, the server might add a delay to each response. It might queue the requests and process them as capacity becomes available.
\nThe goal is to smooth out spikes and avoid a \"jagged\" traffic pattern.
\nThrottling is excellent for protecting backend resources like databases or third-party APIs. If you know your database can only handle 500 writes per second, you might throttle incoming requests to stay just under that limit.
\nIt gives a better experience during occasional spikes. Instead of a 429 error, the user might just see a slightly longer loading time.
\nIf the traffic stays high for too long, though, the queue eventually fills up, and the system has to start rejecting requests anyway.
\n\nTo implement these strategies, engineers rely on specific models. Knowing them helps you tune your system.
\nImagine a bucket that holds \"tokens.\" Every time a request comes in, a token is removed. If the bucket is empty, the request is rejected. Tokens are added back to the bucket at a constant rate.
\nImagine a bucket with a small hole at the bottom. Requests are poured into the bucket. They \"leak\" out of the hole at a constant rate to be processed. If the bucket overflows, new requests are dropped.
\nHow does this look in the real world? Here are two ecosystems where these patterns matter.
\nLaravel makes rate limiting simple through its RateLimiter facade and the throttle middleware. The built-in limiter uses a fixed-window counter backed by your cache store.
In Laravel 11 and 12, you define named limiters in the boot method of AppServiceProvider (the old RouteServiceProvider was dropped from the default skeleton):
use Illuminate\\Cache\\RateLimiting\\Limit;\nuse Illuminate\\Http\\Request;\nuse Illuminate\\Support\\Facades\\RateLimiter;\n\nRateLimiter::for('api', function (Request $request) {\n return Limit::perMinute(60)->by($request->user()?->id ?: $request->ip());\n});\nThis tells Laravel to allow 60 requests per minute per user ID or IP address. If the limit is hit, Laravel throws a ThrottleRequestsException, which results in a 429 response.
Despite the middleware's name, this is a classic \"hard stop\" rate limit.
\nShopify uses a leaky bucket algorithm for its GraphQL Admin API on every plan, not just Plus.
\nWhen you make a GraphQL call, the response carries cost details under extensions.cost, including the requestedQueryCost and a throttleStatus.
THROTTLED error.Strictly speaking, Shopify itself rejects an over-budget query. The leaky bucket sets the pace, and the throttling (slowing down) is what your client has to do to match it.
\nFor high-volume commerce, that means building back-off logic into your application. A well-behaved client reads throttleStatus.currentlyAvailable and slows down before it hits the wall, rather than firing blindly and retrying on every THROTTLED error.
When you do get throttled, wait long enough for restoreRate to refill the points your next query needs.
| Feature | \nRate Limiting (Laravel Default) | \nThrottling (Shopify-style leaky bucket) | \n
|---|---|---|
| Primary Action | \nBlock (HTTP 429) | \nClient delays or queues to match the leak rate | \n
| Best For | \nSecurity & Quotas | \nResource Stability | \n
| Logic | \nFixed Window | \nLeaky Bucket | \n
| User Experience | \nInstant Error | \nLatency Spike | \n
When designing your system, you must decide where to place these controls.
\nImplement rate limiting at the WAF or API gateway level. This stops malicious traffic before it even touches your application code, which saves money on compute.
\nImplement throttling within your service logic. Use it when calling external APIs or writing to a shared database. Use a queue system like Redis or Amazon SQS to hold requests that exceed your immediate capacity.
\nRate limiting at the perimeter blocks the \"bad actors,\" while internal throttling keeps your services from melting down during a legitimate surge.
\nI have found that the most resilient systems use both. This is especially important when building agentic commerce systems that may generate many automated API calls in a short period.
\nIf those calls hit a paid LLM, the stakes shift from CPU to your bill. See rate limiting and the denial-of-wallet problem for the token-cost angle.
\nRetry-After header so they know when they can try again. My HTTP headers cheat sheet shows example values.If you're hardening an API for production, here's how I help teams ship it.
\nHow do you decide between rejecting a request immediately or making the user wait 500ms longer to maintain system stability?
\nTraditional RAG retrieves once and hopes. Agentic RAG plans and loops. Corrective RAG checks the context before it answers. Each step up costs more latency and tokens.
\n\n- Traditional RAG is one search and one LLM call: fast and cheap, but it has no way to notice when the retrieved context is wrong.\n- Agentic RAG puts an LLM agent in a plan-act-observe loop that can split questions, call tools and search again, at higher cost and latency.\n- Corrective RAG (CRAG) adds a lightweight evaluator that grades retrieval and falls back to refinement or web search before generating.\n\nMost developers build a basic RAG system only to find it hallucinating or failing on complex questions within a week of deployment.
\nIf your vector search returns the wrong context, your LLM will confidently lie to your face. This gap between basic search and reliable answers is why RAG architectures have grown from simple pipelines into self-correcting, agentic systems.
\nThe problem with many first AI implementations is their static nature. You feed a query into a vector database, pull some chunks, and hope the LLM makes sense of them.
\nThis is the traditional RAG model. It works for simple FAQs, but it breaks down when a query needs multi-step reasoning or when the retrieved data is irrelevant. The result is \"garbage in, garbage out\": the model burns tokens trying to answer from junk context.
\nThis guide compares three RAG architectures so you can pick the right one for your workload: the speed of a traditional pipeline, the reasoning of agentic RAG, or the verification of corrective RAG (CRAG).
\nTraditional RAG is the foundation of most AI-driven applications. It follows a strictly linear, one-shot path: retrieve, then generate.
\nYou start by converting your documents into vector embeddings and storing them in a database like pgvector or Pinecone. When a user asks a question, the system embeds the query, runs a similarity search, and injects the top results into the prompt.
\nThis architecture is prized for its low latency and simplicity. If you are building a simple internal search tool for a Shopify store or a basic documentation bot, Traditional RAG is often enough.
\nIt is cost-effective because it typically involves only one LLM call and one vector search.
\nIts simplicity is also its biggest weakness. It assumes the first retrieval always succeeds.
\nIf the vector search returns noise, the LLM has no mechanism to tell that the context is wrong. It will try to answer regardless. This often leads to the 7 RAG mistakes in production that plague early-stage AI projects.
\nAgentic RAG turns retrieval from a passive pipeline into an active control loop. Instead of a fixed sequence, an LLM agent acts as a \"brain\" that manages the whole workflow.
\nThe agent can plan its approach, decide which tools to use, and iterate until it finds a satisfactory answer.
\nIn an Agentic RAG system, the agent might decide that a single search is not enough. It might break a complex request into three sub-queries, search different data sources for each, and then combine the results into one answer.
\nThis is particularly useful for agentic commerce on Shopify, where a user might ask for a comparison between several products across different categories.
\nThe core of this architecture is the plan-act-observe cycle (the same loop behind ReAct-style agents). The agent plans a step, performs an action (like calling a search tool), observes the result, and decides whether it needs more information.
\nThis lets it solve multi-hop problems, where the answer to the first part of a question provides the search terms for the second.
\nAgentic RAG is more capable, but it is also more expensive and slower than traditional methods. Each iteration requires another LLM call, which increases both the token cost and the time the user waits.
\nManaging these loops takes solid infrastructure, often including an API gateway in the AI stack to handle the extra traffic and orchestration complexity.
\nCorrective RAG, or CRAG, adds a self-correction step to retrieval. Its goal is to cut hallucinations by putting a lightweight retrieval evaluator between the retrieval and generation steps to judge how relevant the retrieved documents are.
\nThe original CRAG paper uses a fine-tuned T5-large model as that evaluator.
\nThe evaluator sorts each retrieval into one of three confidence levels: correct, incorrect, or ambiguous.
\nCRAG suits high-stakes environments where accuracy is non-negotiable. It adds a layer of grounded verification to the otherwise probabilistic behavior of LLMs, deciding whether to trust, refine, or replace retrieved context.
\nBy checking whether the ground truth is actually present in the retrieved data, CRAG keeps the model from inventing facts when the database comes up empty.
\nChoosing between these RAG architectures is a trade-off between performance, cost, and complexity. Use the table below to guide your decision.
\nIf you are still settling on the storage layer underneath, picking the right RAG stack covers the vector database choices in depth.
\n| Feature | \nTraditional RAG | \nAgentic RAG | \nCorrective RAG | \n
|---|---|---|---|
| Flow Type | \nLinear | \nCyclic / Iterative | \nEvaluative Loop | \n
| Latency | \nLow | \nHigh | \nMedium | \n
| Cost | \nLow | \nHigh | \nMedium-High | \n
| Use Case | \nSimple FAQ, Lookup | \nComplex Research, Workflows | \nHigh-Accuracy Q&A | \n
| Reliability | \nModerate | \nHigh (Reasoning) | \nVery High (Grounding) | \n
| Implementation | \nEasy | \nComplex | \nModerate | \n
For most startups, the most logical path is to start with Traditional RAG and then add a Corrective layer. You ship quickly and still have a roadmap for more reliability as your dataset grows.
\n\nBefore building a full CRAG loop, add one cheap relevance check: ask a small model \"does this context answer the question? yes/no\" before generating. Return \"I don't know\" on a no, and count how often it fires.\n\nBuilding these architectures takes more than an LLM API key. It needs a solid backend to handle state, queues, and data processing.
\nIn a Laravel environment, you can run the agentic loops with job queues and dedicated service classes. Laravel's ecosystem suits the orchestrator logic that drives an agentic RAG system.
\n// Conceptual Laravel Service for a Corrective RAG Flow\nclass RagOrchestrator {\n public function handleQuery(string $userQuery) {\n // 1. Retrieve initial chunks\n $chunks = $this->vectorStore->search($userQuery);\n\n // 2. Evaluate with a Critic\n $evaluation = $this->critic->evaluate($userQuery, $chunks);\n\n if ($evaluation->isIrrelevant()) {\n // 3. Corrective action: Web Search or Retry\n $chunks = $this->webSearch->search($userQuery);\n }\n\n // 4. Final generation\n return $this->llm->generate($userQuery, $chunks);\n }\n}\nWhen deploying these systems, Docker and cloud infrastructure management matter a lot. You need to scale your vector database and your orchestration layer independently.
\nTools like Coolify for self-hosting, or managed GCP services, help with the compute demands of multi-step agentic loops.
\n\nCache agent sub-query results in Redis, keyed by a hash of the normalized sub-query, with a short TTL. Track the hit rate for a week to see how much repeated work your loops were doing.\n\nIf you're deciding which RAG architecture fits your product, here's how I help teams ship it.
\nWhich architecture currently provides the best balance of cost and accuracy for your production workloads?
\nMicroservices are not the default goal. For most Laravel and Shopify teams, a well-structured monolith ships faster and costs less.
\n\n- A monolith is one deployable unit: simple deploys, shared memory, ACID data and easy end-to-end tests.\n- Microservices buy independent scaling, team autonomy and fault isolation, but charge a \"network tax\" of latency, partial failures and eventual consistency.\n- A modular monolith with strict internal boundaries is often the best middle ground, and it makes a later split much easier.\n\nComplexity is the silent killer of engineering velocity. You start with a clean codebase and a clear vision.
\nWithin months, your deployment times have doubled and your team is tripping over each other's pull requests. You face the classic crossroads: monolith vs microservices.
\nChoosing the wrong path early can lead to catastrophic technical debt, or operational overhead that drains your budget before you find product-market fit.
\nIn modern web development, especially with Laravel or Shopify, the pressure to adopt microservices is huge. The industry often treats distributed systems as the default goal.
\nFor many businesses, a well-structured monolith is both the starting point and the most efficient way to scale. Here is the engineering behind both patterns, to help you decide for your next project on Google Cloud or AWS.
\nA monolithic architecture is a unified software model where all components are interconnected and interdependent.
\nIn Laravel, your routes, controllers, models and background jobs live in one repository. It is a single deployable unit: when you push code, the whole application is built and shipped to your server or container.
\nMonoliths are often unfairly called \"legacy.\" In reality, they offer big advantages for rapid development.
\nAll modules share the same memory space and database, so communication between parts of the system is nearly instant. No network latency, API versioning or complex distributed transactions. For a Shopify app or a custom web solution, that simplicity means faster shipping.
\nMicroservices break the application into small, independent services that talk over a network. Each focuses on one business domain, such as billing, user management or inventory.
\nThey are often containerized with Docker and managed by orchestrators like Kubernetes or Google Kubernetes Engine (GKE).
\nThe main drivers are scalability and team autonomy. When your team grows beyond 15 or 20 developers, a monolith can become a bottleneck.
\nMicroservices let different squads own parts of the system. One team updates the billing service in Go while another updates the Shopify sync engine in PHP, and a bug in one service is less likely to take down the whole application.
\nThe choice is always a trade-off between simplicity and flexibility. There is no silver bullet: you weigh operational cost against developer experience.
\n| Feature | \nMonolith | \nMicroservices | \n
|---|---|---|
| Operational complexity | \nLow | \nHigh | \n
| Development speed | \nFast (early stage) | \nSlow (early stage) | \n
| Scalability | \nVertical / horizontal (whole) | \nIndependent per service | \n
| Data consistency | \nStrong (ACID) | \nEventual consistency | \n
| Network latency | \nMinimal | \nSignificant (API hops) | \n
| Testing | \nSimple | \nComplex distributed testing | \n
Microservices add a \"network tax.\" Every call from one service to another adds latency, and you must handle partial failures.
\nIf the user service is down, how does the order service react? Patterns like circuit breakers and retries become mandatory. That is code with nothing to do with your business logic: pure infrastructure overhead.
\nEvery service boundary you add is a network call that can fail and a contract you have to keep.
\n\nCount how many teams deploy your app independently today. If the answer is one, write down the specific pain a split would fix before you plan any services.\n\nFor most Shopify development or Laravel projects, the \"Majestic Monolith\" is the better choice.
\nShopify gives you a solid platform-as-a-service (PaaS) foundation. Your backend's main job is often to handle OAuth, process webhooks and manage a custom database.
\nSplitting a Shopify app into microservices too early often leads to \"distributed monolith\" syndrome: the complexity of microservices with components that are still tightly coupled.
\nIf you cannot deploy Service A without also deploying Service B, you have missed the benefits of the architecture. You have only added network latency and deployment pain.
\nGoogle Cloud Platform (GCP) has good tools for both patterns.
\nFor a monolith, Cloud Run is a strong choice. It hides server management and scales your container based on request traffic. It is cost-effective because you only pay when your code is running.
\nFor microservices, you might use GKE or a set of Cloud Run services connected through Pub/Sub. That gives you asynchronous communication.
\nWhen a Shopify order is created, the \"Order Service\" publishes an event to Pub/Sub. The \"Shipping Service\" and \"Email Service\" both listen for it and act independently.
\nThis architecture is capable, but it needs real investment in observability tools like Cloud Logging and Cloud Trace to debug issues across service boundaries.
\nYou don't have to choose between a \"big ball of mud\" and a complex mesh of services. The modular monolith is an effective middle ground.
\nYou keep a single codebase and database, but you strictly enforce boundaries inside the code.
\nIn Laravel, this means separate namespaces or even local packages for different domains. Your \"Billing\" code should not call the \"Inventory\" models directly. It should use internal interfaces or events.
\nYou get the deployment simplicity of a monolith while making it easy to extract a module into a standalone microservice later.
\n\nI go deeper on the mechanics (contracts, schema-per-module and the friction signals that justify a split) in modular monoliths first. \"Deferred decision making\" is often the wisest move in software engineering.
\nWhen the time comes to carve a module out, my guide on going from monolith to microservices walks through the strangler-fig migration step by step.
\n\nPick two domains in your Laravel app (for example Billing and Inventory) and search for direct model calls between them. Replace one with an interface or event this week.\n\n\n- **Start with a monolith.** Unless you are building for a large organization with multiple teams, microservice overhead will slow you down.\n- **Enforce boundaries early.** A modular structure inside your monolith prevents spaghetti code and keeps future scaling possible.\n- **Use managed services.** GCP Cloud Run for hosting and Cloud SQL for your database keep the DevOps burden low.\n- **Scale components before splitting apps.** Use queues (Laravel Horizon) and background workers for heavy tasks before moving to full microservices.\n- **Monitor and trace.** Whatever the architecture, invest in central logging and performance monitoring to find bottlenecks before users feel them.\n\nIf you're weighing a split, here's how I help Laravel and Shopify teams choose between a modular monolith and services.
\nAre you finding that your current monolithic deployment is hitting performance ceilings that vertical scaling can no longer solve?
\nPrompt engineering is about how you ask. Context engineering is about what the model can see when you ask. Past a certain point, only the second one moves accuracy.
\n\n- Prompts hit an \"instruction ceiling\": no phrasing fixes a model that lacks the data, and crowded context windows lose details in the middle.\n- Context engineering builds the system around the model: retrieval (RAG), tools (MCP), memory and workflow state.\n- In practice that means a context layer in your backend that assembles the right documents, history and tools before the model sees the user's input.\n\nYou spent three hours tweaking a system prompt so your LLM would stop inventing details about customer orders. It worked for ten minutes.
\nThen a user asked about a return policy from 2024, and the model confidently described a 90-day cash-back guarantee that doesn't exist.
\nYou've hit the instruction ceiling. No amount of \"You are a helpful assistant\" or \"Think step by step\" lets a model reason its way out of missing data.
\nYou're treating a data problem as a language problem. As we move from simple chatbots to agentic systems, the focus shifts from how we talk to models to how we feed them.
\nPrompt engineering was the first discipline of the AI era. You craft instructions, examples and formatting rules to steer a Large Language Model (LLM), and you might spend days testing phrasing or adding few-shot examples.
\nThat works well for creative writing and basic text transformation.
\nPrompts are stateless and limited. You squeeze logic, data and constraints into a finite context window, and when it gets crowded the model loses track of details.
\nResearchers call this \"lost in the middle\": models use information at the start and end of a long context better than information buried in the middle. Relying only on prompts also means relying on training data, which is static and often outdated.
\nIn production, especially in Shopify development or custom enterprise software, static instructions are dangerous. If the model doesn't know real-time stock levels or the specs of a new product, clever phrasing won't prevent a made-up answer.
\nContext engineering is the programmatic management of everything around the model. It covers the text you send, and also the architecture that decides which data, tools and memories the model can reach at any moment.
\nIf prompt engineering is writing a script for an actor, context engineering is building the stage, supplying the props and hiring the research team that feeds facts into the actor's earpiece.
\nMoving from \"telling\" to \"showing\" lowers the load on the model. Instead of asking it to remember every policy, you build a system that finds the one relevant policy and puts it in front of the model.
\nRetrieval-augmented generation (RAG) is the most common form of context engineering. It closes the knowledge gap by connecting your LLM to a vector store: Postgres with the pgvector extension, or a dedicated database like Pinecone, Weaviate or Qdrant.
\nWhen a user asks a question, the system first converts it into an embedding (a numeric representation of meaning). It searches for the most similar documents, then injects those snippets into the prompt.
\n\nThat change cuts the chance of the model making things up. RAG has its own pitfalls, though, and it's worth knowing the common RAG mistakes in production.
\nThe Model Context Protocol (MCP) is one of the most useful pieces of context engineering. Anthropic introduced it in November 2024 and donated it in December 2025 to the Agentic AI Foundation, a Linux Foundation fund co-founded by Anthropic, Block and OpenAI.
\nSay you're building an agent that checks GitHub issues, looks up a customer in a CRM and writes a technical summary. Traditionally you'd write custom integration code for each source. With MCP, servers expose those sources in a standard format any MCP client can discover and call.
\nMCP lets the model pull context when it needs it, so it explores your data instead of passively receiving one big prompt. It turns context from a static block of text into a queryable interface, and it's a core part of a modern API gateway for AI stacks.
\n| Feature | \nPrompt engineering | \nContext engineering | \n
|---|---|---|
| Focus | \nPhrasing and tone | \nData and tools | \n
| Tooling | \nText editors, prompt playgrounds | \nVector DBs, MCP, RAG pipelines | \n
| State | \nMostly stateless | \nStateful and persistent | \n
| Scalability | \nHard to maintain as docs grow | \nBuilt for large document sets | \n
| Primary goal | \nBetter behavior | \nBetter accuracy | \n
A model can only be as right as the context you hand it.
\n\nTake your longest system prompt and highlight every fact in it (policies, prices, product details). Move each one into a retrieval source or a tool, and keep only behavior rules in the prompt.\n\nPrompting focuses on the \"now.\" Context engineering also handles the \"always\": long-term memory.
\nIn a Laravel application, you might keep user preferences or past interactions in a database or Redis. When the user returns, the context layer retrieves those fragments and summarizes them for the model.
\nContext is what separates a basic chatbot from an AI agent. A chatbot waits for a prompt and responds. An agent works inside a context: tools (via MCP), knowledge (via RAG) and history (via memory stores).
\nWith a context-first mindset, you spend less time in a prompt playground and more time building reliable data connectors. You care about the quality of your data more than the cleverness of your adjectives.
\nFor Laravel and Vue.js stacks, this means a context layer in your backend that gathers everything the AI needs before it sees the user's input.
\n// A simple context orchestrator in Laravel\nclass AIContextManager\n{\n public function buildContext(User $user, string $query): array\n {\n $knowledge = $this->vectorStore->search($query);\n $history = $this->memory->getRecent($user->id);\n $tools = $this->mcp->getAvailableTools(['inventory', 'shipping']);\n\n return [\n 'system_prompt' => view('prompts.system')->render(),\n 'retrieved_docs' => $knowledge,\n 'user_history' => $history,\n 'available_tools' => $tools,\n 'user_query' => $query,\n ];\n }\n}\nIf you're building that context layer for production, here's how I help teams design RAG, MCP and memory for their AI features.
\nHow are you balancing long system prompts against external retrieval in your current AI work?
\nAn ML engineer builds the model. An AI engineer builds the product around a model someone else trained. Hire the wrong one and nothing ships.
\n\n- ML engineers train and tune models: data pipelines, feature stores, experiments and metrics like F1 score and loss curves.\n- AI engineers orchestrate foundation models through APIs: RAG, agents, MCP, guardrails, latency and cost.\n- Hire an ML engineer when no existing model fits your problem, and an AI engineer when a smart assistant with your data will do.\n\nYou are ready to ship AI features, but your hiring roadmap is a mess of overlapping buzzwords and conflicting job descriptions.
\nHiring a researcher to build a production chatbot often means a month of experiments with zero shipped code. Asking a standard full-stack developer to optimize a vector search index usually ends in a high-latency disaster.
\nThe gap between training a model and orchestrating a system is wider than most founders realize.
\nThe choice between an ML engineer and an AI engineer decides whether you build a proprietary brain or a fast application powered by existing intelligence.
\nThe machine learning engineer lives in the world of weights, biases and data distributions. Their main goal is to create, train or fine-tune models that solve specific predictive or generative tasks.
\nThey build the logic that lives inside the API, instead of calling someone else's.
\nAn ML engineer might spend the day in PyTorch or TensorFlow designing a custom architecture. They handle the \"dirty work\" of data: cleaning large datasets, managing feature stores and dealing with data drift.
\nTheir mental model is rooted in applied statistics and experimental iteration. If you need a model that predicts fraudulent transactions from proprietary financial data, you need an ML engineer. They make sure the model generalizes well and doesn't overfit the training set.
\nTheir stack sits at the heavy end of the data world: Spark for data processing, Weights & Biases for experiment tracking and NVIDIA GPUs for heavy lifting.
\nThe output is a serialized model file or a dedicated inference service that other systems call. This is a deep-tech role focused on the \"how\" of intelligence, and the split between training and inference shapes most of their day.
\nThe AI engineer is a fairly new kind of developer. They treat the model as a black-box service.
\nTheir focus is the infrastructure around the LLM that makes it useful for users, rather than training it. They are the bridge between raw intelligence and a working product.
\nAn AI engineer spends their time on system design and software engineering. They work with foundation models like Claude, GPT and Gemini through APIs, and focus on RAG (retrieval-augmented generation) and agentic workflows.
\nInstead of tweaking hyperparameters, they tweak system prompts and design tool-use schemas.
\nThe AI engineer owns the user experience of AI. That includes managing latency, adding guardrails and making sure the output is structured correctly for the frontend.
\nIf you are building a custom support bot on Shopify, you want an AI engineer. They know how to connect the store's data to the LLM and handle multi-turn conversations.
\nOne role decides what the model knows. The other decides what users get from it.
\nTo see the differences, look at the tools used in 2026. There is overlap, but the main focus areas are distinct.
\n| Feature | \nML Engineer | \nAI Engineer | \n
|---|---|---|
| Core Models | \nCustom PyTorch/TensorFlow, XGBoost | \nClaude, GPT, Llama, Gemini APIs | \n
| Primary Task | \nTraining, Fine-tuning, Evaluation | \nPrompt Engineering, RAG, Agents | \n
| Data Focus | \nFeature engineering, labeling, drift | \nChunking, indexing, retrieval quality | \n
| Programming | \nPython (heavy), C++, CUDA | \nPython, TypeScript, PHP | \n
| Infrastructure | \nKubernetes, GPUs, Feature Stores | \nVector DBs (pgvector), MCP, Serverless | \n
| Success Metric | \nF1 Score, Accuracy, Loss curves | \nLatency, Hallucination rate, User NPS | \n
The ML engineer owns the pipeline from raw data to a deployed model. The AI engineer owns the pipeline from a user query to a relevant, cited response.
\nOne builds the engine; the other builds the car and picks the right fuel.
\nRetrieval-augmented generation (RAG) is the most common meeting point for these two roles. A solid RAG system needs high-quality embeddings and a fast, scalable vector database.
\nAn ML engineer might train a domain-specific embedding model or a custom reranker to improve search relevance. They look at the mathematical similarity between vectors and tune the distance functions.
\nThe AI engineer usually builds the end-to-end RAG system. They decide how to chunk documents, how to manage metadata and how to run hybrid search with tools like pgvector or Pinecone.
\nThey also handle context management: making sure the LLM gets exactly what it needs without exceeding its context window or blowing the budget. If you are just starting, avoid common RAG mistakes in production by focusing on retrieval quality before model size.
\n\nBefore you consider a bigger model or a custom embedding, build a set of 20 real questions with known answers and measure how often your retrieval returns the right chunk.\n\nIn 2026, the focus has shifted from simple chatbots to agentic systems: AI agents that use tools, browse the web and run code to finish complex tasks.
\nAI engineers are the main drivers of this shift. They use frameworks like LangGraph or the Claude Agent SDK to build multi-step workflows.
\nThey implement the Model Context Protocol (MCP) to give agents access to internal databases, local files and external APIs. That takes deep software engineering skills, because agents need reliable error handling and human-in-the-loop checkpoints to be useful in a business.
\nML engineers support this by building the specialized tools agents call. An agent might call a custom fraud-detection model built by an ML engineer as part of its reasoning loop.
\nThe AI engineer orchestrates the logic, while the ML engineer provides the specialized predictions.
\n\nThe deployment cycle looks very different for each role.
\nML engineers deal with MLOps. See DevOps vs MLOps for how that discipline differs from classic ops.
\nIt means specialized infrastructure for model training, managing GPU clusters and watching for model drift over time. They often use Docker and Kubernetes so their models can scale to millions of inference requests.
\nAI engineers focus on LLMOps, or \"AI DevOps.\" Their concerns are API reliability, cost management and caching.
\nThey need their AI apps to stay fast and to swap a model version (for example, moving a Claude or GPT model to a newer release) without breaking the system. Tools like Coolify or cloud setups on Google Cloud are common here for hosting the middleware that connects the LLM to the world.
\nThe decision depends on the \"intelligence\" you need.
\nIf your problem is unique and no pre-trained model can solve it, you need an ML engineer. That is true for niche medical imaging, high-frequency trading or specialized sensor data analysis.
\nIf a very smart assistant with access to your company data can solve it, you need an AI engineer. Most modern software falls into this group.
\nYou don't need to train a new model to build an AI-powered project management tool. You need a great system around a foundation model. This often comes down to a RAG vs fine-tuning decision, and the answer is usually RAG.
\nAI vs traditional development is becoming less about the code and more about data orchestration. The AI engineer is a full-stack developer who has learned to manage non-deterministic outputs.
\n\nWrite one sentence describing the AI feature on your roadmap. If it needs a model that doesn't exist yet, start with an ML engineer; if it needs an existing model wired to your data, start with an AI engineer.\n\n\n- **ML engineers** are model builders focused on training, data science and applied statistics.\n- **AI engineers** are system builders focused on app development, RAG, agents and model orchestration.\n- **The overlap** is Python and data basics, but the daily toolsets are moving apart.\n- **RAG** is the main point where both roles add to one production feature.\n- **Agents** are where AI engineering is heading, and they need strong backend logic and API integration skills.\n- **Choosing the right role** avoids wasted budget and gets your AI features into production.\n\nIf you need the AI engineering side (RAG, agents and MCP wired into your product), here's how I help teams ship production AI features.
\nWhen you look at your roadmap, is the main bottleneck the lack of a custom model, or making an existing model work reliably with your data?
\nA reverse proxy decides how a request is handled. A load balancer decides where it goes. Most production stacks need both.
\n\n- A reverse proxy shields your app: it terminates SSL, caches responses, routes by path and hides your backend servers.\n- A load balancer spreads traffic across a pool of identical servers with round robin, least connections or IP hash, and drops unhealthy nodes.\n- In production you layer them: a load balancer at the edge, Nginx on each node, and your Laravel app behind it.\n\nYour server is gasping for air. A traffic spike from a marketing campaign or a product launch on Shopify has pushed your single Laravel instance to its limit.
\nCPU is pinned at 99 percent, requests are timing out and the database is struggling with the number of open connections. You know you need to scale and put a buffer between the internet and your app.
\nWhen you look at architecture diagrams, you see two terms used almost interchangeably: load balancer and reverse proxy.
\nChoosing the wrong one, or missing how they work together, leads to \"Frankenstein\" architectures. You might add layers that add latency without adding value, or leave your app exposed to risks a proper proxy would have blocked.
\nTo build a solid Laravel application or a fast Shopify app, you need to know how these two components differ.
\nA reverse proxy is a server that sits in front of one or more web servers and intercepts client requests. It acts as a shield and a middleman.
\nUsers talk to the reverse proxy first. It decides how to handle the request before passing it to your backend Laravel or Node.js application.
\nIf you are fuzzy on which direction it faces, see forward proxy vs reverse proxy. They sit on opposite ends of the connection.
\nIn the world of Coolify and self-hosted SaaS, Nginx is the most common reverse proxy.
\nInstead of exposing your application server directly to the internet, the reverse proxy handles the \"dirty work\" of HTTP communication.
\n/api to one service and everything else to a frontend app.A load balancer's mission is high availability and horizontal scaling.
\nIf you have five identical Laravel servers running in a Docker cluster, the load balancer makes sure no single server gets overwhelmed while others sit idle.
\nLayer 4 load balancers work at the transport level (TCP/UDP). They decide based on IP addresses and ports without looking at the request content.
\nLayer 7 load balancers work at the application level (HTTP/HTTPS). They can route on headers, cookies or URL parameters.
\nLoad balancers use algorithms to decide which server gets the next request:
\nThe proxy cares about each request. The balancer cares about the whole pool.
\nThe confusion often comes from tools like Nginx, HAProxy and Traefik that can do both jobs. The difference in concept still matters for system design.
\n| Feature | \nReverse Proxy | \nLoad Balancer | \n
|---|---|---|
| Primary Goal | \nSecurity, routing, and efficiency | \nAvailability and throughput | \n
| Backend Pattern | \nUsually one logical service | \nA pool of identical nodes | \n
| Caching | \nExcellent support for static + dynamic | \nUsually minimal or none | \n
| Health Checks | \nBasic (is the backend up?) | \nAdvanced (latency, error rates, load) | \n
| OSI Layer | \nMostly Layer 7 (Application) | \nLayer 4 (Transport) or Layer 7 | \n
Both are reverse-proxy roles wearing different hats. A load balancer focuses on distribution (spreading traffic across a pool). A reverse proxy focuses on transformation (SSL, caching, routing, security).
\nThe same tool (Nginx, HAProxy, Traefik) can do both. In a full API gateway, these roles merge into a single entry point that manages both security and distribution across your microservices.
\nWhen building custom web solutions, you rarely pick only one. You layer them. A production Laravel app usually looks like a \"sandwich\" of these components.
\nIn a typical cloud setup on GCP or AWS, the request flow looks like this:
\nThis gives you redundancy. If one Nginx instance fails, the cloud load balancer sees it through a health check and stops sending traffic to that node, so users stay served.
\n\nShopify apps have their own needs. During big events like Black Friday, webhook traffic can spike hard and in bursts.
\nShopify gives you only five seconds to respond to each delivery before it counts as failed. It retries failed deliveries 8 times over the next 4 hours, and after 8 consecutive failures a subscription created through the Admin API is deleted. You cannot ride that out on a single server.
\nUse a load balancer to take in these webhooks and spread them across a fleet of worker nodes. Acknowledge fast and process the payload asynchronously.
\nA reverse proxy at the edge can add rate limiting. That stops a single store from hogging your app's resources and keeps your agentic commerce systems responsive for all merchants.
\nModern web development has moved away from the \"one server\" model. Even for small startups, a managed load balancer costs little compared to downtime.
\nA reverse proxy like Nginx or Traefik is standard practice because it keeps your application code simple. You don't want SSL handling logic inside your Laravel controllers.
\nCombining a load balancer and a reverse proxy lets you run blue-green deployments. You spin up a new version, test it, then switch the load balancer from the old version to the new one.
\nIf something breaks, you flip the switch back. You need both components working together to get this level of control.
\n\nAdd a health check endpoint (for example `/up`) to your Laravel app and point your load balancer's health check at it, so a broken node leaves the rotation on its own.\n\n\n- **Use a reverse proxy** if you have a single server but need SSL, caching and basic security.\n- **Add a load balancer** as soon as you need to scale across multiple servers or need high availability.\n- **Use Nginx or HAProxy** to fill both roles in one software layer for small and medium projects.\n- **Offload SSL termination** to the proxy layer to keep your Laravel or Node.js app responsive.\n- **Set up health checks** in your load balancer to remove unhealthy servers from the rotation automatically.\n- **Choose layer 7 balancing** if you need to route on HTTP headers or URL paths.\n\nIf you're putting a proxy and load balancer in front of a Laravel or Shopify app, here's how I help teams set up SSL, caching and scaling layers for production.
\nWhat is your current bottleneck: scaling the number of concurrent connections, or managing the complexity of your request routing?
\nYour dashboard shows a latency spike. It can't tell you why the database connection is timing out. That gap is the difference between monitoring and logging.
\n\n- Monitoring tracks numbers over time and tells you the system is unhealthy. Logging records events and tells you why.\n- Use structured JSON logs with a `request_id` so you can search and trace one request across your whole stack.\n- Link the two with correlation IDs, alert on user-focused SLOs, and turn frequent log patterns into metrics.\n\nYour production environment is failing and customers are reporting 500 errors. You check your dashboard and see a latency spike, but the graph doesn't say why.
\nYou dive into your text files for the stack trace, but the logs are unstructured and impossible to search during a crisis.
\nEngineering teams often blur these two pillars of observability. The result is slow incident response and fragile systems.
\nPicking tools is only part of it. You need a clear strategy for your DevOps solutions that keeps availability high and debugging fast. One is the pulse of your system. The other is the forensic record.
\nMonitoring tracks metrics over time to understand the state of a system. It answers \"Is the system healthy?\" by looking at numbers.
\nYou monitor CPU usage, memory and request latency. When a metric crosses a threshold, your monitoring system fires an alert.
\nLogging records discrete events inside an application or infrastructure. It answers \"What exactly happened?\" by giving context.
\nA log entry might hold a user ID, a timestamp and the error message from a failed Laravel job. Logs are granular and tell the story of a single request through your stack.
\n| Feature | \nMonitoring | \nLogging | \n
|---|---|---|
| Data type | \nMetrics (numbers/counters) | \nEvents (text/structured data) | \n
| Purpose | \nDetection and health | \nDiagnosis and forensics | \n
| Frequency | \nAggregated over time | \nRecorded per occurrence | \n
| Alerting | \nThreshold-based (e.g., CPU > 80%) | \nEvent-based (e.g., Fatal Error) | \n
| Storage | \nTime-series databases | \nLog aggregators/search engines | \n
Monitoring focuses on numbers. It is the first line of defense in any production environment.
\nBy watching trends, you can predict failures before they happen. A slow rise in memory use over 48 hours might point to a memory leak in a long-running Docker container.
\nIn a stack using tools like Coolify and Docker, monitoring shows whether your containers perform as expected. Focus on the \"Four Golden Signals\":
\nGood monitoring needs meaningful thresholds. An alert that fires every time CPU hits 70% is noise if your app is meant to be CPU-intensive.
\nDefine Service Level Objectives (SLOs) that match user experience. If users don't notice a 100ms delay, don't wake up an engineer for it.
\n\nPick one alert that fired last week without anyone needing to act. Rewrite it against a user-facing SLO (for example error rate or P95 latency) or delete it.\n\nIf monitoring tells you the plane is losing altitude, logging tells you why the engine stalled. Logs give you the evidence to debug complex issues.
\nIn a distributed system, one user action might touch several services. Without central logging, finding the root cause is like finding a needle in a haystack.
\nA common mistake is using unstructured logs. Text lines like [2026-06-21] Error: something went wrong are hard for machines to parse.
Modern teams use structured logging, formatting each log as a JSON object. You can then filter and query on fields like user_id, request_id or environment. When a log line won't parse, my JSON formatter points to the exact character.
{\n \"timestamp\": \"2026-06-21T14:30:05Z\",\n \"level\": \"error\",\n \"message\": \"Payment gateway timeout\",\n \"context\": {\n \"user_id\": 4502,\n \"order_id\": \"ORD-9921\",\n \"gateway\": \"stripe\",\n \"attempt\": 3\n },\n \"request_id\": \"req-a1b2c3d4\"\n}\nIncluding a request_id in every log entry lets you trace a single request across your infrastructure. This matters a lot in Shopify development, where webhooks and API calls often chain together.
A dashboard tells you something broke. A searchable log tells you what to fix.
\nLaravel has a capable logging system built on the Monolog library. Out of the box, it supports channels such as single files, daily rotated files, syslog and Slack, plus stack channels that send one message to several channels at once.
In production, configure Laravel to ship logs to a central service. Sending them to a service like Logstash, Sentry or Datadog means you don't lose data when a server instance is terminated.
\nMonitoring a Laravel app means more than checking that the web server is up. Watch the health of your background queues too.
\nIf your Redis queue backs up, customers might not get order confirmation emails or account activation links. Tools like Laravel Pulse or Horizon give real-time monitoring for these app-level metrics.
\nA common pattern is to log exceptions to a dedicated tracker while watching the error rate on a dashboard. A spike in the \"Error Rate\" metric prompts you to check the \"Exception Logs\" for the actual stack trace.
\n\nAdd a middleware that sets a `request_id` with `Log::withContext()` at the start of each request, so every log line in that request carries it.\n\nHigh-scale Shopify apps need their own approach to logging and monitoring. They live at the mercy of external API rate limits and webhook delivery speed.
\nIf your app handles thousands of webhooks per minute, you can't log every successful event to a text file. You will quickly run out of disk space.
\nInstead, use monitoring to track the \"Webhook Success Rate\" and your API rate-limit use. Shopify's GraphQL Admin API meters calls by calculated query cost using a leaky bucket (on the Standard plan, your quota restores at 100 points per second, and a single query can't cost more than 1,000 points). The REST Admin API uses a request-based bucket. To see how fast a full sync can run inside those limits, use my Shopify rate limit planner.
\nWatch how close you run to those ceilings, then use logging to investigate the specific failed payloads when the success rate drops.
\nThis matters even more for agentic commerce apps where AI agents act on behalf of a merchant. If an agent fails to update an inventory count, you need the specific log to understand the logic failure.
\nFor Shopify Plus merchants, performance matters a lot. Monitoring the \"Time to First Byte\" (TTFB) of your app's embedded components makes sure you aren't slowing down the merchant's admin.
\nMature engineering teams don't treat logging and monitoring as separate silos. They link them.
\nWhen an alert fires in your monitoring tool, it should link straight to the logs for that timeframe and service.
\nThis is often done with \"Correlation IDs.\" When a request enters your system, you give it a unique ID.
\nThat ID is attached to every log entry and trace written during the request. When a metric spikes, you filter the logs for that time window and follow the IDs to the requests that caused it.
\nYou should also monitor your logs. Counting how often certain log patterns appear gives you new metrics.
\nIf \"Database connection lost\" shows up more than five times in a minute, turn that log event into a metric that triggers a high-priority alert.
\ndebug for local development, info for general production events, and error or critical for things that need a human.If you're building observability for a Laravel or Shopify app, here's how I help teams set up structured logs, SLO alerts and correlation IDs.
\nHow do you tell a noisy alert from a real system failure in your current production stack?
\nTwo boxes sit at the front of almost every production system. One spreads traffic so nothing falls over. The other decides who gets in and where they go.
\n\n- A load balancer spreads traffic across identical backends at L4 or L7, and stays fast because it ignores who is calling.\n- An API gateway enforces policy at the edge: auth, rate limits, transformations, caching and routing by intent to the right service.\n- Mature stacks run both: a load balancer at the edge, a gateway behind it, and internal load balancers in front of each service.\n\nEngineers constantly confuse these two components. They look similar on an architecture diagram, but they solve very different problems.
\nPick the wrong one and you either bolt fragile business logic onto a component built to be dumb and fast, or you pay for an expensive policy engine to do a job simple round-robin could handle.
\nThe load balancer vs API gateway decision shapes how you scale, secure and observe everything behind it. Here is what each one does, where they differ and why mature systems almost always run both.
\nA load balancer has one job: take incoming traffic and spread it across a pool of identical backends so no single server gets overwhelmed.
\nIt is the component that lets you run three copies of your app instead of one and survive a traffic spike.
\nLoad balancers work at one of two layers. An L4 (transport layer) balancer routes by IP and port. It does not look inside the request and just forwards TCP/UDP packets, which makes it extremely fast.
\nAn L7 (application layer) balancer understands HTTP. It can route based on the path or host header and terminate SSL/TLS. Algorithms like round-robin, least-connections and IP-hash decide which backend gets the next request.
\nA load balancer is largely stateless and content-agnostic. It cares about availability and distribution, not about who you are or whether you are allowed to make this call.
\nThat ignorance is a feature: it keeps the component simple, fast and reliable. If you have wondered how this differs from a reverse proxy, the two overlap heavily. See load balancer vs reverse proxy for where they diverge.
\nAn API gateway is the opposite kind of component. It is opinionated, aware and full of business rules. It sits in front of your services as a single, managed front door for every API call.
\nA gateway handles the cross-cutting concerns you don't want duplicated in every microservice:
\nInstead of each service re-implementing auth, the gateway enforces it once at the edge.
\nA gateway routes by intent as well as availability. It maps /orders to the orders service and /users to the users service, applies the right policy to each and stitches a fleet of microservices into one coherent API.
That makes it a natural fit once you move from a monolith to microservices and need one stable front door over many services. It is also the base of a clean API gateway for an AI stack, where the gateway governs which model endpoints and tools a request can reach.
\nThe load balancer asks \"is a server free?\" The gateway asks \"should this request get in at all?\"
\nBoth sit at the front and both can route HTTP, but their purpose, intelligence and state are very different.
\n| Aspect | \nLoad Balancer | \nAPI Gateway | \n
|---|---|---|
| Primary job | \nDistribute traffic across servers | \nEnforce policy + route by intent | \n
| Layer | \nL4 (TCP/UDP) or L7 (HTTP) | \nL7 (HTTP/API aware) | \n
| Awareness | \nContent-agnostic | \nInspects auth, headers, payload | \n
| Core features | \nHealth checks, SSL, algorithms | \nAuth, rate limiting, caching, transforms | \n
| State | \nMostly stateless | \nTracks clients, quotas, sessions | \n
| Optimizes for | \nAvailability + throughput | \nSecurity + control | \n
The rule of thumb: reach for a load balancer when you need raw distribution and uptime, and an API gateway when you need to apply rules (who can call this, how often and in what shape).
\nIn production, it is common to see both components working together. Each layer does a different job: one spreads the traffic, the other decides what gets through.
\nThis separation of concerns lets you scale your infrastructure independently of your business rules.
\nYou can rewrite authentication logic in the gateway without touching load balancer configuration, and you can add or remove backend replicas without the gateway noticing.
\n\nWhen developing with Laravel, you often don't need a dedicated hardware appliance. You can build these patterns with open-source software like Nginx, Traefik or Kong.
\nUsing Docker containerization makes this setup very portable.
\nIf you use Traefik as an API gateway in a Docker Compose file, you can define your routing and middleware directly in the labels. That keeps your configuration close to your code.
\nservices:\n laravel-api:\n image: your-org/laravel-app:latest\n labels:\n - \"traefik.http.routers.api.rule=Host(`api.example.com`)\"\n - \"traefik.http.middlewares.api-auth.forwardauth.address=http://auth-service\"\n - \"traefik.http.middlewares.rate-limit.ratelimit.average=100\"\n - \"traefik.http.routers.api.middlewares=api-auth,rate-limit\"\nHere, Traefik plays the API gateway role: it checks with an external auth service and enforces rate limits before the request ever reaches the Laravel application.
\n\nAdd a `ratelimit` middleware to one Traefik router in staging, hit it with a quick load test and confirm you get HTTP 429 responses once you pass the limit.\n\nFor Shopify developers, the API gateway pattern is very useful for \"headless\" storefronts. You talk to both the Shopify Storefront API and your own custom backend, and a gateway can unify these separate sources.
\nIt can cache frequent Shopify queries to reduce API consumption and speed things up for the end user.
\nA gateway also lets you add logic Shopify doesn't support natively: transforming payloads or merging multiple sources into one response, which cuts the number of round trips the browser makes.
\nThat pattern is a foundation for building agentic workflows in modern commerce.
\nIf you're untangling the front door of your production stack, here's how I help teams design load balancer and API gateway layers.
\nHow do you handle cross-cutting concerns like rate limiting and authentication across your current microservice fleet?
\nYour database is at 90% CPU and users feel the lag. The real question is whether you buy a bigger server or add more of them.
\n\n- Vertical scaling adds CPU and RAM to one server: no code changes, but a hard ceiling and a single point of failure.\n- Horizontal scaling adds servers behind a load balancer: high availability and elasticity, but your app must be stateless.\n- Most teams end up hybrid, scaling the app layer out while keeping the database on one strong primary with read replicas.\n\nYour application is slowing down. Users report latency, your database is hitting 90% CPU usage and your background workers are lagging.
\nYou know you need more power. Choosing the wrong scaling path can lead to wasted budget, needless complexity or a hard ceiling on your growth.
\nAdding hardware is only half of the job. The other half is understanding the trade-offs between vertical and horizontal strategies.
\nIf you wait too long to scale out, you risk a single point of failure. If you scale out too early, you drown in the complexity of distributed systems.
\nVertical scaling, or \"scaling up,\" means adding more power to your existing server. This usually means more CPU, RAM or storage on a single instance.
\nIn a cloud like Google Cloud or AWS, this is as simple as changing your machine type from a small instance to a high-memory or high-compute one.
\nThe application still lives on one machine, so you don't need to change your code. There is no need for load balancers, service discovery or complex networking.
\nYour Laravel application or monolith works exactly as it did before, just faster.
\nEvery hardware generation has a limit. Eventually the top machine size available is either too expensive or simply not enough.
\nVertical scaling also does not solve redundancy. If that one big server goes down, your entire system goes down with it.
\n\nBefore you buy a bigger box, run your numbers through my [server capacity calculator](/tools/capacity-calculator/) to see how many workers your traffic really needs.\n\nHorizontal scaling, or \"scaling out,\" means adding more machines to your infrastructure. Instead of one giant server, you have a fleet of smaller servers working in parallel.
\nA load balancer sits in front of this fleet and spreads incoming traffic across the available nodes.
\nThis is the gold standard for high availability and modern cloud infrastructure. When one server fails, the load balancer routes traffic to the healthy ones.
\nIt is also very elastic. With auto-scaling, you add nodes during a traffic spike and remove them when things quiet down, so you only pay for what you use.
\nHorizontal scaling requires architectural changes. Your application must be stateless.
\nIf a user uploads a file to Server A, Server B must be able to access it. If you store session data in Server A's local memory, the user gets logged out when the next request hits Server B. Move state to shared databases or object storage.
\nA fleet only helps when any server can answer any request.
\nYour choice depends on your current stage and technical needs. Here is how the two compare on the engineering metrics that matter most.
\n| Feature | \nVertical scaling (scale up) | \nHorizontal scaling (scale out) | \n
|---|---|---|
| Primary action | \nAdd CPU/RAM to one node | \nAdd more nodes to the pool | \n
| Complexity | \nLow (no code changes) | \nHigh (distributed system) | \n
| Fault tolerance | \nLow (single point of failure) | \nHigh (redundancy by design) | \n
| Scalability limit | \nHard limit (hardware ceiling) | \nVirtually unlimited | \n
| Maintenance | \nRequires downtime for resizing | \nZero-downtime rolling updates | \n
| Cost | \nExpensive at the high end | \nMore efficient through auto-scaling | \n
When you move to a horizontal model, the database becomes the main bottleneck.
\nScaling the application layer is fairly easy because app servers are usually stateless. Scaling a database is harder because data must stay consistent across multiple nodes. Every new app server also opens its own database connections, and my Postgres connection pool calculator shows when you will hit max_connections.
For many Shopify apps or custom web platforms, the first step into horizontal scaling is adding read replicas.
\nYou keep one \"primary\" database for writes and multiple \"replica\" databases for reads. This takes the heavy read load off the main node.
\nIf your data grows beyond what a single primary node can handle, you might look into sharding. That means splitting your data across different database clusters.
\nIt works, but it adds a lot of complexity to how you query and join data.
\n\nCheck what share of your database queries are reads. If it's most of them, point those reads at a replica before you think about sharding.\n\nIn the Laravel ecosystem, scaling often starts with vertical upgrades. I have seen many projects stay on a single strong VPS for years.
\nAs traffic grows, the move to horizontal scaling usually follows a clear pattern:
\nfile driver to redis or database.Tools like Coolify or Docker Swarm make managing multiple containers easier. They let you define your infrastructure as code, so spinning up new nodes is simple.
\n\nThere is no one-size-fits-all answer. Your choice should balance cost, time and reliability.
\nChoose vertical scaling if:
\nChoose horizontal scaling if:
\nIn reality, most modern architectures mix both. You might scale your database vertically as far as you can while scaling your application layer horizontally.
\nThat gives you the reliability of a fleet with the simplicity of a single data source.
\n\n- Vertical scaling makes a single server stronger through CPU and RAM upgrades.\n- Horizontal scaling adds more servers to a pool and uses a load balancer to manage traffic.\n- Vertical scaling is simpler and needs no code changes, but it has a performance ceiling and no redundancy.\n- Horizontal scaling offers high availability and elasticity, but it requires a stateless application.\n- For most web apps, scaling out starts by externalizing sessions, files and background jobs.\n- Databases are the hardest part to scale horizontally, often needing replicas or sharding.\n\nIf you're planning a scaling path for your stack, here's how I help teams move from one server to a fleet without breaking sessions or data.
\nAt what point does the cost of managing a distributed horizontal system outweigh the cost of simply buying a larger server for your specific workload?
\nAn LLM gives you great answers and zero execution. An agent wraps it in tools, memory and a loop so it can actually do the work.
\n\n- An LLM is a stateless next-token predictor: no memory between calls, a knowledge cutoff and no access to outside systems.\n- An agent runs the LLM inside a loop (think, act with a tool, observe the result) backed by memory, tools, planning and reasoning.\n- Use a plain LLM for simple, linear, low-latency text tasks, and an agent when the work needs several steps or external systems.\n\nMost developers start their AI journey by sending a prompt to an API and waiting for a text response. That works for simple summaries or creative writing.
\nIt fails when the task needs real-world actions. A single chat completion cannot run a multi-step refund process in a Shopify store or navigate a complex database schema to build a report.
\nWhat holds you back is the architecture around the model.
\nRelying only on Large Language Models (LLMs) is like hiring a genius professor who has no hands, no memory of yesterday and no computer. You get great answers but nothing gets done.
\nAI agents wrap the LLM in tools, memory and planning logic. Moving from \"talking to a model\" to \"building an autonomous system\" is one of the main changes in how we architect software today.
\nA Large Language Model is a stateless next-token predictor. When you send a prompt, the model uses its training to calculate the most probable sequence of words to follow your input.
\nIt does not \"think\" in the human sense. It runs one forward pass per generated token through a very large neural network.
\nThe core traits of a standalone LLM:
\nFor simple applications, this is fine. If you are building a basic API gateway for AI, a thin wrapper around a model might be all you need.
\nAs soon as you need the system to take initiative, the LLM alone falls short.
\nAn AI agent is an autonomous system that uses an LLM as its central reasoning engine. Unlike a single LLM call, an agent works inside a loop.
\nIt observes the environment, thinks about what to do next, takes an action with a tool, and then observes the result to decide its next step.
\nThis loop lets the agent correct itself. If a database query fails, a standalone LLM would simply report the failure in its final response.
\nAn agent sees the error, works out what went wrong and tries a different query. Interleaving a reasoning step with an action and then observing the result is the ReAct (Reasoning + Acting) approach introduced by Yao et al. in 2022.
\nThe model answers. The loop around it is what gets things done.
\nTo turn an LLM into an agent, you add four layers of infrastructure.
\nLLMs have a \"context window,\" but it is temporary and expensive. Agents use external memory systems to keep state across sessions.
\nThat includes short-term memory for the current task steps and long-term memory for user preferences or historical data. We often use vector databases like pgvector to store and retrieve these memories. The difference between the context window and persistent memory is what separates a toy from a production agent.
\nTools are the hands of the agent: external functions, APIs or scripts the agent can choose to run.
\nWhen the agent sees it needs information it doesn't have, it generates a structured command (usually a JSON object that matches the tool's schema) to call a tool. This is what lets agents act on stacks like Laravel or a Shopify store.
\nComplex goals need to be broken into smaller tasks. An agent uses the LLM to create a roadmap.
\nFor example, a high-level request like \"Audit our cloud spend\" gets split into sub-tasks: \"List all AWS instances,\" \"Query pricing API\" and \"Generate CSV report.\"
\nReasoning governs how the agent uses the other three pillars. It is the control loop that keeps the system running until the goal is reached or a stopping condition is met.
\n\nTake one task you currently handle with a single prompt and write down its steps. If it has more than one step that touches an outside system, it is a candidate for an agent loop.\n\n| Feature | \nLarge Language Model (LLM) | \nAI agent | \n
|---|---|---|
| Execution | \nPassive (prompt-response) | \nActive (goal-oriented) | \n
| Persistence | \nNone (stateless) | \nPersistent (external memory) | \n
| Capabilities | \nText generation and analysis | \nTool use, API calls, web browsing | \n
| Reasoning | \nOne-shot internal logic | \nMulti-step iterative loop | \n
| Connectivity | \nIsolated | \nIntegrated with external systems | \n
| Architecture | \nModel-centric | \nSystem-centric | \n
Retrieval-Augmented Generation (RAG) was the first step toward more capable AI. In a standard RAG setup, you retrieve relevant documents from a vector store and stuff them into the LLM prompt. It is a linear process.
\nAgentic RAG goes further. Instead of a fixed retrieval step, the agent decides when to search, what keywords to use and whether the results were actually useful.
\nIf the first search results are poor, the agent refines its query and searches again. I cover this in more depth in traditional vs agentic vs corrective RAG.
\n\nTake the 7 common RAG mistakes. An agentic approach can reduce problems like \"hallucinated retrieval\" by cross-referencing multiple sources or checking facts against a structured database.
\nBuilding an agentic system takes more than an API key. You need a solid backend to manage state and run the tools.
\nFrameworks like LangGraph and CrewAI are popular with Python developers. In the PHP ecosystem, you can build capable agentic backends with Laravel and tools like Claude MCP.
\nThe Model Context Protocol (MCP) is especially useful for agent development. It gives agents a standard way to connect to local and remote data sources.
\nInstead of writing custom connectors for every tool, you can use MCP to give your agent direct access to your filesystem, GitHub repos or database schemas.
\n// Example of a tool definition in a Laravel-based agent\npublic function getTools(): array\n{\n return [\n [\n 'name' => 'query_shopify_orders',\n 'description' => 'Retrieves order details from Shopify using GraphQL.',\n 'parameters' => [\n 'type' => 'object',\n 'properties' => [\n 'order_id' => ['type' => 'string'],\n ],\n ],\n ],\n ];\n}\nClear tool definitions tell the agent exactly what it can do. The LLM then acts as the router, deciding which tool to trigger based on the user's intent.
\nNot every feature needs to be an agent. Agents are more complex to build, harder to test and can cost more because of multiple LLM calls.
\nUse a standalone LLM when:
\nUse an AI agent when:
\nIf you want help turning a prompt-response feature into an agent that takes action, here's how I work with teams shipping agents with tools, memory and MCP.
\nIs your current AI implementation stuck in a passive prompt-response loop, or are you ready to build systems that actually take action?
\nDevOps ships code that behaves the same every day. MLOps ships models that quietly get worse as the world changes.
\n\n- DevOps treats code as the main artifact and relies on deterministic builds, tests and deployments.\n- MLOps adds two moving parts, data and models, so it needs a third loop: Continuous Training (CT) on top of CI and CD.\n- MLOps monitoring tracks statistical health (data drift, concept drift, accuracy) as well as system health (latency, errors).\n\nShipping application code to production is difficult. Shipping a machine learning model that keeps performing accurately over time is much harder.
\nYou might have a perfectly automated pipeline that builds your Docker images and deploys them to a cluster in seconds.
\nIf your model starts making wrong predictions because consumer behavior changed, your automated deployment of \"correct\" code is shipping a broken product. That is the core friction between traditional software operations and machine learning operations.
\nDevOps focuses on the reliability and speed of software delivery. It treats code as the primary artifact.
\nMLOps takes the same principles and adds two volatile new variables: data and models. DevOps solves \"it works on my machine.\" MLOps solves \"it worked on last month's training data.\"
\nUnderstanding this divide is essential for any team moving beyond simple heuristic-based software and into AI engineering.
\nDevOps is built on the continuous integration and continuous delivery (CI/CD) pipeline. The goal is to shorten the development life cycle and ship changes continuously with high software quality.
\nIn a standard DevOps environment, like one running a Laravel application on Docker, the process is fairly deterministic.
\nIf the source code passes its unit tests, the integration tests succeed and the environment configuration is correct, the application should behave predictably in production.
\nThe artifacts are static. Once a binary or a container image is built, its internal logic does not change based on the data flowing through it.
\nTechnical teams use DevOps to manage:
\nMLOps, or Machine Learning Operations, is a set of practices for deploying and maintaining machine learning models in production reliably and efficiently.
\nCalling it \"DevOps for AI\" misses the point. It borrows the automation mindset, but the artifacts in MLOps are fundamentally different.
\nA machine learning system has three distinct components: code, data and the model. In DevOps, you mostly care about the code.
\nIn MLOps, a change in the data can break the system even if the code is untouched. That brings a level of non-determinism traditional CI/CD pipelines are not designed to handle.
\nIf you are building agentic commerce solutions on Shopify, your recommendation engine might fail without any bug in the Python script. The cause could be a shift in the distribution of customer purchases during a holiday sale.
\nThis is known as data drift, and it requires a lifecycle with continuous monitoring and retraining.
\nThe code didn't change. The world did, and the model never noticed.
\nHere are the main technical distinctions between the two methodologies.
\n| Feature | \nDevOps | \nMLOps | \n
|---|---|---|
| Primary artifact | \nSource code and binaries | \nCode, datasets, and model weights | \n
| Lifecycle | \nBuild, test, deploy | \nData prep, train, evaluate, deploy, monitor | \n
| Testing | \nUnit and integration tests | \nModel validation, accuracy metrics, bias checks | \n
| Deployment | \nStatic (service/binary) | \nDynamic (model endpoint or batch job) | \n
| Monitoring | \nLatency, CPU, error rates | \nData drift, model decay, precision, recall | \n
| Feedback loop | \nCode bug fixes | \nContinuous retraining (CT) | \n
The main architectural difference is a third loop. DevOps relies on CI (Continuous Integration) and CD (Continuous Delivery). MLOps adds CT (Continuous Training).
\nCI here tests the data as well as the code. You must validate the schema of incoming datasets and make sure feature engineering logic is the same in the training environment and in production.
\nA mismatch leads to training-serving skew, a common reason for model failure.
\nCD covers the automated deployment of the model service. This often means deploying a prediction API or updating a weights file in a running inference engine.
\nBecause models are large and expensive to run, MLOps pipelines often use canary deployments or shadow testing to check performance on live data before a full rollout.
\nThis is the signature of MLOps. A CT pipeline triggers a retraining job when certain conditions are met, either on a schedule or when performance degrades.
\nThe pipeline pulls new data, runs the training script, evaluates the new model version against a benchmark and registers it in a model registry if it passes.
\n\nIn traditional DevOps, monitoring centers on system health. You watch memory usage, response times and HTTP 500 errors. If the server is up and the code runs, the system is usually considered \"healthy.\"
\nIn MLOps, system health is only half the story. You also need to monitor statistical health.
\nA model might return a \"200 OK\" status code with 50ms latency, but if its predictions are nonsensical, the system is failing.
\nThere are two main types of drift that MLOps engineers must track:
\nDetecting drift requires careful logging of every prediction and its eventual outcome. That creates a feedback loop that informs the next training cycle, a step often missed in early-stage AI development.
\n\nStart logging every prediction with its input features and a request ID, then join the real outcome back to it later. Without that table, you can't measure drift at all.\n\nThe toolsets for these two fields are moving apart. Docker and Kubernetes are shared, but the special needs of machine learning have created a new stack.
\nBuilding a bridge between these tools is the mark of a mature technical organization. You need the stability of DevOps to host the infrastructure and the flexibility of MLOps to manage the intelligence layer.
\nFor teams running complex web applications, the goal is integration, not picking one side.
\nA modern stack might use a Laravel backend for user management, hosted on a Coolify-managed VPS, while calling an MLOps-managed inference endpoint for personalized content.
\nOne practical way to start is versioning your datasets. You would never deploy code without a commit hash, so never train a model without knowing exactly which version of the data was used.
\nThis makes results reproducible, which is the first step toward a working MLOps workflow.
\nThe same rigor applies to retrieval systems. Treat vector databases and retrieval pipelines as managed infrastructure inside your DevOps workflow, not as ad-hoc scripts.
\nThat discipline alone heads off many of the RAG failures teams hit in production.
\n\nTag your next training run with the dataset version or hash it used, and store that tag in the model registry entry. If you can't reproduce a model from its tag, fix that before adding more tooling.\n\n\n- **DevOps is about code reliability:** CI/CD pipelines, automated testing and deterministic deployments.\n- **MLOps is about model reliability:** data validation, experiment tracking and statistical monitoring.\n- **The CT loop is essential:** continuous training keeps models from going stale as real-world data changes.\n- **Monitoring must cover two layers:** system metrics (latency, errors) and model metrics (drift, accuracy).\n- **Tooling is specialized:** standard DevOps tools for hosting, MLOps tools for training and the registry.\n- **Data is an artifact:** version your training data with the same rigor you apply to source code.\n\nIf you're adding models to a product that already ships with CI/CD, here's how I help teams build MLOps pipelines with drift monitoring and retraining.
\nHow do you plan to handle the statistical monitoring of your models once they move from a static environment into the unpredictable flow of production data?
\nA context window is a desk. Memory is a library. Stop trying to fit the whole library on the desk.
\n\n- The context window is the model's short-term working memory: big, temporary and paid for on every request.\n- Long-term memory lives outside the model, usually in a vector database such as pgvector, and RAG pulls only the relevant chunks into the window.\n- Production agents use tiers: recent messages in context, episodic and semantic memory through RAG, and tools (often via MCP) to fetch the rest on demand.\n\nLarge language models have a memory problem that costs you money and performance. You spend thousands of tokens feeding the same documentation, user history and context into every prompt.
\nYet the moment the session ends, the model forgets everything. It is like hiring a genius consultant who gets total amnesia the instant they walk out the door.
\nThis cycle of \"context stuffing\" doesn't hold up in production. Relying only on a massive context window leads to high latency and the \"lost in the middle\" effect, where models ignore data buried in the center of a large prompt.
\nTo build agentic systems that actually remember, you must separate the short-lived context window from persistent long-term memory.
\nA solid AI architecture mixes RAG, vector databases and careful context management. This guide breaks down the technical differences between context and memory, and how to build a hybrid that scales.
\nThe context window is the model's immediate working memory. It is the total number of tokens (words or parts of words) the model can process at one moment.
\nWhen you send a prompt to a frontier model like Claude or GPT, the window holds your current instruction, the previous conversation and any files you attached. To see how much of a window your own prompt uses, paste it into my context window checker.
\nThink of the context window as a physical desk. You can only fit so many papers on it at once. Add a new stack of documents and you have to push the old ones off.
\nOnce a token falls out of this window, the model loses all awareness of it. It does not matter if the information was vital. It is gone.
\nModern models offer very large context windows: current Claude and Gemini models both go up to one million tokens. That is impressive, but it is still temporary, and it is not a replacement for a database.
\nUsing a large window for everything is an expensive way to handle data that should be stored permanently. Bigger is not automatically better either: accuracy and recall drop as the window fills, a problem the field now calls \"context rot.\"
\n\nPaste your largest production prompt into the [context window checker](/tools/context-window-checker/) and mark which parts are the same on every request. Those repeated parts are your first candidates for memory instead of context.\n\nMany developers mistake a long context window for long-term memory. It is a dangerous technical shortcut.
\nIf you are building for Shopify Plus, for example, you might be tempted to dump your entire product catalog into the prompt. That creates three immediate problems:
\nTrue long-term memory is persistent. It lives outside the model.
\nIt lets an AI agent remember a user's preference from three months ago without including that preference in every API call. This is where Retrieval-Augmented Generation (RAG) becomes the hero of your architecture.
\nA bigger desk doesn't help if you never build the library.
\nRetrieval-Augmented Generation (RAG) is the bridge between a stateless LLM and a persistent knowledge base.
\nInstead of stuffing everything into the context window, you store your data in a vector database. When a query comes in, you run a semantic search to find only the most relevant \"chunks\" of information.
\nThose chunks are then injected into the context window. The prompt stays lean and the model stays focused. RAG turns the \"desk\" (context window) into a tool for reasoning over a \"library\" (vector database).
\nFor teams in the Laravel ecosystem, RAG has become much easier with tools like pgvector. You can store embeddings directly in your PostgreSQL database, so your relational data and your AI logic live side by side.
\n| Feature | \nContext Window | \nLong-Term Memory (RAG) | \n
|---|---|---|
| Duration | \nSession-based (Temporary) | \nPersistent (Permanent) | \n
| Capacity | \nLimited (up to ~1M tokens) | \nVirtually Unlimited | \n
| Cost | \nHigh per-request cost | \nLow per-request (fixed storage) | \n
| Update Speed | \nInstant for the session | \nRequires indexing/embedding | \n
Choosing between a larger context window and a RAG pipeline is an economic decision.
\nFor a small internal tool with ten users, context stuffing is fine. The engineering hours needed to build a RAG pipeline would exceed the token savings.
\nFor a SaaS application or a high-volume e-commerce store, the math shifts quickly. Sending 100,000 tokens per request at scale will destroy your margins. A well-optimized RAG system might send only 2,000 tokens per request.
\nRun both numbers through my LLM cost calculator to see the monthly gap.
\n\nA vector database like Pinecone, or a self-hosted pgvector instance, often costs a fraction of the wasted tokens. The trade-offs between managed and self-hosted stores are worth weighing up front: see picking the right RAG stack.
\nEither way, you avoid the common RAG mistakes in production by focusing on retrieval quality rather than prompt size.
\n\nPut your current tokens per request and your daily request count into the [LLM cost calculator](/tools/llm-cost-calculator/), then repeat with 2,000 tokens per request. The difference is your monthly budget for building RAG.\n\nIf you run a Laravel application, you do not need a separate, complex vector database for basic memory. PostgreSQL with the pgvector extension is often the best choice for mid-sized applications. It lets you run similarity searches alongside your standard Eloquent queries.
\nImagine a user searching for \"shoes for rainy weather.\" A traditional database looks for the keyword \"rainy.\" With embeddings and pgvector, the system understands the semantic link between \"rainy\" and \"waterproof.\"
\n// Example of a semantic search in Laravel using pgvector\n// Pseudocode: inject your embedder; there is no Laravel AI:: facade.\n$queryEmbedding = $embedder->embed(\"shoes for rainy weather\");\n\n$products = Product::query()\n ->selectRaw('name, description, embedding <=> ? as distance', [$queryEmbedding])\n ->orderBy('distance')\n ->limit(5)\n ->get();\nBy storing these embeddings, you create a persistent memory of your product catalog that is always ready for the LLM. You only pull the relevant products into the context window when they are needed.
\nThe Model Context Protocol (MCP), created by Anthropic, changes how we handle context. It lets models connect directly to external data sources like Google Drive, Slack or your local filesystem.
\nInstead of you managing what goes into the context window, the model uses Claude MCP servers to fetch what it needs on demand.
\nThis blurs the line between context and memory. The model \"remembers\" where to find information and retrieves it when needed.
\nThis is the foundation for agentic systems. An agent that can query its own database or read its own logs via MCP doesn't need a massive context window.
\nIt needs a high \"reasoning capacity\" to know which piece of memory to grab at the right time.
\nAdvanced AI systems do not choose one over the other. They use a tiered memory strategy:
\nThis layered approach gives the model what it needs to solve a problem without the bloat of an oversized prompt.
\n\nSort everything in your current prompt into the four tiers above. Anything that isn't short-term context belongs in a store you retrieve from, not in every request.\n\n\n- **Stop context stuffing.** If your prompt is regularly over 50k tokens, you are likely wasting money and reducing accuracy.\n- **Use RAG for persistence.** Move static or long-term data into a vector database like pgvector or Pinecone.\n- **Optimize for latency.** Smaller prompts mean faster responses and a better user experience.\n- **Use MCP.** The Model Context Protocol gives your agents a standard way to \"reach out\" to their memory stores.\n- **Monitor token counts.** Track how many tokens actually help the final output and how many are noise.\n\nAs models get smarter, the \"size\" of the desk matters less than the \"organization\" of the library. If you're designing memory for a production agent, here's how I help teams build RAG memory that stays fast and affordable.
\nAre you building a bigger desk, or a better library?
\nYour AI is only as good as the data under it. The data engineer builds the factory; the data scientist refines the product.
\n\n- Data engineers work upstream: pipelines, warehouses, data models and reliable infrastructure that moves data from source to storage.\n- Data scientists work downstream: exploration, models, experiments and turning clean data into business answers.\n- AI and RAG projects need both, but the engineering foundation usually comes first, because you can't do science on broken pipes.\n\nYour business is drowning in data, but your AI models are still hallucinating or returning irrelevant results.
\nYou have invested in the latest large language models and vector databases, yet the output stays inconsistent because the underlying data is a fragmented mess of raw logs and poorly formatted JSON.
\nWithout a solid foundation, even the most advanced neural network is a sophisticated guessing machine. The fix starts with understanding the distinct but complementary roles of the data engineer and the data scientist.
\nThese two roles often share an office and use the same languages, like Python and SQL, but their core missions are different.
\nWhen you deploy agentic systems or RAG architectures, the split between infrastructure and inference decides whether a project scales or stays stuck in prototyping.
\nTo see the difference, picture the construction of a smart city.
\nThe data engineer is the civil architect and utility provider. They design the power grids, the water filtration systems and the high-speed transit tunnels.
\nTheir goal is reliability, throughput and structural integrity. If the water stops flowing or the lights flicker, the city stops working.
\nThe data scientist is the urban planner and behavioral psychologist. They study traffic patterns, energy use and population growth to decide where to build the next park or how to improve the public transport schedule.
\nThey do not build the pipes. They use what flows through the pipes to make the city smarter and more efficient.
\nThe data engineer manages the flow of information from source to storage. The data scientist then pulls that information to train models, run experiments and generate business insights.
\nOne role keeps the data flowing. The other figures out what it means.
\nThe data engineer designs, builds and maintains the systems that collect, store and move data. They work at the \"upstream\" end of the lifecycle, and their main focus is the \"three Vs\": Volume, Velocity and Variety.
\nData engineers spend a lot of time on system architecture and DevOps principles.
\nThey are the ones who add circuit breakers for vector databases and make sure your RAG pipeline doesn't fail because of a sudden surge in API requests.
\n\nPick one report your team relies on and trace where its data comes from. If you can't name every source and the job that moves it, that pipeline needs a data engineer.\n\nOnce the data is clean, formatted and accessible, the data scientist takes over. They work \"downstream,\" using curated datasets to solve specific business problems or build predictive features.
\nData scientists ask questions like \"Why is our churn rate increasing?\" or \"What is the best price point for this new subscription service?\"
\nThey rely heavily on the data engineer for high-quality data. If the data is corrupted, the model will be biased or inaccurate.
\nThe tools and skill sets overlap, but how they use them differs. Here is how the roles compare in a production environment.
\n| Feature | \nData Engineer | \nData Scientist | \n
|---|---|---|
| Primary Goal | \nBuild and maintain data systems | \nAnalyze and model data for insights | \n
| Common Languages | \nSQL, Python, Java, Scala | \nPython, R, SQL | \n
| Key Tools | \nSpark, Kafka, Airflow, dbt, Docker | \nscikit-learn, PyTorch, Jupyter, Tableau | \n
| Main Output | \nCleaned datasets, APIs, pipelines | \nPredictive models, insights, reports | \n
| Mindset | \nEngineering and reliability | \nStatistics and experimentation | \n
| Cloud Focus | \nInfrastructure (GCP, AWS, Terraform) | \nManaged ML services (SageMaker, Vertex AI) | \n
A data scientist might use SQL to pull a specific cohort of users for analysis. A data engineer uses SQL to optimize a transformation job that processes millions of rows.
\nWith LLMs and generative AI, the lines between these roles are blurring, but you still need both specialists. A similar boundary is being redrawn between the ML engineer and the AI engineer. Consider a Retrieval-Augmented Generation (RAG) system.
\nThe data engineer builds the ingestion pipeline that scrapes documentation, cleans the text, generates embeddings and stores them in a vector database like pgvector.
\nThey make sure the API gateway is secure and that the index stays fresh, whether through scheduled batch jobs or near-real-time streaming.
\nThe data scientist then evaluates how well the RAG system performs. They experiment with different embedding models, adjust the \"top-k\" retrieval parameters and fine-tune the prompts so the AI gives the most accurate and helpful responses.
\n\nWithout the engineer, the scientist has no data to query. Without the scientist, nobody knows if the answers are any good.
\nBoth are essential for moving beyond basic chatbots and into agentic commerce solutions.
\nIf you are a startup or an established business modernizing your digital presence, you might wonder which hire to make first.
\nOften, the best approach is to build the engineering foundation first. It is impossible to do science on a pile of broken pipes.
\n\n- **Data engineers** focus on the \"how\" of data movement and storage. They are software engineers specialized in distributed systems and infrastructure.\n- **Data scientists** focus on the \"what\" and \"why\" of the data. They are statisticians and analysts specialized in extracting meaning and building models.\n- **Collaboration** is key for AI success: a good RAG or agentic system needs solid engineering to feed solid science.\n- **Tooling** overlaps in Python and SQL, but engineers lean toward orchestration (Airflow, dbt) while scientists lean toward modeling (scikit-learn, PyTorch).\n- **Foundation first.** You can't do meaningful data science without a reliable data engineering infrastructure.\n\nIf your AI project is stuck because the data underneath it is a mess, here's how I help teams build the data pipeline and RAG foundation first.
\nHow are you currently handling the gap between your raw data infrastructure and your analytical insights?
\nA cloud engineer builds the ground your app stands on. A DevOps engineer builds the conveyor belt that ships code onto it. Hire the wrong one first and you pay for it.
\n\n- Cloud engineers design and run the infrastructure: networks, IAM, databases, storage and cloud cost (FinOps).\n- DevOps engineers automate delivery: CI/CD pipelines, deployments, observability and fast recovery.\n- The two roles overlap in infrastructure as code and often merge into platform engineering. Hire cloud first for a new foundation, DevOps first when releases are the bottleneck.\n\nThe cloud engineer vs DevOps engineer question trips up almost every team that outgrows a single server. The titles overlap, and job posts blur them together.
\nHiring the wrong specialty has a real cost: infrastructure that is over-engineered and expensive, or fast to ship but prone to outages. The two roles solve different problems. This guide breaks down where they diverge, where they overlap and which one to hire first.
\nA cloud engineer acts as the architect of your digital workspace. Their main job is to design, build and manage the cloud infrastructure where your applications live.
\nThis role focuses on the \"where\" and the \"what\" of your stack. They spend their time in the large catalogs of providers like AWS, Google Cloud Platform (GCP) or Azure.
\nCloud engineers specialize in the environment itself. That includes designing Virtual Private Clouds (VPCs), configuring subnets and managing identity and access management (IAM) policies.
\nThey make sure your database clusters are highly available and your storage is cost-effective.
\nBy 2026, cloud engineering has grown to include serious FinOps practices. Cloud engineers monitor resource consumption to prevent surprise bills from unoptimized cloud usage.
\nWhen you need to move a legacy system to a modern cloud-native architecture, the cloud engineer leads the way. They translate business requirements into technical blueprints.
\nFor instance, they might design an API gateway architecture to manage traffic for a distributed AI application. Their focus is the resilience, security and scalability of the infrastructure layer.
\n\nOpen last month's cloud bill and find the three most expensive line items. If nobody on the team can explain why each one costs what it does, you have a cloud engineering gap.\n\nWhile the cloud engineer builds the foundation, the DevOps engineer builds the machinery that moves code onto it.
\nDevOps is a methodology focused on the software delivery lifecycle. A DevOps engineer is the bridge between development and operations teams, and their goal is to maximize the speed and reliability of releases through automation.
\nThe core of a DevOps engineer's toolkit is the CI/CD pipeline (continuous integration and continuous delivery). They use tools like GitHub Actions, GitLab CI or Jenkins to automate building, testing and deploying code.
\nWhen a developer pushes a change, the DevOps engineer's systems scan it for vulnerabilities and deploy it to a staging environment without manual steps.
\nReliability is another major pillar. DevOps engineers set up observability stacks with Prometheus and Grafana, and alerting that catches issues before they affect end users.
\nIn modern setups, they often use tools like Docker and Coolify for hosting to simplify deployment workflows. They care deeply about the \"how\" of software delivery.
\nTo see which role fits your current needs, compare their daily priorities. There is a lot of overlap, but each role has a different center of gravity.
\nBoth rely heavily on infrastructure as code (IaC) with tools like Terraform or Pulumi, the same tooling I weigh up in Terraform vs Ansible.
\n| Feature | \nCloud Engineer | \nDevOps Engineer | \n
|---|---|---|
| Primary Goal | \nInfrastructure design and management | \nAutomation and software delivery | \n
| Key Focus | \nNetworking, Security, Storage, Cost | \nCI/CD, Deployment, Monitoring, Culture | \n
| Cloud Expertise | \nDeep (multi-service architecture) | \nFunctional (deployment targets) | \n
| Coding Focus | \nInfrastructure scripts (Terraform, HCL) | \nGlue code and automation (Bash, Python) | \n
| KPIs | \nUptime, Cost efficiency, Security | \nDeployment frequency, Mean Time to Recovery | \n
| Tools | \nAWS/GCP/Azure, VPC, IAM, RDS | \nGitHub Actions, Docker, Kubernetes, Ansible | \n
Cloud engineers are often the ones who decide whether to use a managed database service or a self-hosted one. DevOps engineers make sure the migration to that database happens smoothly with zero downtime.
\nOne role decides where the system lives. The other decides how code gets there.
\nThe line between cloud and DevOps keeps getting blurrier. In high-growth startups and mature enterprises, the roles often merge into platform engineering.
\nThe platform team builds an \"Internal Developer Platform\" (IDP). It lets developers self-serve their own infrastructure needs within pre-defined guardrails.
\nPlatform engineering combines the architecture knowledge of a cloud engineer with the automation mindset of a DevOps engineer. They use Kubernetes (EKS, GKE or AKS) as the base layer.
\nBy hiding the complexity of the cloud, they help product teams move faster. If this sounds close to site reliability, it is. I unpack the boundaries in SRE vs platform engineer.
\nFor companies running Laravel applications, this might mean a standard Docker-based environment, so the code runs the same way on a local machine as it does in a production cluster on GCP.
\nBrowse the DevOps category to see how these workflows are structured in practice.
\nWhich role to hire first depends on your current technical debt and business stage.
\n\nIf you are starting from scratch or moving away from a single-server setup, you need a cloud engineer. They will make your architecture secure and scalable from day one. Without this foundation, your automation will sit on top of a fragile environment.
\nIf you already have a working cloud setup but your developers are burning hours on manual deployments and firefighting, you need a DevOps engineer.
\nThey will untangle your release process and add the guardrails you need for rapid iteration. They focus on developer experience and velocity.
\nLarge e-commerce businesses on Shopify Plus often have hybrid needs. They might need custom middleware hosted on cloud infrastructure, plus a strict CI/CD process for their custom Shopify apps.
\nIn these cases, a specialist who understands both the infrastructure and the automation pipeline is a strong first hire.
\n\nWrite one sentence for each role describing the problem it would solve on your team this quarter. Hire for the sentence that names a problem costing you money or outages today.\n\n\n- **Cloud engineers** specialize in infrastructure architecture, cloud-native services, security and cost optimization.\n- **DevOps engineers** focus on the software delivery pipeline, automation, CI/CD and system reliability.\n- **Tools overlap** heavily: both roles need Terraform, Docker and at least one major cloud provider.\n- **Platform engineering** combines both roles to give developers self-service infrastructure.\n- **Hire a cloud engineer** when you need to build or modernize the \"hardware\" of your digital business.\n- **Hire a DevOps engineer** when you need to speed up your release cycle and automate manual operational tasks.\n\nIf you're deciding between a cloud hire and a DevOps hire, or need both skill sets for a while, here's how I help teams balance infrastructure and delivery.
\nHow does your current team balance the need for architectural stability with the demand for rapid feature delivery?
\nA cache stops your server from doing the same work twice. A CDN stops your data from crossing the world twice. A high-traffic site needs both.
\n\n- Caching is a technique: store the result of expensive work (queries, rendered fragments, assets) so you don't repeat it.\n- A CDN is a global network of edge servers that applies caching close to the user, which cuts the distance and latency of each request.\n- Layer them (browser, CDN edge, server cache, database) and plan purging from day one, or users will see stale data.\n\nHigh-traffic applications often collapse under the weight of redundant data requests.
\nEvery time a user in London requests a file from a server in New York, the latency hurts the user experience and drains your server resources.
\nIf you do not optimize how data is stored and delivered, your infrastructure costs will climb while your performance plateaus.
\nMany developers use \"CDN\" and \"cache\" interchangeably, but they serve different masters in the stack.
\nThe CDN-vs-cache decision is rarely either/or. Getting a global CDN and local caching to work together is what keeps a high-traffic site fast under load.
\nLatency is the silent killer of conversion rates. A standard request involves multiple round-trips between the client and the origin server.
\nIf the data has to cross the Atlantic, physical distance imposes a speed limit that no amount of code optimization can fix.
\nThis is a latency problem, not a bandwidth one. A wider pipe won't fix the lag when the bottleneck is round-trip distance.
\nCaching keeps data closer to the execution point. A CDN keeps that cache closer to the user.
\nWhen the two work together, they form a multi-layered shield that protects your origin server from unnecessary load.
\nCode can't beat the speed of light. Only a closer server can.
\nCaching is the temporary storage of data for faster retrieval. It is not limited to one location. In a modern stack, caching happens at several layers so the most expensive operations are never repeated.
\nThe most common layers:
\nThe goal of caching is to reduce the workload on your primary infrastructure. By storing the result of a complex calculation or a heavy query, you trade a small amount of memory for a large gain in response time.
\n\nTurn on slow query logging for a day and list the five queries that run most often with the same parameters. Those are your first server-cache candidates.\n\nA CDN is a specialized cache that lives at the \"edge\" of the network.
\nA standard cache might live on your main server. A CDN is a globally distributed network of servers that hosts your content in dozens of locations at once.
\nWhen a user visits your site, the CDN intercepts the request, acting as a reverse proxy in front of your origin.
\nIf the requested asset is already stored at the edge server closest to the user, it is served immediately, without the trip back to your origin. This matters most for static assets like images, video files and large JavaScript bundles.
\nFor businesses on Shopify, the platform handles much of this CDN logic automatically. Its CDN (backed by Cloudflare) ships on every plan, not only Plus.
\nEven so, understanding how to manage cache headers and purge cycles remains a key skill for custom development.
\nTo choose the right tool, you need to know where each one excels.
\n| Feature | \nLocal/Server Cache | \nContent Delivery Network (CDN) | \n
|---|---|---|
| Primary Goal | \nReduce CPU and database load. | \nReduce network latency and distance. | \n
| Location | \nOn or near the origin server. | \nAt the network edge (Points of Presence). | \n
| Scope | \nOften specific to a single server or user. | \nShared across many users globally. | \n
| Ownership | \nPart of your application infrastructure. | \nTypically a third-party service provider. | \n
| Best For | \nDB queries, session data, fragments. | \nImages, JS, CSS, static HTML, video. | \n
Caching is about how data is stored. A CDN is about where that data is distributed.
\nHow you implement this depends on your stack.
\nIn a Laravel environment, you manage caching through a unified API that supports multiple drivers.
\n// Example of caching a database query in Laravel\n$users = Cache::remember('active_users', 3600, function () {\n return DB::table('users')->where('active', true)->get();\n});\nThis queries the database only once per hour for this data.
\nTo go further, configure your web server to set Cache-Control headers that a CDN like Cloudflare or Akamai can interpret. My HTTP headers cheat sheet lists the caching headers with real example values.
In Shopify development, you rely heavily on the platform's native CDN. Shopify uses a large edge network to serve product images and storefront assets.
\nWhen you build custom apps or headless storefronts, though, you must manage how your API responses are cached yourself to avoid performance bottlenecks.
\nThe most resilient architectures combine a \"cache-aside\" pattern with an edge delivery strategy. This creates a multi-step defense for your origin server:
\nThis layered approach is essential for modern DevOps. Even during a massive traffic spike, like a Black Friday sale or a viral product launch, your core application stays responsive. For the application tier itself, see how Laravel Octane handles high traffic.
\n\nRunning two caching systems brings the challenge of cache invalidation.
\nIf you update a product price in your database but the CDN still serves a cached version of the page, you will face customer service issues.
\nWhen data changes on your origin, you need a way to clear both your internal cache and the matching edge cache on your CDN.
\nMany developers use webhooks or event listeners to automate this. Users then see accurate data without you giving up speed.
\n\nChange one product price in staging and time how long it takes to show up on the live page through the CDN. If the answer is \"until the TTL expires,\" wire a purge call into the save event.\n\n\n- **Caching** is the technique of storing data temporarily to avoid repeating expensive operations.\n- **A CDN** is a network of servers that use caching to deliver content from a location physically close to the user.\n- **Distance matters.** A CDN reduces the travel time of data, while a local cache reduces the processing time.\n- **Use both.** Cache database results on the server and serve static assets from a CDN.\n- **Invalidation is key.** Make sure you can purge outdated data from all layers together to keep data accurate.\n\nIf your site slows down every time traffic spikes, here's how I help teams layer CDN and server caching for high traffic.
\nHow do you handle cache invalidation across multiple edge locations when your application data changes in real time?
\nCI proves your code works. CD gets that working code to users without a human copying files at midnight.
\n\n- Continuous Integration builds and tests every push, so integration bugs show up in minutes instead of at release time.\n- Continuous Delivery keeps every passing build ready to ship behind a manual button. Continuous Deployment removes the button.\n- For Laravel, that means a clean-room test pipeline. For Shopify, it means theme checks and deploying to an unpublished theme before you swap it live.\n\nShipping a broken build to production is an expensive way to learn your local environment isn't a mirror of reality.
\nFor developers on Laravel backends or Shopify storefronts, the gap between \"it works on my machine\" and a stable release is often filled with manual steps that invite human error.
\nManual FTP uploads, SSHing into servers to run migrations, or publishing Shopify themes by hand are risks modern engineering cannot afford.
\nContinuous Integration (CI) and Continuous Delivery (CD) remove this uncertainty. By automating the path from a commit to a live environment, teams ship faster while making their applications more stable.
\nThis guide breaks down the core differences between CI and CD and how to apply them to modern stacks.
\nContinuous Integration is the practice of merging every developer's working copy into a shared mainline several times a day.
\nIn a Laravel or Vue.js environment, every push to a branch makes an automated system build the application and run a suite of tests.
\nThe goal of CI is to catch \"integration hell\" before it happens. Instead of finding out at a weekly release that two developers changed the same core service, the pipeline flags the conflict within minutes.
\nA standard CI pipeline for a Laravel application includes:
\nBy the time a pull request is ready for review, CI has already given a \"green light\" that the code is syntactically correct and doesn't break any behavior your tests cover.
\n\nOpen your last five merged pull requests and check which of the four steps above actually ran on each. Any missing step is the next job to add to your pipeline.\n\nCI focuses on the quality of the code. CD focuses on the delivery of that code. Two flavors of CD often get confused.
\nEvery build that passes the CI pipeline is ready to deploy to production, but the actual release needs a manual trigger.
\nThis is common in regulated industries, or for high-stakes storefronts where a marketing manager wants to time a release.
\nThis removes the manual trigger. Every change that passes the pipeline goes to production automatically.
\nIt demands deep confidence in your test suite and the observability to catch regressions fast. Without it, a bad commit reaches users before anyone notices.
\n\nFor developers using Coolify for self-hosted SaaS, CD is the engine behind rapid iteration. Once CI passes, Coolify can pull the latest Docker image and update the running containers with zero downtime.
\nThe main difference is scope and goal. CI is about the developer's experience and code integrity. CD is about the user's experience and the release process.
\n| Feature | \nContinuous Integration (CI) | \nContinuous Delivery (CD) | \n
|---|---|---|
| Primary goal | \nDetect bugs and integration issues early. | \nEnsure code is always in a releasable state. | \n
| Trigger | \nTriggered by every code commit or push. | \nTriggered after CI passes successfully. | \n
| Key output | \nA validated code base and test reports. | \nA deployable artifact (Docker image, ZIP, etc.). | \n
| Manual step | \nFully automated process. | \nManual approval (Delivery) or fully automated (Deployment). | \n
In 2026, the lines are blurring as tools become more integrated. Most teams treat CI and CD as one pipeline where code flows from a developer's IDE into staging or production.
\nCI asks \"is this code good?\" CD asks \"can users have it now?\"
\nLaravel gives you a solid base for CI because of its built-in testing tools. When you set up a pipeline, aim for a \"clean room\" environment: pin the same PHP and MySQL versions you run in production.
\nA typical GitHub Actions workflow for Laravel might look like this:
\nname: Laravel CI\n\non: [push, pull_request]\n\njobs:\n laravel-tests:\n runs-on: ubuntu-latest\n services:\n mysql:\n image: mysql:8.4\n env:\n MYSQL_DATABASE: testing\n MYSQL_ALLOW_EMPTY_PASSWORD: \"yes\"\n ports:\n - 3306:3306\n options: >-\n --health-cmd=\"mysqladmin ping\"\n --health-interval=10s --health-timeout=5s --health-retries=5\n\n steps:\n - uses: actions/checkout@v7\n - name: Setup PHP\n uses: shivammathur/setup-php@v2\n with:\n php-version: \"8.5\"\n - name: Install Dependencies\n run: composer install --prefer-dist --no-interaction\n - name: Execute Tests\n run: vendor/bin/phpunit\nWith branch protection requiring this check, no code merges into main unless it passes every test case. That safety net lets the team move fast without fear of breaking core application logic.
\n\nTurn on branch protection for `main` and mark your CI job as a required status check. Until you do, a red build is only a suggestion.\n\nShopify development has traditionally meant editing themes in the browser. With the rise of agentic commerce on Shopify, the need for real CI/CD is much higher.
\nFor Shopify, CD is about managing \"theme versions\" rather than server binaries. With the Shopify CLI, you can automate theme deployments to different environments:
\nshopify theme check in your CI pipeline to catch Liquid errors before they go live.The cutover is instant, so customers never see a half-deployed storefront. It is a blue-green style of deployment for e-commerce.
\nA broken layout or a missing \"Add to Cart\" button caused by a Liquid syntax error never reaches the live store.
\n\nAdd `shopify theme check` as a CI step on your theme repo, then do your next release by pushing to an unpublished theme and publishing it from the CLI instead of the theme editor.\n\nCI/CD pays off most when you move past simple tests and add more advanced checks.
\nModern pipelines in 2026 often include visual regression testing. The system takes screenshots of the UI and compares them to the previous version to catch unintended layout shifts.
\nFor teams building API gateways for the AI stack, CI/CD is non-negotiable.
\nWhen your application relies on LLMs or external APIs, the pipeline must include integration tests that confirm those connections still work and behave as expected. If agents write part of your code, see the gate I run on AI-authored PRs.
\n\n- **CI is for quality.** Run tests, lint code and check for security vulnerabilities on every push.\n- **CD is for delivery.** Automate packaging and deployment to staging or production.\n- **Speed is a feature.** A CI pipeline that takes 30 minutes is a pipeline developers will try to bypass.\n- **Immutable artifacts.** Build once (e.g., a Docker image) and promote that same artifact through staging and production.\n- **Zero downtime.** Use blue-green deployments or symlink swaps so users aren't interrupted during a release.\n- **Shift left.** Move security and quality checks as early as possible to cut the cost of fixing bugs.\n\nIf your deployment process still involves a checklist a human has to follow, you aren't doing CD. Automation is the only way to scale a software business without scaling production outages.
\nIf you want your Laravel or Shopify releases to go from commit to production without the manual checklist, here's how I help teams automate delivery.
\nHow much time does your team spend manually verifying releases before they go live?
\nAPIs connect software to software. MCP connects AI agents to software. Mix them up and you spend weeks writing glue code that breaks on the next schema change.
\n\n- Traditional APIs are rigid contracts built for developers who read docs. MCP is a standard, self-describing layer built for models that discover tools at runtime.\n- MCP turns the M x N integration problem (every AI client times every tool) into M + N.\n- MCP does not replace your APIs. It usually wraps them, so your business logic and security stay in your core services.\n\nYour AI agents are trapped in a silo of brittle, custom-coded API integrations that break the moment you update a schema.
\nYou spend weeks writing boilerplate glue code just to give a Large Language Model (LLM) access to a single database or a CRM.
\nThis manual mapping of endpoints to prompts is the hidden tax of modern AI engineering. Every new tool needs a fresh round of documentation reading and prompt tuning.
\nAPIs have powered the internet for decades, but they were never designed for the non-deterministic, reasoning-heavy nature of AI agents.
\nWe are moving from hard-coded connections to dynamic capability discovery. This is where the Model Context Protocol (MCP) comes in.
\nTraditional APIs (Application Programming Interfaces) are the foundation of modern software. Whether it is a RESTful endpoint, a GraphQL query or a gRPC stream, the core idea is the same.
\nIt is a defined contract for software-to-software communication. You know exactly what input to send, and you get a predictable output back.
\nIn AI development, APIs are like the power grid. They provide the raw energy and data your application needs.
\nIf you are building a Shopify application, you use the Shopify Admin API to fetch orders or update inventory. The logic is rigid, the schemas are fixed, and the developer must know exactly which endpoint to call and when.
\nThe problem starts when you want an LLM to use these tools. You have to write \"tool definitions\" or \"function schemas\" that describe the API to the model.
\nYou are translating the API documentation into a format the model can understand. This works for one or two tools. It does not scale to an entire ecosystem of internal data sources and external services.
\nThe Model Context Protocol (MCP) is an open standard introduced by Anthropic to solve the \"M x N\" integration problem. Anthropic donated it to the Agentic AI Foundation, a Linux Foundation fund, in December 2025.
\nIn a traditional setup, if you have M AI clients (like Claude, ChatGPT and an internal agent) and N tools (like GitHub, Slack and your internal DB), you build M x N individual integrations.
\nMCP changes this to an M + N problem. With a standard \"universal plug,\" any MCP-compatible AI client can connect to any MCP-compliant server. Think of it as the USB-C of the AI world.
\nInstead of writing custom code to explain how to search your database, you host an MCP server that describes its own capabilities. When the agent connects, it \"discovers\" what the server can do without you hard-coding the instructions.
\nMCP uses JSON-RPC 2.0. A server can offer tools, resources (like files or data) and prompt templates to the model.
\nSince the 2026-07-28 spec, the protocol core is stateless request and response. The connection handshake and session IDs are gone, and each request carries its own protocol version and client capabilities.
If your app needs state across calls, the server hands the model an explicit handle to pass back. I cover what that means in practice in stateless MCP servers on Laravel.
\n\nCount the integrations you maintain today: AI clients times tools. If the number is above five or six, sketch which tools would become one MCP server each.\n\nThe core difference between API and MCP lies in who consumes the interface and how they find what they need.
\n| Feature | \nTraditional API | \nModel Context Protocol (MCP) | \n
|---|---|---|
| Primary Consumer | \nHuman Developers | \nAI Agents & LLMs | \n
| Discovery | \nManual (Reading Docs) | \nAutomatic (Self-describing) | \n
| Interface | \nHeterogeneous (REST/GraphQL) | \nStandardized (JSON-RPC) | \n
| State | \nMostly Stateless | \nStateless core since 2026-07-28; explicit handles for app state | \n
| Flexibility | \nRigid / Pre-scripted | \nDynamic / Reasoning-based | \n
APIs are built for humans to read and implement. You look at a Swagger UI, understand the authentication and write a fetch request.
\nMCP is built for models to navigate. An MCP server lists its tools and resources along with natural language descriptions. The model reasons about which tool to call based on the user's intent.
\nIf you are comparing AI vs traditional development, this is a prime example of the shift. Traditional development builds the pipes. AI engineering with MCP builds the connectors that let the model choose its own pipes.
\nDevelopers read API docs. Models read MCP descriptions.
\nAgentic systems need a level of autonomy that traditional APIs struggle to provide efficiently.
\nAn agent asked to \"research a company and draft a proposal\" might need a search tool, a CRM, a file system and a PDF generator.
\nWith standard APIs, you feed the descriptions of all those endpoints into the model's context window. This eats tokens and raises the risk of the model getting confused.
\nWith MCP, the client asks the server what it offers (and can cache that list), and the agent calls tools on demand. You can surface only the tools relevant to the task instead of pre-loading every schema.
\nThe result is less wasted context and fewer irrelevant choices for the model to wade through.
\nMCP also defines a \"resource\" primitive. An API endpoint returns data on request. An MCP resource can be subscribed to, so the server sends a notification when the underlying file or record changes.
\nFor lightweight retrieval-augmented generation (RAG) cases, an MCP server can expose your local files or database to the model in a standard way. That spares you a custom ingestion layer for every experiment.
\nIt does not replace a vector store for large-scale semantic search. Get that wrong and you hit the usual production RAG mistakes.
\nMCP does not replace APIs. In fact, MCP usually wraps existing APIs. Your business logic, security constraints and data persistence still live in your core services.
\nSay you have a Laravel backend with REST endpoints for managing customer orders.
\nTo make them accessible to an AI agent, you build a small MCP server (perhaps in Node.js or Python) that sits between the agent and your Laravel API.
\nThe MCP server describes the get_order_history tool in a way an LLM understands.
When the LLM calls that tool through MCP, the server makes the actual REST call to your Laravel backend, receives the JSON and passes the relevant context back to the model.
\nThis separation lets your core engineering team focus on solid, well-tested APIs while your AI team builds the MCP wrappers that enable agentic behavior.
\n\nWhen should you stick to direct API calls, and when should you implement MCP? It depends on the complexity of your integrations and the role of the LLM.
\nOne boundary worth drawing early: MCP covers the agent-to-tool axis only. The moment two agents need to delegate work to each other, you are in a different protocol's territory.
\nSee MCP vs A2A vs ACP for how the two layers stack.
\n\n- APIs are the \"what\" (the data and logic), while MCP is the \"how\" (how the AI interacts with that data).\n- MCP removes the M x N integration problem with a universal protocol for AI tool discovery.\n- Traditional APIs suit deterministic software-to-software paths; MCP suits non-deterministic AI-to-tool reasoning.\n- Implementing MCP often means wrapping existing REST or GraphQL APIs to make them self-describing for LLMs.\n- MCP cuts token waste and prompt complexity by standardizing how context is shared with the model.\n- For modern AI engineering, MCP is becoming the standard layer for building agentic workflows that scale.\n\nIf you're wrapping your existing APIs in MCP so agents can use them safely, here's how I help teams ship it.
\nWhat is the biggest friction point you face when giving your AI agents access to your internal production databases?
\nMost SaaS founders overpay their cloud provider without noticing. The \"cloud tax\" eats your margins before you even find product-market fit.
\n\n- Managed cloud starts cheap with credits, then bills you for idle databases, NAT gateways and complexity you don't need yet.\n- Coolify gives you a self-hosted, git-push PaaS on your own server, with automatic SSL, one-click databases and PR previews.\n- ARM servers make it cheaper still: a small app can run on a Hetzner ARM box for about $13.50 a month, or on Oracle's Always Free tier.\n\nWe have been conditioned to believe that unless our small CRUD app runs on a multi-region, auto-scaling AWS EKS cluster, we are doing it wrong.
\nThat is a lie designed to keep you paying for complexity you do not need.
\nIt starts innocently enough. You sign up for AWS or GCP because they give you $1,000 in credits.
\nYou spin up an RDS instance for your database, a few S3 buckets for storage, and maybe a managed Kubernetes service because it feels \"professional.\"
\nThen the credits run out.
\nSuddenly you are paying $200 a month for a database that is 99% idle. You are paying for NAT gateways, provisioned IOPS and \"management fees\" for services that could easily run on a cheap VPS.
\nYou are stuck in a web of proprietary APIs and IAM roles that need a full-time DevOps engineer just to update an environment variable.
\nManaged services feel like a superpower at the start, but they become a golden cage as you scale. The complexity overhead alone can slow your development velocity to a crawl.
\nWhen I look at how AI is changing traditional development, it is clear we need to move faster, not get bogged down in infrastructure molasses.
\n\nSort your cloud bill by service and flag anything you pay for that runs under 10% utilization most of the month. That list is what a single self-hosted server can absorb.\n\nThe tool I have moved my entire stack to is Coolify.
\nCoolify is an open-source, self-hostable alternative to Vercel, Heroku and Netlify. It gives you the same \"git push to deploy\" experience, but it runs on your own hardware.
\nWhether you have a small ARM VPS on Hetzner (the CAX11 starts around €6/month) or a free ARM instance on Oracle Cloud, Coolify turns it into a private PaaS.
\nI recently wrote about how Coolify and Docker are changing SaaS hosting, but the shift goes deeper than a tool change. It is a move toward technical sovereignty.
\nOnce you have self-hosting in place, ARM is the multiplier that takes the bill from \"cheap\" to \"almost free.\" Stop renting x86 and start renting ARM.
\nAWS says its Graviton instances cost up to 20% less than comparable x86 instances.
\nThe real \"cheat code\" right now is Oracle Cloud Infrastructure (OCI). Its \"Always Free\" tier gives you 2 ARM Ampere A1 OCPUs and 12 GB of RAM at no cost, forever.
\n(Oracle halved this from 4 OCPUs / 24 GB in mid-2026, so size accordingly.) I can run a small production workload (frontend, backend, database and Redis) on that single free instance using Coolify.
\nWhen you pair ARM efficiency with a self-hosted orchestrator, the math changes. A startup paying $500/month on AWS can often move that entire workload to a sub-$15/month ARM instance on Hetzner, or to OCI's free tier outright.
\nThat difference goes straight back into your runway or your marketing budget.
\nRent ARM, run Coolify, and keep the difference as runway.
\nCoolify relies heavily on Docker, which is the industry standard for a reason. It makes sure that what works on my machine works on the server.
\nAs I move deeper into the \"vibe coding\" era, I'm also looking at how technologies like Nix can make our environments even more stable.
\nUsing Nix flakes to define the development environment and Docker to package the runtime gives you a repeatable deployment pipeline.
\nWhen I use tools like the Model Context Protocol (MCP), I want my AI agents to have a clear, reproducible environment to work in. Self-hosting doesn't mean \"unprofessional.\" It means total control over the stack.
\nLet's look at the numbers for a standard Laravel or Node.js app with a database and a background worker.
\n| Line item | \nManaged path (Vercel + Supabase + AWS S3) | \nCoolify path (Hetzner VPS) | \n
|---|---|---|
| Hosting | \nVercel Pro: $20/month per developer seat | \n4 vCPU ARM / 8GB RAM (CAX21): ~$12.50/month | \n
| Database | \nSupabase Pro: $25/month | \nIncluded (container on the VPS) | \n
| Storage / backups | \nAWS S3 + bandwidth: $15/month | \nBackups to S3-compatible storage: $1/month | \n
| Platform | \nIncluded | \nCoolify: $0 (open source) | \n
| Total | \n$60+/month (and rising with every user/teammate) | \n~$13.50/month | \n
The \"managed\" path costs roughly 4x more before you even have your first 100 users, and the gap widens with every teammate and every GB of egress. To run the numbers for your own stack, including the hours you will spend on upkeep, try my self-hosting cost calculator.
\nThe half hour it takes to install Coolify on a fresh Linux box is worth the thousands of dollars saved over the life of the project.
\n\nRebuild this table with your own invoices: list each managed service you pay for and the line it would become on one Coolify server. If the right column is under a third of the left, you have your business case.\n\nIf you are tired of the cloud tax, here is my recommended path to freedom:
\nDockerfile. For Vite or Next.js, Coolify has built-in builders that don't even need a Dockerfile.We are entering a cycle where efficiency matters more than ever. The days of \"VC-subsidized\" infrastructure are over. Whether you are building a small tool or a large enterprise SaaS, you owe it to your bottom line to look at self-hosting.
\nCoolify has matured to the point where the developer experience is close to the big players. The real difference is who owns the keys to the castle.
\nWhen you do outgrow a single box, the platform scales with you. I cover that in advanced Coolify deployment strategies.
\n\n- Cloud credits hide the real cost: idle databases, NAT gateways and complexity you don't need yet.\n- Coolify gives you git-push deploys, automatic SSL, one-click databases and PR previews on your own server.\n- ARM instances on Hetzner or Oracle's Always Free tier cut the bill further.\n- For a typical small app, the managed path costs roughly 4x more, and the gap grows with your team.\n- Start with one throwaway project, then move your real app and its database.\n\nIf your cloud bill is growing faster than your users, here's how I help teams move to self-hosted Coolify on ARM, or drop a note via contact.
\nWhat is the most \"expensive\" mistake you have ever made on a cloud bill: a forgotten NAT gateway or a runaway Lambda function?
\nMachine learning predicts. Generative AI creates. Pick the wrong one and you burn GPU credits on a model that hallucinates when it should be calculating.
\n\n- Traditional ML is discriminative: it learns `P(y | x)` from structured data to classify or predict, cheaply and precisely.\n- Generative AI learns how data is distributed so it can create text, code or images, at a much higher compute cost and with hallucination risk.\n- Strong production systems are hybrid: ML for scoring and routing, GenAI for the language layer users see.\n\nYour data may be sitting idle in a warehouse while competitors ship agentic systems that write code and predict churn in the same stack.
\nMarketing hype blurs the line between these two technologies, and stakeholders use the terms interchangeably. That mismatch breeds technical debt.
\nBuild a generative model for a task that demands strict mathematical precision and you set yourself up for failure. Rely on traditional models for creative automation and your product feels rigid and outdated.
\nThe fix is understanding the engineering differences between predictive and creative systems. If you want to zoom out first, my breakdown of AI vs machine learning shows where both sit in the broader AI hierarchy.
\nTraditional Machine Learning (ML) is the backbone of modern data science. It is mainly a discriminative technology.
\nIts job is to tell different types of data apart or to predict a number from historical patterns. You ask it to categorize an input or forecast a trend.
\nML models thrive on structured data. They look at rows and columns in a database to find correlations.
\nFor instance, a regression model might analyze thousands of Shopify transactions to predict next month's revenue. A classification model might look at server logs to spot a potential DDoS attack.
\nTraditional ML is often task-specific. If you want to detect fraud, you train a fraud detection model, and that model cannot suddenly start recommending products.
\nIt is a precision tool built for a single objective. That focus makes ML very efficient in high-stakes settings where accuracy and predictability are what matter.
\nTraditional ML analyzes data to make a choice; GenAI uses data to create something new.
\nIt is built on deep learning architectures, mainly Transformers and Diffusion models. Instead of only classifying data, these systems learn the probability distribution of their training set so they can generate new outputs.
\nWhen you prompt a frontier Large Language Model (LLM) like Claude or GPT, you are using a generative system. It predicts the next most likely token in a sequence, and the result is coherent text or a code snippet.
\nThat creative ability is what makes GenAI so useful for building agentic commerce solutions on Shopify.
\nGenAI models are often foundation models, trained on huge unstructured datasets with billions of parameters. They handle a wide range of tasks without being retrained from scratch.
\nOne model can summarize a legal document, write a Python script and brainstorm marketing copy. That flexibility is its main strength, and it also brings the risk of hallucinations.
\nOne kind of model draws the line between answers. The other writes answers that never existed before.
\nThe core technical difference lies in their mathematical objectives. To know which one to deploy, look at how each processes information.
\nDiscriminative models learn the boundary between classes. Mathematically, they model the conditional probability P(y | x): \"given the input x, what is the probability of the label y?\"
Generative models learn how the data itself is distributed. They model the joint probability P(x, y) or the probability of the input itself P(x): \"what does a typical example of this data look like?\"
| Feature | \nMachine Learning (Discriminative) | \nGenerative AI (Generative) | \n
|---|---|---|
| Primary Goal | \nClassify or Predict | \nCreate New Content | \n
| Data Type | \nStructured (Tables, Logs) | \nUnstructured (Text, Images) | \n
| Output Type | \nNumbers, Labels, Scores | \nContent, Code, Media | \n
| Model Complexity | \nLow to Medium | \nVery High | \n
| Training Data | \nTask-Specific, Labeled | \nMassive, Unlabeled/Self-Supervised | \n
The right tool depends on your business goals and your data. Using an LLM to predict a stock price is an expensive mistake. Using a linear regression model to write a blog post is impossible.
\nML is the better fit for tasks that need high precision and deterministic logic. If you are working through complex technical challenges in a real product, ML is often the right choice for the \"back-office\" logic.
\nGenAI shines when you need to bridge human language and machine logic. It is a strong fit for intuitive interfaces and creative automation.
\nIn 2026, strong engineering teams don't choose one over the other. They build hybrid architectures, using traditional ML for the heavy data processing and GenAI for the user-facing interaction layer.
\nConsider a modern customer support system. An ML model runs sentiment analysis and routes the ticket to the right department based on urgency.
\nOnce routed, a GenAI agent drafts a response using internal knowledge bases. The system ends up both efficient and empathetic.
\nAnother example comes from AI vs traditional development. An ML model predicts which users are likely to churn.
\nA GenAI model then creates a custom discount offer and a personalized email tailored to that user's interests to keep them engaged.
\n\nThe two need different infrastructure strategies.
\nTraditional ML models are often small enough to run on standard CPUs or even edge devices. They are easy to containerize with Docker and cheap to host on platforms like Coolify or AWS.
\nGenAI models are a different beast. Even \"small\" LLMs need significant VRAM, and the cost profile of training vs inference shapes every hardware decision.
\nIf you self-host, you need solid GPU orchestration. Many businesses use API-based options like Claude or OpenAI to avoid managing hardware. For teams concerned with data privacy or high volume, deploying open-weights models on Google Cloud or AWS with specialized inference servers is the standard approach.
\nWhichever you choose, keep the architecture clean. Containerize for environment consistency and monitor the right failure mode: model drift in ML, hallucination rates in GenAI.
\nThat rigor keeps your AI solutions scalable and maintainable over time.
\n\n- **Define your goal first:** use ML for prediction and classification, GenAI for content creation and reasoning.\n- **Data type matters:** ML excels with structured, tabular data. GenAI is built for unstructured data like text, images and code.\n- **Cost and latency:** traditional ML is usually cheaper and faster to run. GenAI needs much more compute and has higher latency.\n- **Hybrid works best:** combine the precision of ML with the flexibility of GenAI.\n- **Plan infrastructure early:** use containers and cloud-native services for each technology's hardware needs.\n- **Avoid the hype:** don't use an LLM because it is trending. Pick the model that solves the engineering problem most efficiently.\n\nIf you're deciding where ML ends and GenAI begins in your product, here's how I help teams design and ship hybrid AI features.
\nIs your current data strategy focused on predicting what will happen next, or are you ready to start creating the outcomes you want to see?
\nStandard PHP rebuilds your whole Laravel app for every request. Octane builds it once and keeps it warm.
\n\n- PHP-FPM boots the framework on every request, so a complex app can spend 30ms to 50ms on setup before your code even runs.\n- Octane keeps the app in memory on FrankenPHP, Swoole, Open Swoole or RoadRunner, and feeds requests through a worker pool you size for CPU-bound or I/O-bound work.\n- Long-lived workers make leaks dangerous, so avoid request data in statics and singletons, and use `--max-requests` to recycle workers.\n\nStandard PHP is a bit like a restaurant that fires its entire staff and rebuilds the kitchen for every customer order. You walk in, they hire a chef, buy a stove, cook your meal and then demolish the building as soon as you leave.
\nIt's consistent and safe, but it wastes a lot of energy when you're trying to serve thousands of people at once.
\nEvery request boots the entire Laravel framework, loads your service providers, parses your config and instantiates your objects. For low-traffic sites, that's fine.
\nAt real scale, say a busy multi-tenant SaaS, those milliseconds of \"boot time\" become a wall you can't climb without throwing expensive hardware at it. Horizontal scaling buys you room, but it doesn't fix the underlying latency.
\nLaravel Octane boots your application once, keeps it in memory and feeds requests to it through a fast worker pool. PHP goes from \"short-lived script\" to \"long-lived process.\"
\nThe overhead of traditional PHP costs you efficiency as well as speed. In a standard request, your CPU spends a big chunk of time just getting the application ready to work.
\nOnce it finally starts, it runs the database query, renders the view and then dies.
\nIf you run a complex Laravel monolith with dozens of packages and custom service providers, your boot time might be 30ms to 50ms before a single line of business logic runs.
\nUnder high traffic, this leads to CPU thrashing. You're paying for the \"setup\" over and over again.
\nOctane removes this boot cycle. With application servers like Swoole or RoadRunner, your app stays resident in memory.
\nThe first request boots the framework, and later requests hit a \"warm\" application. You can move from 50ms responses to sub-10ms responses just by changing how the process is managed.
\n\nOctane runs on top of an application server, and it supports four: FrankenPHP, Swoole, Open Swoole and RoadRunner.
\nFrankenPHP (a Go-based server built on Caddy) is the first option in the Octane docs, and it's the easiest path to HTTP/2, HTTP/3 and automatic HTTPS. In production, the two I reach for most are Swoole and RoadRunner, and they solve the \"persistent state\" problem differently.
\nSwoole is a C extension for PHP. It's a fast networking engine that lets PHP handle asynchronous tasks, coroutines and long-lived connections.
\nRoadRunner is written in Go. It acts as a load balancer and process manager that talks to your PHP workers over a fast binary protocol (Goridge).
\n| Server | \nInstall effort | \nPick it when | \n
|---|---|---|
| FrankenPHP | \nSmoothest setup | \nYou're starting out and want HTTP/3 for free | \n
| RoadRunner | \nSingle Go binary | \nYou want no PHP extension to manage | \n
| Swoole | \nC extension, fiddly | \nYou need every millisecond, task workers or Octane cache | \n
For most developers starting out, I'd pick FrankenPHP at the install prompt and not overthink it. Reach for Swoole only when you're chasing every last millisecond or need its concurrency features like async task workers, ticks and the Octane cache.
\nThe secret sauce of Octane is the \"worker pool.\" Instead of one process per request, you have a fixed number of workers waiting for incoming traffic.
\nIf you misconfigure it, you'll either leave performance on the table or crash your server. My server capacity calculator turns requests per second and latency into the number of workers you need.
\nThe right pool size depends on whether your app is CPU-bound or I/O-bound:
\n# Starting Octane with 16 workers for an I/O-heavy app\nphp artisan octane:start --server=swoole --workers=16\nDon't forget task workers if you're using Swoole. Sized separately with the --task-workers flag, they run in their own processes.
They're a good fit for fanning out concurrent queries without blocking the main request cycle.
\n\nCheck whether your app is CPU-bound or I/O-bound under load, then start Octane with workers equal to your cores (CPU-bound) or 2x your cores (I/O-bound) and compare P95 latency.\n\nThe main hurdle when moving to Octane is the shift in mindset. In traditional PHP, \"leaky\" code barely matters because the process dies after 100ms.
\nUnder Octane, a leak doesn't die with the request. It waits and grows.
\nIf a service provider has a static array that you append to on every request, that array grows until your server runs out of RAM. Be careful with:
\nglobal variables at all costs (which you should be doing anyway).Laravel \"resets\" some core services between requests, but it can't catch everything. If you're migrating an old codebase from monolith to microservices, audit your service providers for any long-lived state.
\nThe best insurance against memory leaks is the --max-requests flag. It tells Octane to gracefully restart a worker after it has handled a set number of requests.
Octane already does this every 500 requests by default. Tune the number to match how fast your workers accumulate memory.
\n# Restart workers every 1000 requests to prevent memory bloat\nphp artisan octane:start --max-requests=1000\nThis keeps memory use predictable while still giving you the speed of a warm application.
\n\nSearch your service providers for `static` properties and singletons that store request data. Move that data into the request or clear it between requests before you switch to Octane.\n\nOnce Octane is running, a few technical levers can squeeze out more performance.
\ndatabase.php config and make sure the worker count doesn't blow past your DB server's max connection limit. This matters even more once you add read replicas.Cache::store('octane'). It's an in-memory Swoole table that's very fast and shared across every worker on the box. Use it for frequently accessed configuration or small datasets that rarely change. Remember it's wiped when the server restarts.opcache.validate_timestamps=0 in production since your code won't change while the server is running.php artisan octane:reload. It gracefully restarts the workers without dropping current connections, which you need for zero-downtime deployments.Octane lets PHP compete with Node.js and Go for high-concurrency apps while keeping the Laravel developer experience. If you're moving a busy Laravel app to Octane, here's how I help teams tune workers, connections and deploys for high traffic, or drop a note via contact.
\nAre you running Octane in production yet, or is the fear of memory leaks keeping you on PHP-FPM?
\nAI agents now shop for people inside ChatGPT, Google AI Mode, Gemini, Copilot and Meta. Before an agent can check out on your store, it reads one JSON file at /.well-known/ucp.
Your store can be invisible to a fast-growing group of buyers: AI agents acting for shoppers.
\nIf an agent can't find a machine-readable profile for your Shopify store, it can still mention your products, but it can't complete a purchase for the shopper.
\nTraditional SEO was built for humans with eyeballs. UCP is built for agents with wallets.
\nThe rollout is moving fast, and every new AI channel that adopts UCP can sell your products without a new integration. Here is how to make sure your store is ready.
\nFor ten years we tuned meta tags and JSON-LD schema. That was enough when Google was a list of links. We've now moved into agentic commerce on Shopify.
\nAgents don't browse your collections or look at hero banners. They want three answers:
\nIf your site doesn't answer these in a standard way, the agent moves on to a store that does. Product schema tells an agent what a product is. UCP tells it how to transact.
\n| Layer | \nClassic SEO and schema | \nUCP profile | \n
|---|---|---|
| Audience | \nSearch crawlers and humans | \nAI shopping agents | \n
| Describes | \nPages and products | \nServices, capabilities and payment handlers | \n
| Lives at | \nPage markup | \n/.well-known/ucp | \n
| Lets the reader | \nRank and display products | \nBuild a cart and complete checkout | \n
UCP is an open standard co-developed by Shopify and Google. Shopify lists backers including Amazon, American Express, Etsy, Mastercard, Meta, Microsoft, Salesforce, Stripe, Target, Walmart and Visa.
\nIt defines a handshake between a business and an AI agent, covering cart creation, checkout, payment and post-purchase.
\n/.well-known/ucpThe core of the handshake is a JSON profile at /.well-known/ucp. When an agent reaches your domain, it fetches this file to learn what you support, then negotiates the capabilities you have in common.
The spec requires version, services and payment_handlers inside the ucp object, and capabilities is optional. Versions are dates; the latest stable release is 2026-08-25. Shopify's own Admin and Storefront APIs use dated versions too, released every quarter, and my Shopify API version calendar shows which ones are still supported.
Here is a trimmed version of the profile a live Shopify store serves today:
\n{\n \"ucp\": {\n \"version\": \"2026-08-25\",\n \"services\": {\n \"dev.ucp.shopping\": [\n {\n \"version\": \"2026-08-25\",\n \"transport\": \"mcp\",\n \"endpoint\": \"https://your-store.myshopify.com/api/ucp/mcp\",\n \"schema\": \"https://ucp.dev/2026-08-25/services/shopping/mcp.openrpc.json\"\n }\n ]\n },\n \"capabilities\": {\n \"dev.ucp.shopping.checkout\": [\n {\n \"version\": \"2026-08-25\",\n \"schema\": \"https://ucp.dev/2026-08-25/schemas/shopping/checkout.json\"\n }\n ],\n \"dev.ucp.shopping.cart\": [{ \"version\": \"2026-08-25\" }],\n \"dev.ucp.shopping.catalog.search\": [{ \"version\": \"2026-08-25\" }]\n },\n \"payment_handlers\": {\n \"dev.shopify.shop_pay\": [{ \"id\": \"shop_pay\", \"version\": \"2026-04-08\" }],\n \"com.google.pay\": [{ \"id\": \"gpay\", \"version\": \"2026-01-11\" }]\n }\n }\n}\nThe agent learns which capabilities you support and where to send each request. No scraping, and no guessing where the add-to-cart button is.
\nIf an agent can't read your checkout, it shops somewhere else.
\nOn a standard Shopify or Shopify Plus store, you don't write this JSON by hand. Shopify generates and hosts the profile, and agentic storefronts are active by default for eligible stores.
\nOpen Sales channels > Agentic in your Shopify admin. Allow Shopify to manage for me is on by default. It shares your products through Shopify Catalog, turns on direct checkout where it's supported and enrolls you in new AI channels automatically.
\nTurn it off if you want to control Shopify Catalog access, direct checkout and auto-enrollment yourself, channel by channel. For Google AI Mode and Gemini, products also come through the Google & YouTube channel or Google Merchant Center.
\nOn Hydrogen or a custom frontend, your domain may not serve Shopify's profile at all. Make sure your edge or middleware passes /.well-known/ucp through to Shopify.
I often see agentic discovery break because Vercel or Netlify rewrites, or a catch-all route, swallow the .well-known directory and return the homepage instead.
Search is shifting from links to agents. Shopify's agentic storefronts already reach ChatGPT, Google AI Mode and Gemini, Microsoft Copilot and Meta, and when a new AI platform adopts UCP, Shopify stores are ready to sell there with no extra integration.
\nThis sits next to the Model Context Protocol work covered in MCP context-aware agents. UCP even lists MCP as one of its transports, which is how the profile above exposes the shopping service.
\nIf your profile is missing, your store is a black box. An agent might see your products in a search index, but it can't fulfill \"buy me the best hiking boots from a local store.\" It picks a store where it can confirm checkout right away.
\nDon't flip a switch and hope. Confirm the profile is reachable from outside.
\nhttps://yourdomain.com/.well-known/ucp.ucp key.Check robots.txt too, and make sure you aren't disallowing the .well-known directory.
If a custom Laravel backend sits in front of your store, add an explicit route or proxy rule for this path. I often see middleware strip hidden directories for \"security\" reasons. Here, that rule only costs you sales.
\n\nRun `curl -s https://yourdomain.com/.well-known/ucp` from a terminal and pipe it into my [JSON formatter](/tools/json-formatter/). If the output is HTML or an error, fix the route before you touch anything else.\n\n\n- **Confirm agentic storefronts** under Sales channels > Agentic in your Shopify admin.\n- **Verify `/.well-known/ucp`** returns valid JSON on your own domain, not only on myshopify.com.\n- **Check the payment handlers** in the profile (for example Shop Pay and Google Pay) match what you expect.\n- **Fix headless rewrites** so the `.well-known` path reaches Shopify.\n- **Check your proxy configs** if you host custom middleware, for example on [Coolify Docker SaaS hosting](/blog/coolify-docker-saas-hosting/), so the path isn't dropped.\n\nCommerce is moving from a web built for clicks to one built for agents, and UCP is the first real step toward making your store a first-class seller there. If your store is headless or runs custom middleware, here's how I help Shopify brands get agent-ready.
\nAre you letting agents shop your store yet, or are you still building only for people who click?
\nCustom integrations for every tool are a maintenance tax that never ends. The Model Context Protocol (MCP) replaces them with one plug.
\n\n- Without a standard, M AI apps and N data sources need M x N custom integrations. MCP turns that into one server per tool that any compatible host can use.\n- An MCP setup has three parts: the host the user works in, the client that speaks the protocol, and the server that exposes resources, tools and prompts.\n- Local servers talk over `stdio` and remote servers use OAuth 2.1 with PKCE, so you can give each tool only the access it needs.\n\nEvery time I build a custom integration for a new tool, I add one more thing to maintain forever. We have reached a point where building the product is often faster than setting up the pipes to connect it to our data.
\nThat is the exact problem Claude's MCP (Model Context Protocol) was built to kill.
\nIf you have spent any time building agentic workflows and vibe coding, you know the feeling. You have a very capable LLM like Claude, but it is locked in a room with no windows.
\nTo give it context, you copy-paste code, export CSV files, or spend three days writing a brittle wrapper for a third-party API just so your assistant can \"see\" your work.
\nThis fragmentation is a major bottleneck in modern software development. Our models are capable, but they are cut off from our local files, our databases and our production logs.
\nIt is like having a world-class architect who isn't allowed to visit the construction site. They are guessing based on the photos you decide to send them. Enter the Model Context Protocol, or in Anthropic's own framing, a USB-C port for AI applications.
\nThe problem is simple. Every AI application (Claude Desktop, a custom agent, an IDE extension) wants to talk to your data. On the other side, every data source (your GitHub repos, your Postgres databases, your Slack channels) has its own API and authentication flow.
\nWithout a standard, we are stuck in an M×N problem. If you have 5 AI apps and 10 data sources, you need 50 different integrations.
\n(If you are weighing whether to expose a plain REST endpoint or an MCP server here, I broke that down in API vs MCP.)
\nThis is why most \"AI-powered\" tools feel shallow. They support a few basic integrations, and if you want to use your internal company data, you are back to writing custom glue code.
\nThe cost is real. We waste hours rebuilding the same connectors, every new integration is another potential leak, and the \"magic\" of AI evaporates the moment we hit a data silo.
\n\nWrite down every AI tool your team uses and every data source it needs. Multiply the two numbers. That is how many integrations you would maintain without a shared protocol.\n\nClaude MCP (Model Context Protocol) is a serious, open attempt to standardize how AI applications discover and use data and tools. Instead of building a connector for every model and every tool, you build an MCP server.
\nThis server sits between your data and the AI and exposes a consistent interface that any MCP-compliant host (like Claude Desktop) can understand.
\nIt works like the USB standard. It doesn't matter if you plug in a mouse, a keyboard or an external drive. The protocol is the same, so it just works.
\nThis changes how you build context-aware agents. Instead of hard-coding logic into the agent, you \"plug in\" the servers you need.
\n\nThere are three main players in the MCP ecosystem:
\n2026-07-28 spec there is no up-front handshake: each request carries its own protocol version and capabilities.Say I want Claude to read my local project files. I run a local MCP server that exposes those files as \"resources.\"
\nThe host (Claude Desktop) asks the server: \"what do you have?\" The server replies: \"I have these 10 files and a tool to run grep searches.\"
\nThe model can then call the \"grep\" tool whenever it needs to find a function definition. I didn't write a single line of logic inside Claude to make that happen. I connected the server.
\nOnce a server is built, anyone can use it. The community has already built servers for almost everything. Here are a few I use in my daily workflow:
\nWiring these up is the step that separates chatting with a model from delegating to one. It is the point where a workflow crosses into the agentic levels instead of staying a faster autocomplete.
\nIt also helps with agentic commerce for Shopify. Imagine an agent that talks to your Shopify store via MCP to check inventory levels or update product descriptions in real time, over a secure, standard connection.
\nBuild the connector once. Every host that speaks MCP gets it for free.
\n\nBefore you write a server, search the [MCP Registry](https://registry.modelcontextprotocol.io/) for the tool you need and check whether the vendor maintains an official one.\nWhen you are ready to add a server to Claude Desktop, Claude Code, Cursor or VS Code, my MCP config generator writes the config file and checks the one you already have.\n
\nThe first question I get when I talk about connecting dev tools to an LLM is: \"is it safe?\"
\nSecurity is a first-class concern in MCP's design. The server runs as a separate process that only exposes the tools and resources you grant it.
\n(Note: a local stdio server still runs with your normal user privileges. It is process isolation, not a hardened sandbox, so only run servers you trust.)
\nFor local servers, the protocol typically uses stdio (stdin/stdout). The server can only talk to the host through that narrow pipe. It has no open network ports listening for connections, and it only exists while the host is running it.
For remote servers, MCP standardizes on OAuth 2.1 (with PKCE) for authorization. That allows fine-grained, scope-based permissions. You can authorize a GitHub MCP server to read only public repositories, or a database server to access only specific tables.
\nThis is a big step up from the \"give me your master API key\" approach we have seen in the past. You can treat AI tools with the same least-privilege mindset you use for any other service in your stack.
\nThat matters when you are trying to avoid RAG mistakes in production, where data leakage is a top-tier risk.
\nI have seen plenty of \"standards\" come and go. What makes MCP different is its simplicity and its backing.
\nAnthropic made it open source, and in December 2025 donated it to the Agentic AI Foundation, a Linux Foundation fund, so no single vendor controls it.
\nWe are moving toward \"agentic\" software development. In this world, AI does more than write snippets of code. It acts as an orchestrator that can reach into your cloud infrastructure on GCP, check your Docker logs and suggest fixes for a failing Laravel app.
\nWithout a protocol like MCP, that vision is too expensive and too risky to build at scale. With MCP, tools become plug-and-play.
\nIf you are ready to experiment, here is what I recommend:
\nIf you want your internal tools and dev stack exposed to Claude through scoped MCP servers, here's how I help teams build and secure them, or get in touch directly.
\nWhat is the one internal tool you wish you could \"plug in\" to Claude right now?
\nThe cheapest LLM call is the one you never make. A two-tier Redis cache answers repeat questions in milliseconds and keeps them off your API bill.
\n\n- Put an exact-match cache first (normalized, hashed query keys) and a semantic cache second (RedisVL vector lookup with a strict distance threshold).\n- Guard against the semantic trap: version your context, partition by tenant and invalidate on data changes.\n- Measure hit rate and semantic precision, and review the borderline hits every week before you loosen the threshold.\n\nYou shipped your RAG pipeline. Retrieval is accurate and the LLM is quick. Then you look at your cloud bill and your P99 latency.
\nEvery query, even \"what are your shipping times?\" for the tenth time, runs the full chain: embedding, vector search and an expensive LLM call.
\nAt scale you pay for the same computation over and over. Users wait two seconds for answers that could take twenty milliseconds, and your \"denial of wallet\" risk climbs.
\nThe fix is a smarter cache. Semantic caching with Redis turns a multi-second chain into a single-digit-millisecond lookup. At the hit rates a busy FAQ bot reaches, it can cut LLM API spend by roughly half. Provider prompt caching saves on a different part, the long prefix you resend on every call, and my prompt caching calculator shows when that pays off.
\nStandard caching relies on exact matches. \"How do I reset my password?\" and \"how do i reset my password\" can share a key if you normalize the string. \"Can you help me change my password?\" misses a traditional cache.
\nA plain key-value entry in Redis. I normalize the query (lowercase, trim, strip punctuation) and hash it. My SHA hash generator shows what those keys look like.
\nIt's the first line of defense: almost free, with zero false positives.
\nIf the exact cache misses, I embed the query and look for a close match in a Redis vector index. If a previous question is close enough (a cosine distance of 0.1 or less, about 0.9 similarity), I serve the cached answer instead of calling the LLM.
\nYou never do the heavy lifting twice for the same intent.
\nMost developers think of Redis as a key-value store. With the Redis Vector Library (RedisVL), it also works as a fast vector index.
\nWhy not use your main vector DB, like Pinecone or Weaviate? Latency.
\nYour main vector DB is tuned to search millions of document chunks. The semantic cache is much smaller: recent queries and their answers. Redis usually already sits next to your application, so you skip network hops.
\nI typically see Redis vector lookups finish in under 5 ms. An embedding API call takes around 100 ms and an LLM generation around 1,500 ms.
\n| Path | \nTypical latency | \nLLM cost | \n
|---|---|---|
| Exact cache hit | \nA few ms | \nNone | \n
| Semantic cache hit | \nEmbedding plus under 5 ms lookup | \nNone | \n
| Full RAG pipeline | \n1.5 to 2+ seconds | \nFull call | \n
Every cache hit is an LLM call you don't wait for and don't pay for.
\nThe key setting is the similarity threshold. Too loose and you give users wrong answers (the \"semantic trap\"). Too strict and you never get a hit.
\nI start with a distance threshold of 0.1. RedisVL measures Redis cosine distance on a 0 to 2 scale, where lower is stricter, so 0.1 is roughly 90% similarity.
\nfrom redisvl.extensions.cache.llm import SemanticCache\n\n# Initialize the cache with a conservative threshold and a 4-hour TTL\nllm_cache = SemanticCache(\n name=\"production_rag_cache\",\n redis_url=\"redis://localhost:6379\",\n distance_threshold=0.1,\n ttl=14400,\n)\n\ndef answer(query: str) -> str:\n hit = llm_cache.check(prompt=query)\n if hit:\n return hit[0][\"response\"]\n\n response = run_rag_pipeline(query)\n llm_cache.store(prompt=query, response=response)\n return response\nThis wrapper embeds the incoming query, runs the vector search in Redis and returns the closest cached response when one is within the threshold.
\n\nExport a week of user questions and count exact duplicates after normalizing them. That number is your exact-cache hit rate before you write any semantic code.\n\nSemantic caching is useful but risky if you aren't careful. When your underlying data changes, the cache can keep serving old, wrong answers.
\nI include a context_version in cache keys or metadata. When I re-index the product catalog or update the docs, I bump the version. Old entries stop matching and the next request rebuilds them with fresh data.
If User A asks \"what is my balance?\", you can't serve that answer to User B. I partition the cache:
\ncache:tenant_id:query_hashtenant_id as a filterable field on the vector index.Semantic matches then only happen inside the right security boundary. My post on Laravel multi-tenancy uses the same isolation principles.
\n\nIn a standard cache you set a 3,600-second expiry and forget it. With a semantic cache I use tiers:
\nRedisVL refreshes the TTL on an entry every time check() matches it. A popular answer can live forever on its own, which is why versioning and proactive invalidation matter more than the TTL.
I've written about similar event-driven patterns for handling these updates at scale.
\nDon't turn on the cache and walk away. Track two numbers:
\nI log every semantic hit with its distance. Once a week I sample the hits closest to the threshold (distance between 0.07 and 0.1) and review them by hand. If too many are different questions that only look alike, I tighten the threshold.
\n\nAdd the distance score to your semantic hit logs today. Next week, read 20 hits from the band just inside your threshold and mark each one right or wrong.\n\n\n- **Install `redisvl`** and set up a basic vector index in your dev environment.\n- **Look up in two tiers:** exact first, then semantic.\n- **Start strict:** a distance threshold of 0.05 to 0.1.\n- **Partition and version:** add `tenant_id` and `context_version` to avoid cross-talk and stale answers.\n- **Measure:** watch hit rate, precision and your API bill.\n\nFor more on production RAG, see 7 RAG mistakes in production or browse the architecture archive. If you want a cache like this in front of your own pipeline, here's how I help teams cut RAG latency and LLM spend, or drop me a note.
\nWhich query in your system keeps hitting the LLM when it shouldn't?
\nWhen CPU sits at 20% and inference requests time out, your autoscaler is watching the wrong number. Scale AI workloads on queue depth, KV-cache pressure and time to first token.
\n\n- CPU tracks tokenization and HTTP handling; the real work happens on the GPU, so a CPU-based autoscaler stays calm while users wait.\n- Waiting requests, KV-cache usage and TTFT are leading signals, and vLLM exports all three as Prometheus metrics you can feed to KEDA.\n- Add a forecast for predictable peaks so capacity is warm before the spike, since a new replica can take a minute or more to load model weights.\n\nYour AI app is lagging and users are complaining, but the cloud dashboard says everything is fine. CPU hovers at a comfortable 20% while inference requests time out.
\nTraditional auto-scaling was built for web servers, where CPU and memory are the bottlenecks. For LLM inference, those numbers tell you very little.
\nIf you wait for CPU to hit 80% before adding a pod, the service is in trouble long before the second instance finishes booting. GPU-bound workloads need a different playbook.
\nTo run a cost-effective AI SaaS, move past reactive hardware metrics. Scale on queue pressure, cache pressure and how fast users get their first token.
\nThe Kubernetes Horizontal Pod Autoscaler (HPA) is most often set up to watch CPU utilization. For a Laravel or Node.js API, that works: more requests mean more CPU cycles.
\nAI models are different. The CPU handles tokenization, request routing and HTTP headers. The heavy lifting happens on the GPU.
\nI've seen production clusters where the GPU is pinned at 100% while the CPU sits idle. The autoscaler sees low CPU, decides the pods are fine and never adds a replica, while the queue keeps growing.
\nWhen you start monitoring GPUs, you meet two confusing metrics: utilization and occupancy.
\nGPU utilization is the percentage of time the GPU was running work during a sample period. It lags. By the time it reads 90%, your request queue may have been building for a while.
\nOccupancy measures how many warps are active on each streaming multiprocessor (SM) compared with the maximum. You can have high utilization and low occupancy if your batch size is too small.
\nFor scaling, treat utilization as a baseline and look at what happens before a request even reaches the GPU.
\nTo stop fires before they start, watch queue depth. In vLLM it's the number of requests waiting for a slot in the engine, exported as vllm:num_requests_waiting. SGLang exposes a similar metric.
Queue depth predicts latency. If your model handles 16 concurrent requests before P99 latency starts to climb, set the scaling trigger at 12. To get a real P99 from your own response times, paste them into my latency percentile calculator.
\nScaling on queue depth adds capacity while the current hardware is still inside your SLO. That buys the head start you need to pull a fresh container and load 20 GB of model weights into GPU memory.
\n| Signal | \nvLLM metric | \nWhat it tells you | \n
|---|---|---|
| Waiting requests | \nvllm:num_requests_waiting | \nWork is piling up before the engine | \n
| KV-cache usage | \nvllm:kv_cache_usage_perc | \nHow close you are to preemptions | \n
| Preemptions | \nvllm:num_preemptions | \nRequests are being evicted and recomputed | \n
| Time to first token | \nvllm:time_to_first_token_seconds | \nWhat users feel when they hit enter | \n
| CPU utilization | \n(node metric) | \nTokenization and HTTP load, rarely the bottleneck | \n
In Prometheus, the counter shows up as vllm:num_preemptions_total and the TTFT histogram as vllm:time_to_first_token_seconds_bucket.
Watch the line of waiting requests. It moves before the chip looks busy.
\n\nGraph `vllm:num_requests_waiting` next to your P99 latency for one busy day. Find the queue length where latency starts to climb, and set your scale-out threshold about 25% below it.\n\nIn generative AI, requests vary a lot in size. A 10-token summary is light. A 4,000-token RAG analysis is heavy.
\nThe KV cache is the GPU memory that stores attention keys and values for in-flight sequences. When it's nearly full, the engine preempts a request to free blocks.
\nIn vLLM V1, the default preemption mode is recompute: the evicted request's cache is dropped, and its prompt plus generated tokens are processed again when it resumes. Latency jumps, and P99 starts to look like a mountain range.
\nI recommend scaling on a combination of:
\nvllm:kv_cache_usage_perc).When the cache is full, low GPU utilization doesn't help. You can't fit more work on that chip, so you add replicas.
\n\nReactive scaling always plays catch-up. Even with fast boots, there's a delay. For apps with predictable traffic, I use ARIMA (Auto-Regressive Integrated Moving Average) models to forecast load.
\nIf traffic always spikes at 9:00 am on Mondays, I don't wait for the queue to grow. A time-series forecast brings up the base-load pods at 8:55 am.
\nYou still pay for what you use, and the capacity is there before the first user clicks \"Generate.\"
\nHere is how I structure it:
\nSmarter triggers save money on idle GPUs and spare your users the \"thinking...\" spinner. If you're running inference on Kubernetes, here's how I help teams scale AI workloads on the right signals, or drop me a note.
\nAre you still scaling on CPU, or have you moved to queue-based triggers yet?
\nYour load balancer says CPU is at 40% and everything is fine. Your GPUs are maxed out and your P99 latency is in the gutter.
\n\n- LLM requests vary wildly in weight, so round-robin and CPU metrics hide where the real load is.\n- Prefill (reading the prompt) is compute-bound and decode (writing tokens) is memory-bound, so they compete when mixed on one GPU.\n- Route on token queue depth, KV cache use, TTFT and inter-token latency, and send shared prefixes to the GPU that already has them cached.\n\nYou just scaled your RAG application to a hundred concurrent users. Suddenly, your latency spikes.
\nSome users get their answers in two seconds, while others stare at a loading spinner for thirty. This is the problem GPU-aware load balancing exists to solve.
\nYou are treating your AI models like traditional web servers. Sending a 4,000-token prompt to the same GPU that is generating a 50-token summary is a recipe for disaster.
\nRound-robin routing is a relic of the past for LLM inference. If you don't account for how GPUs handle compute and memory, you waste money and hurt your user experience at the same time.
\nThe fix is a load balancer that understands what is happening inside the model. That means GPU-aware routing, prefill vs decode disaggregation, and treating your KV cache as one of the most valuable assets in your stack.
\nIn traditional software, a request is a request. Whether it's a GET /users or a POST /orders, the variance in resource use is usually small and predictable.
Standard load balancers like Nginx or HAProxy work great here. They look at basic health checks and send traffic to the next available worker.
\nAI is different. One user might ask \"what is 2+2?\" while another uploads a 50-page PDF and asks for a deep analysis.
\nIf your load balancer sends both to the same GPU, the heavy request hogs the compute, and the light request waits in a queue.
\nA GPU can be at 100% utilization while doing very different types of work. Some work is compute-bound and needs raw processing power.
\nOther work is memory-bound, limited by how fast data moves in and out of VRAM. To route well, you have to look inside the inference lifecycle.
\nLLM inference happens in two distinct phases. Understanding the difference is the \"aha!\" moment for GPU load balancing.
\nIn prefill, the model reads your entire prompt and processes all the tokens at once. It is a heavy, compute-intensive task that builds the KV cache (key-value cache).
\nPrefill loves big batches and fast tensor cores. It is where the heavy lifting happens.
\nIn decode, the model generates the response one token at a time. Each new token only needs the previous tokens and the KV cache.
\nThis phase is light on compute but very heavy on memory bandwidth. It is slow and long-lived.
\nWhen you mix the two on one GPU without a smart scheduler, the prefill of a new request often pauses the decode of existing requests. That causes the jittery, stuttering text users hate.
\nGPU-aware load balancing lets you treat these phases differently across your fleet.
\n| Phase | \nWhat it does | \nBound by | \nMetric to watch | \n
|---|---|---|---|
| Prefill | \nReads the full prompt at once | \nCompute | \nTime to first token (TTFT) | \n
| Decode | \nWrites the answer token by token | \nMemory bandwidth | \nInter-token latency (ITL) | \n
To build a better router, stop looking at CPU and start looking at these four metrics:
\nI often recommend tools like vLLM because they expose these metrics out of the box.
\nYou can pipe them into a custom gateway that routes on real-time VRAM availability instead of a simple \"is the server up?\" check.
\n\nScrape TTFT, inter-token latency and KV cache usage from one inference node and put them on a single dashboard next to your P99. Watch which one moves first when latency spikes.\n\nThe most expensive part of a RAG request is often re-processing the same system prompt or long context again and again.
\nIf you send five follow-up questions about the same document to five different GPUs, each GPU runs prefill for that document from scratch. On hosted APIs, prompt caching attacks the same cost, and my LLM cost calculator shows the effect on your monthly bill.
\nPrefix-aware routing (also called KV-cache-aware routing) fixes this. Your load balancer tokenizes the start of the prompt and looks for a GPU that already has that content in its KV cache.
\nMatching the prefix lets you skip prefill for the shared part and reuse the cached keys and values directly. That can turn hundreds of milliseconds of TTFT into a near-instant first token.
\nEvery repeated prefill is a GPU redoing work it already finished somewhere else.
\nThis is one of the most effective optimizations in a production RAG system. I've written before about common RAG mistakes, and ignoring cache locality is one of them.
\nFor the layer above this (caching whole responses and embeddings), see Redis and semantic caching for RAG.
\nAs you scale, stop treating every GPU as a generalist. Build disaggregated inference fleets instead.
\nI like to split my GPUs into two pools:
\nIf your users upload huge documents but ask for short summaries, scale your prefill pool. If they have long, chatty conversations, scale your decode pool.
\nThis is the same logic used in modern DevOps with Coolify. You wouldn't put your heavy database on the same tiny instance as your frontend, so why mix heavy prefill work with light decode work?
\n\nYou don't need to build a custom engine from scratch. Here is the practical path I follow when setting this up for a new SaaS:
\nconversation_id recently and isn't at 100% KV utilization, send the follow-up there so the warm cache gets reused.Managing AI compute means moving from \"black box\" infrastructure to \"context-aware\" infrastructure. When your load balancer knows the difference between a 10-token greeting and a 10,000-token context window, your costs go down and your users stay happy.
\nIt's easy to get lost in the hype of \"agentic systems\" and context-aware agents, but none of that matters if your infrastructure is buckling under unoptimized routing.
\n\n- **Round-robin fails for LLMs** because request weight varies from a few tokens to whole documents.\n- **Prefill is compute-bound, decode is memory-bound.** Mixing them on one GPU causes stutter.\n- **Route on token queue depth, KV cache use, TTFT and ITL,** not CPU.\n- **Prefix-aware routing reuses the KV cache** and cuts time to first token.\n- **Split prefill and decode pools** so you can scale each for your real workload.\n\nIf round-robin is hurting your inference latency, here's how I help teams build GPU-aware routing for LLM serving, or drop a note via contact.
\nIf you are still using round-robin for your AI models, what is the main bottleneck you are seeing in your P99 latency right now?
\nA slow vector database can take your whole app down without ever going offline. A circuit breaker is the switch that stops that from happening.
\n\n- In production, a slow dependency is often more dangerous than a dead one, because waiting requests eat your workers, memory and connections.\n- A circuit breaker has three states (closed, open, half-open) and fails fast once a failure threshold is hit, so the struggling service can recover.\n- Pair the breaker with fallbacks (keyword search, a Redis cache, LLM-only mode) and monitor how often it trips.\n\nYou built a beautiful RAG pipeline. It works perfectly on your machine with a few hundred vectors. Then you launch, traffic spikes and your managed vector database starts sweating.
\nA similarity search that used to take 50ms now takes 5 seconds. Your web workers are all tied up waiting for responses that aren't coming.
\nThe database isn't technically down, but it is slow enough to kill your entire application. Your users see spinning loaders until the request finally times out.
\nThis is a classic cascading failure (the kind a circuit breaker exists to stop), and it is the fastest way to drain your \"innovation budget\" and your users' patience.
\nWe often treat external APIs and databases as if they are always healthy. We write code that assumes the vector DB will return results. When it doesn't, we wait, and while we wait we hold onto memory and CPU cycles.
\nThe fix is an old-school electrical engineering concept applied to software: the circuit breaker. This guide shows you how to wrap your AI infrastructure in protective logic so a slow dependency doesn't take your whole SaaS down with it.
\nThe circuit breaker pattern is a state machine that sits between your application code and your external service.
\nThe idea comes straight out of Michael Nygard's Release It!. It monitors every call you make and has three states that dictate how it handles traffic.
\n| State | \nWhat happens to calls | \nMoves on when | \n
|---|---|---|
| Closed | \nCalls flow through while the breaker counts failures | \nThe failure threshold is hit, then it opens | \n
| Open | \nCalls fail fast or get a fallback, no network call | \nThe cooldown ends, then it goes half-open | \n
| Half-open | \nA few test calls go through | \nTests pass (closed) or fail (open again) | \n
In the closed state, the circuit is complete. Requests flow through to your vector database or LLM provider normally.
\nThe breaker is silently watching. It keeps a count of how many requests failed or took too long. As long as the failure rate stays below your threshold, it stays closed. This is the \"everything is fine\" mode.
\nOnce the failure threshold is hit (say 50% of requests failed in the last 30 seconds), the breaker \"trips\" and moves to the open state.
\nNow every call to the vector DB immediately throws an error or returns a fallback response, without even attempting the network call. This gives your database room to breathe and recover, and your application stops wasting time on a service that is clearly struggling.
\nAfter a cooldown period, the breaker moves to the half-open state and lets a small number of \"test\" requests through.
\nIf these test calls succeed, the breaker assumes the service is healthy again and moves back to closed. If they fail, it goes straight back to open for another cooldown cycle. This is a controlled way to probe the system before fully re-engaging.
\nRAG pipelines are particularly vulnerable because they usually involve several high-latency network hops. You embed the query, search the vector DB, then call the LLM. If any of these pieces fail or slow down, the whole experience breaks.
\nMost developers only handle hard errors like a 404 or a 500 status code. But in production, \"slow\" is often more dangerous than \"down.\"
A slow vector DB creates a bottleneck that backs up your entire request queue. By the time you notice, your server is out of memory because it is holding open thousands of connections.
\nA dead service fails fast. A slow one quietly holds every worker hostage.
\nIf you have read my post on 7 RAG mistakes in production, you know that reliability is the difference between a demo and a product. The circuit breaker is your insurance policy against these outages.
\n\nList every external call in one RAG request (embedding API, vector DB, LLM) and write down its current timeout. Any call with no timeout, or one longer than your page budget, is your first breaker candidate.\n\nTripping the breaker shouldn't always mean showing an error message to the user. Good AI systems use fallbacks to keep some level of service even when parts of the stack are failing.
\nThink of your vector DB as your \"hot\" knowledge tier. If it fails, you should have a \"cold\" fallback, such as a standard keyword search in your primary Postgres or MySQL database.
\nThe results might be less relevant than a vector search, but a \"decent\" answer is always better than a \"timed out\" error.
\nAnother strong strategy is semantic caching with Redis. If the circuit is open, check the cache for similar queries that were answered recently.
\nEven if you can't generate a fresh answer, you might be able to serve a cached one. This keeps the user moving while your backend recovers.
\nIf retrieval is what's failing, you can still send the user's prompt to the LLM with a note that external knowledge is currently unavailable. The LLM then answers from its general training data.
\nIt is a degraded experience, but it still works. Be open about it: tell the user that \"live\" data isn't available so they know to verify the response.
\n\nSince I spend a lot of time in the Laravel ecosystem, I lean on patterns that make this easy. You don't need to write the state machine from scratch.
\nA community package like ackintosh/ganesha (a PHP circuit breaker), or a small custom wrapper around the illuminate/http client with a Redis-backed failure counter, gets the job done.
The goal is to wrap your API calls in a block that understands these states. When you call your vector DB client, you wrap it in the breaker. If the call fails several times, the breaker trips.
\nIn the catch block, you handle the CircuitBreakerOpenException by returning your fallback data. This keeps your controllers clean and the failure handling in one place.
You can also pair this with your SaaS hosting on Coolify so your containers don't get killed by health checks just because an external API is slow. The breaker prevents the resource bloat that usually triggers those health-check failures.
\nDon't set a circuit breaker and walk away. Monitor it. You need live telemetry to see how often your circuits trip, and tools like Prometheus (or even simple logs piped to a dashboard) can tell you a lot.
\nIf your primary vector DB in us-east-1 keeps tripping but your secondary in eu-west-1 is healthy, you can add smart routing. The breaker can act as a signal to your load balancer or internal router to shift traffic to the healthy region.
This kind of event-driven architecture makes your system self-healing. It doesn't wait for a human to wake up at 3am. It detects the failure, trips the breaker, uses the fallback and tries to recover on its own.
\nIf you are ready to harden your AI infrastructure, start here:
\nThe goal is to fail gracefully. Every system has issues, but the ones that survive don't let a small fire in a dependency burn down the whole house.
\n\n- Treat \"slow\" as a failure, not only `500` errors.\n- Closed, open, half-open: fail fast once the threshold is hit, then probe before you trust the service again.\n- Always have a fallback: keyword search, a cached answer or LLM-only mode with an honest notice.\n- Use a library such as `ackintosh/ganesha` instead of writing the state machine yourself.\n- Monitor trips and alert when a circuit stays open.\n\nIf your RAG stack stalls every time the vector database slows down, here's how I help teams add circuit breakers and fallbacks to AI infrastructure, or reach out through contact.
\nHave you ever had a slow dependency take down your entire application, or are you still relying on long timeouts and luck?
\nIf your embeddings run inside the request cycle, the first 50MB PDF will take your app down. Put that work behind a queue.
\n\n- Document processing (OCR, extraction, chunking, embedding) is slow and unpredictable, so it belongs in background workers, not web requests.\n- Split the pipeline into queued stages with explicit retries, exponential backoff and a dead-letter queue for files that can never succeed.\n- Batch embedding calls, keep messages small and idempotent, and scale workers on queue depth instead of CPU.\n\nIf you run document embeddings inside your request-response cycle instead of behind a message queue, you are playing with fire.
\nI have seen many developers build a beautiful RAG application that falls over the second a user uploads a 50MB PDF. The browser spins, the proxy timeout hits and the database locks up while your worker tries to chunk 500 pages of legal jargon in real time.
\nThis is the classic \"heavy lifting\" problem in AI engineering. Document processing (OCR, text extraction, semantic chunking and embedding) is slow, unpredictable and resource-heavy.
\nForcing it into a synchronous web request gives you a bad user experience and a fragile system.
\nThe fix is decoupling with message queues. Here is why async work belongs in a queue and how to build an ingestion pipeline that doesn't melt your server.
\nIf you want the broker-level foundations first, I cover them in scaling with RabbitMQ.
\nImagine a user uploads a document to your SaaS. Your code receives the file, sends it to an extraction API and waits.
\nThen it loops through the text to create chunks, sends each chunk to an embedding model and finally saves it all to pgvector.
\nIf the whole chain runs long, you hit a timeout somewhere: a proxy gateway timeout (Nginx defaults to 60s), a PHP max_execution_time or a platform request cap. The connection drops.
If the embedding API has a momentary blip, the whole process fails and the user has to start over. While your server is busy with this heavy work, it isn't responding to other users.
\nA good rule of thumb: if it takes more than 100ms, consider making it async.
\nMoving this work to a message queue gives your users immediate feedback (\"we're processing your file!\") while the heavy lifting happens safely in the background.
\n\nA solid RAG pipeline is a series of decoupled stages, not one big function. I break it into modular steps, each triggered by a message in a queue, so you can scale each part on its own.
\nHere is how I usually structure it:
\nfile_path and tenant_id.This stage-based approach is what I discuss in my post on 7 RAG mistakes to avoid in production.
\nIt gives you backpressure control. If your vector DB slows down, the \"index\" queue grows, but the \"parsing\" workers keep going.
\nWhen one stage slows down, its queue grows. Every other stage keeps working.
\nIn the real world, things break. APIs time out. PDFs are malformed. Workers crash.
\nA message queue like Redis (with BullMQ or Laravel Queues) or SQS gives you retries with almost no code, but you do have to opt in:
\nattempts and a backoff strategy explicitly.--tries and backoff.maxReceiveCount.When a worker fails, the message goes back onto the queue to be tried again after a delay. Use exponential backoff instead of hammering a failing API every 5 seconds: wait 10, then 60, then 300.
\nSome documents simply cannot be processed. Maybe it's a password-protected PDF or a corrupted file, and you don't want it retrying forever and clogging your workers.
\nThis is where a dead-letter queue (DLQ) comes in. After a set number of failed attempts, the message moves to the DLQ, a \"quarantine\" zone.
\nSQS gives you this natively through the queue's redrive policy. With BullMQ or Laravel, you route exhausted jobs there yourself (listen for the failed event in BullMQ, or use failed_jobs plus a handler in Laravel). You can then inspect the failed jobs, fix the issue and re-queue them. My PHP unserialize tool decodes a failed_jobs payload so you can see exactly what the job received.
If you process 10,000 chunks, you don't want 10,000 separate API calls to your embedding provider. That is slow and expensive.
\nMost embedding APIs and vector databases work much better with batches. A good worker pulls several messages from the queue (or gathers them in memory) and sends them as one bulk request.
\nIn Laravel, I often use job batching to track the progress of a large document. I can see exactly when 95% of a PDF is processed and update a progress bar for the user.
\nFor how this fits a larger architecture, see my thoughts on event-driven pub/sub systems.
\nQueues help beyond ingestion. You can also use them for prefetching.
\nIf a user is chatting with an AI agent and the conversation is heading toward a topic, you can fire a background job to fetch related documents and warm the cache before they ask the next question. The AI feels very fast because the context is already \"ready\" when retrieval runs.
\nAn event bus decouples the chat interface from these optimization tasks. The chat app just emits a user_asked_question event, and a background worker decides whether to prefetch more data or update the semantic cache.
Once you move to a queue-based system, request latency stops being the only number that matters. You need to watch your queue depth.
\nIf the queue grows faster than your workers can clear it, you have a bottleneck.
\nTools like Docker and Coolify make this easy. I can spin up five more worker containers to handle a sudden surge in uploads.
\nYou can read more about how I manage this infra in my Coolify and Docker guide.
\nA document pipeline is about respecting the time it takes to process data. If big uploads are timing out your RAG app, here's how I help teams build queue-based ingestion pipelines with retries and autoscaling, or drop a note via contact.
\nHow are you handling long-running AI tasks today: still fighting request timeouts, or already using a queue?
\nOne runaway agent loop is all it takes to wake up to a $5,000 LLM bill. In AI apps, rate limits protect your margins as much as your servers.
\n\n- Requests per second hide cost: one request can be 50 tokens, the next 128,000. Limit tokens, not pings.\n- Use three layers of token buckets in Redis: a global provider cap, a per-tenant budget tied to the plan, and a per-user share inside each tenant.\n- Degrade gracefully when a budget runs out, and keep your own caps even with provider hard limits, because a provider cap stops every tenant at once.\n\nIf you're building AI-powered SaaS or RAG systems, the main threat is often a \"denial of wallet\" attack rather than a server crash.
\nA buggy client, a malicious user or your own experimental agent can spam your endpoints and burn through tokens and credits in minutes.
\nTraditional web apps watch requests per second to keep the CPU from melting. With LLMs, you watch tokens per minute to keep the bank account from draining. For a quick feel of how text turns into tokens, try my token counter.
\nYou need limits that understand cost, context and the \"noisy neighbor\" problem before a single prompt reaches your vector DB.
\nIn a standard Laravel or Node app, a request is a request. Some take longer, but they use roughly similar resources.
\nIn AI engineering, one request might be a 50-token greeting and the next a 128,000-token context dump for a RAG pipeline.
\nIf you only limit requests, a user can stay inside \"10 requests per minute\" and still cost you 100x more than an average user. That's where the noisy neighbor problem turns into a financial one.
\nTo protect margins, move from counting pings to counting cost.
\nA denial of wallet (DoW) attack is the AI version of a DDoS. The goal is to exhaust your quotas or budget until the service stops, or until you pay a huge bill.
\nWithout token-aware limits, the provider eventually answers with 429 rate-limit errors. By then the money is already spent.
I use three layers of limits at the API gateway. Even if one tenant goes rogue, the rest of the platform stays healthy.
\nThis is the last line of defense. If your provider quota is 500,000 tokens per minute (TPM), set your internal global limit to 450,000.
\nThe buffer lets you degrade on your own terms instead of hitting a wall of provider 429s, which means failed user requests and retry timing you don't control.
Every customer gets its own bucket, usually tied to the subscription tier. A \"Pro\" tenant might get 50,000 TPM while a \"Free\" one is capped at 2,000. No single company can eat your whole global quota.
\nInside a tenant you still need limits, so one employee can't use the whole organization's allocation. I set these at about 20% of the tenant's capacity.
\n| Layer | \nProtects against | \nExample limit | \n
|---|---|---|
| Global | \nHitting the provider's own limits | \n90% of provider TPM | \n
| Tenant | \nOne customer draining the platform | \nPro 50,000 / Free 2,000 TPM | \n
| User | \nOne person draining their company | \nAbout 20% of tenant budget | \n
Count what a request costs, not how many requests arrive.
\nFor most builds I use a token bucket backed by Redis. It refills at a steady rate and lets a request through only if enough tokens are available, which absorbs short bursts while capping the sustained rate. For how it differs from a leaky bucket, see rate limiting vs throttling.
\nEach tenant or user has a bucket. For every prompt, you estimate the total cost (input tokens plus the requested max_tokens). If the bucket has enough, the request proceeds and the tokens are deducted.
On a Laravel stack, you can do this in middleware with Redis, where atomic operations keep per-tenant counters cheap and race-free.
\n// A simplified token-bucket check in Laravel middleware\npublic function handle($request, Closure $next)\n{\n $tenantId = $request->user()->tenant_id;\n $estimatedTokens = $this->tokenizer->estimate($request->input('prompt'));\n\n if (!$this->limiter->consume(\"tenant:{$tenantId}:tokens\", $estimatedTokens)) {\n return response()->json(['error' => 'token budget exceeded'], 429);\n }\n\n return $next($request);\n}\nWhat happens when a user hits the limit? The default is a 429. I prefer graceful degradation, which I call adaptive throttling.
The user experience stays intact while your margins stay protected.
\n\nIn a RAG system, one user query often triggers several backend actions:
\nIf you only limit the final LLM call, search queries can still hammer your vector database. Treat the whole RAG flow as one unit of work with its own combined budget.
\nThere are plenty of common RAG production mistakes, and limiting only the LLM call is one of the most overlooked.
\nBoth major providers now offer hard spend limits. OpenAI's organization and project hard limits return a 429 (organization_spend_limit_exceeded or project_spend_limit_exceeded) once tracked spend reaches the amount. Enforcement isn't instantaneous, so spend can slightly overshoot, and plain spend alerts only notify.
On Anthropic, a spend limit you set in the Console makes requests fail with a 400 until you raise it or the period resets.
Turn them on as a backstop. A provider cap stops every tenant at once, so the per-tenant caps still belong in your app.
\nmax_tokensNever allow an uncapped response. Set a sane default for max_tokens on every API call.
If an LLM call runs longer than 30 seconds, kill it. Slow calls are often the first sign of a system about to spiral.
\n\nOpen your provider console today and set a hard spend limit at about twice your normal monthly spend, plus alerts at 50% and 80%. Then write down what your app should do when that limit returns errors.\n\n\n- **Limit tokens, not requests.** Request counts hide the 100x cost spread between prompts.\n- **Layer your buckets.** Global, tenant and user limits keep one bad actor from hurting everyone.\n- **Degrade before you deny.** Cheaper models, trimmed context and queues beat a wall of `429`s.\n- **Budget the whole RAG flow.** Embeddings and vector searches cost money too.\n- **Use provider hard limits as a backstop.** Your own caps do the fine-grained work.\n\nRate limiting is part of your business model in the AI era. You can't scale a product that lets one user run up a thousand-dollar bill in the first hour. If you're adding AI to a multi-tenant product, here's how I help teams build token budgets and gateway limits, or drop me a note.
\nHave you seen a \"denial of wallet\" happen in the wild, or are you still relying on provider limits alone?
\nStop exposing your models to the wild. Put a gateway between your users and every LLM call.
\n\n- Calling a RAG orchestrator or an LLM provider straight from the frontend is slow, insecure and the fastest route to a surprise five-figure bill.\n- An API gateway centralizes auth, tenant isolation, model routing and per-user rate limits before any token is spent.\n- AI traffic needs two special settings: no response buffering for streaming, and upstream timeouts long enough for slow generations.\n\nIf you are building a production AI app, sending requests directly from your frontend to a RAG orchestrator (or, god forbid, straight to an LLM provider) is a liability.
\nIt is slow. It is insecure. And it is the fastest way to wake up to a five-figure bill you didn't plan for.
\nI have spent over a decade building software. Engineering for \"it works\" is not the same as engineering for \"it scales.\"
\nIn AI, scale means cost, latency and data safety as much as traffic.
\nImagine a \"denial of wallet\" attack where a malicious script spams your completions endpoint. Without a gatekeeper, your API keys are sitting ducks.
\nOr worse, imagine a multi-tenant app where one user's prompt accidentally retrieves another user's private data from your vector DB.
\nThis is where the API gateway comes in. It is the first line of defense and the brain of your infrastructure. It handles the boring but critical work so your RAG logic can stay focused on being smart.
\nAt its core, an API gateway is a reverse proxy that sits between your users and your backend services.
\nFor an AI stack, forwarding traffic is the smallest part of its job. It is the central point for auth, routing and rate limiting. (If the distinction blurs for you, I break it down in load balancer vs API gateway.)
\nWhen a request hits your gateway, it runs a gauntlet of checks before it ever touches a model. This \"gatekeeper\" makes sure every millisecond of GPU time and every cent of token cost is intentional.
\nEvery token you pay for should pass a checkpoint first.
\nIn a typical SaaS, authentication is about knowing who the user is. In an AI-powered SaaS, it is about data sovereignty.
\nIf you are building a RAG system, your biggest risk is cross-tenant data leakage. To avoid common RAG mistakes, handle identity at the very edge.
\nI prefer JWTs (JSON Web Tokens) with custom claims. When a request hits the gateway, I validate the token and extract the tenant_id.
That ID is injected into the request headers before it is passed to the RAG orchestrator.
\nThe orchestrator receives a verified x-tenant-id header and uses it to apply metadata filters on the vector database. The user only \"sees\" data they are allowed to see.
No tenant ID? No query. Period.
\n\nSend a request to your RAG endpoint with no tenant header and one with a forged tenant ID. Both should be rejected at the gateway before the orchestrator logs anything.\n\nThe AI world moves fast. Today you are using GPT-6 Astra. Tomorrow, Claude Opus 5.5 might be the better play.
\nNext week, you might want to test a fine-tuned open-weights model, like Meta's Muse Glimmer, running on your own infrastructure via Docker and Coolify.
\nIf your model logic is hardcoded into your frontend or a single monolithic backend, switching models is a nightmare. An API gateway solves this with smart routing.
\nI use the gateway to create \"model aliases.\" Instead of calling a specific model, the frontend calls a generic endpoint like /chat/pro. The gateway then decides where to send the request based on:
This level of abstraction is what separates a weekend project from a resilient SaaS product.
\n\nWe used to rate limit to protect our CPUs. Now we rate limit to protect our bank accounts.
\nAI requests are asymmetric. A user sends a 50-word prompt, and the model might generate a 1,000-word response. The cost difference is massive.
\nA good gateway setup allows tiered rate limiting. Set global limits so the whole system can't be overwhelmed, and set per-tenant or per-user limits too.
\nI usually implement this with Redis. The gateway checks the user's quota in real time.
\nIf they have exceeded their daily token limit or their requests-per-minute (RPM) cap, the gateway returns 429 Too Many Requests immediately. I go deeper on the token-budget angle in rate limiting to protect your AI wallet.
This saves your backend from doing expensive work you won't get paid for.
\nIt also stops \"noisy neighbors\": one user scripting an automated tool that hogs your capacity and makes the app slow for everyone else.
\n\nPick your most expensive AI route and set two limits on it today: a per-user daily token cap and a global requests-per-minute ceiling. Return a clear 429 message so the frontend can explain it.\n\nGateways for AI need to handle two things differently than traditional web apps: streaming and long-running requests.
\nMost modern AI apps use Server-Sent Events (SSE) to stream responses word by word. Some older gateways or load balancers \"buffer\" the entire response before sending it to the client. This kills the user experience.
\nMake sure your gateway (whether you use Kong, Tyk or a custom Laravel solution) disables buffering for AI routes.
\nThe data should flow through the gateway like water through a pipe, not like a bucket that needs to fill first.
\nTraditional APIs expect a response in 1 to 2 seconds. A complex RAG query with multiple vector searches and a large model generation might take 30 seconds or more.
\nYou need to adjust your gateway's \"upstream timeout\" settings:
\n| Gateway | \nDefault upstream timeout | \nCan you raise it? | \n
|---|---|---|
| Kong | \n60 seconds | \nYes, per service | \n
| AWS API Gateway (REST) | \n29 seconds | \nOnly on Regional or private REST APIs, via a service quota increase that may cost throttle headroom | \n
Leave those defaults in place and your users will see 504 Gateway Timeout errors even when your models are working perfectly.
You don't need a massive team to set this up. Here is how I usually approach it depending on the project size.
\nUse a cloud-native gateway like AWS API Gateway or Azure API Management. They are serverless, scale automatically and integrate directly with Cognito or Entra ID for auth.
\nKong is the gold standard, with a great ecosystem of plugins for rate limiting and auth. If you are comfortable with PHP, a thin Laravel app acting as a gateway works surprisingly well for custom logic.
\nIf you are building agentic commerce tools, use the gateway to handle Shopify HMAC validation before passing the request to your AI agents. When a signature fails, my Shopify webhook HMAC verifier helps you find the cause.
\nThe API gateway is infrastructure, but it is also a statement: your AI logic is too valuable (and too expensive) to leave unprotected.
\nBy centralizing auth, routing and rate limiting, you make your system more modular. You can swap models, change pricing tiers and update security policies without touching the core code that makes your AI \"smart.\"
\n\n- **Centralize auth.** Never let your RAG orchestrator handle raw user authentication. Do it at the gateway.\n- **Inject tenant context.** Verify the user at the gateway and inject a `tenant_id` header to enforce data isolation.\n- **Set global and per-user limits.** Protect your wallet from both malicious attacks and accidental bugs.\n- **Configure for streaming.** Make sure your gateway doesn't buffer responses, or your \"typing\" effect will break.\n- **Use model aliases.** Route to `/chat/pro` instead of a specific model name to keep your stack flexible.\n\nIf you're putting an LLM-backed product in front of real users and want auth, routing and cost limits done right, here's how I help teams build the gateway layer.
\nAre you still letting your frontend talk directly to your LLM providers? If so, what is the one thing stopping you from putting a gateway in front of it?
\nHeadless used to mean a team of engineers just to show a \"Buy\" button. Shopify Storefront Web Components put products, collections and a cart on any page with one script tag.
\n\n- Storefront Web Components are standard custom elements that talk to the Storefront API for you: no React, no build step, just HTML.\n- You load one module script, set the store with ``, pick a product with `` and render fields with `` and ``.\n- Use them for buy buttons, landing pages and content sites; reach for Hydrogen when you need a full custom storefront with server-side logic.\n\nIf you wanted a custom Shopify experience outside Liquid themes, you used to face a big architectural decision.
\nYou built a full React or Hydrogen app, managed server-side rendering, handled routing and hoped your SEO survived. For many brands that was like buying a semi-truck when they needed a bicycle.
\nTraditional headless kept simple content sites on Astro or WordPress from selling products without a jarring jump to a subdomain. We've spent years over-engineering problems a few custom elements could solve.
\nStorefront Web Components change the math. Adding commerce to a site starts with a script tag.
\nBuilding a headless store feels like building a house from scratch. You handle the foundation (hosting), the plumbing (GraphQL queries) and the wiring (state management).
\nYou can spend weeks configuring Hydrogen or a custom Next.js setup just to get a cart that works. Once it's live, you maintain the Node.js environment and the middleware too.
\nIt's expensive and slow to deploy, and for many use cases it's overkill. We need the flexibility of headless without the upkeep of a heavy framework.
\nStorefront Web Components are framework-agnostic. They're standard Web Components (custom elements) that handle the calls to Shopify's Storefront API.
\nAll you need is HTML: no React, no build step.
\nThis approach embeds products, collections and a working cart (<shopify-cart>) in your existing stack. On a static site generator or an older CMS, they act as a bridge: your content stays where it is and the commerce loads in on the page.
Here's the workflow I use when I want to move fast.
\nAdd the module script and a <shopify-store> element to your HTML. That connects the page to your Shopify domain.
<!-- Load the components script. Docs:\n https://shopify.dev/docs/api/storefront-web-components -->\n<script type=\"module\" src=\"https://cdn.shopify.com/storefront/web-components.js\"></script>\n\n<shopify-store store-domain=\"https://your-store.myshopify.com\"></shopify-store>\nThe public-access-token attribute is optional. Add one from the Headless channel in your Shopify admin when you need inventory, metafields or metaobjects. Titles and prices work without it.
<shopify-context> tells the page which product or collection to load. Pass the handle from your Shopify admin.
<shopify-context type=\"product\" handle=\"awesome-tshirt\">\n <template>\n <!-- your content goes here -->\n </template>\n</shopify-context>\nInside the template, <shopify-data> pulls specific fields with dot notation, which feels familiar if you've worked with Liquid or JavaScript objects.
<shopify-data query=\"product.title\"></shopify-data>\n<shopify-money\n query=\"product.selectedOrFirstAvailableVariant.price\"\n></shopify-money>\nShopify publishes a machine-readable llms.txt for Storefront Web Components. To write one for your own store or docs, my llms.txt generator builds the file from a short form and checks it against the format.
\nIf you've followed agentic workflows and vibe coding, you know context is everything for an AI model. Point your assistant (Claude, ChatGPT or similar) at this file and it learns the components' actual attributes and patterns.
\nInstead of guessing how a Shopify component should look, the model works from the official reference. With this link in my prompt, I can draft product landing pages much faster, and far fewer of them need fixing before they work.
\nFor more on feeding docs to agents, see how MCP context-aware agents handle technical documentation.
\nGive the model the official reference and it stops inventing attributes.
\nHere's my rule of thumb.
\n| Use Storefront Web Components when... | \nUse Hydrogen when... | \n
|---|---|
| You want buy buttons or a mini-cart on an existing site | \nYou're building a full custom storefront | \n
| You're shipping a landing page and speed matters most | \nYou need server-side logic and deep API integrations | \n
| Your team won't maintain React and Node.js | \nYou want streaming, edge rendering and fine performance control | \n
| You want to stay framework-agnostic | \nYou have the team to run it as a product | \n
For most teams starting out or modernizing a site, Web Components are the practical choice. They balance AI-driven development with production stability. If you're still weighing the bigger call, see Shopify Liquid vs. headless.
\nBuilding for the web in 2026 is about removing friction, and these components remove most of it from headless. If you want commerce on a site that isn't Shopify, here's how I help brands add Shopify to Astro, WordPress and custom sites, or drop me a note.
\nAre you still building full React apps for simple stores, or have you tried custom elements yet?
\nYour beautiful Shopify theme is about to become invisible. The next shopper has no eyes, never clicks a button and only reads your data.
\n\n- AI agents now do the discovery, comparison and checkout for a growing share of buyers, so a store that agents can't read drops out of the transaction.\n- Shopify's answer is three layers: the Universal Commerce Protocol (UCP), the Catalog API and Agentic Storefronts, with MCP as the tool binding underneath.\n- The developer job moves from pixels to protocols: native variants, rich Metaobjects, GraphQL, MCP servers and clean product data.\n\nFor a decade, we have obsessed over conversion rate optimization, pixel-perfect layouts and fast-loading Liquid templates. We built for human eyes.
\nBut a new type of shopper is arriving at the digital storefront. They don't have eyes, they don't click buttons, and they don't care about your hero slider.
\nThey are autonomous AI agents.
\nThe shift from human browsing to agentic commerce is a big architectural shift for Shopify. If you are still \"the theme guy\" or \"the Liquid expert,\" your skill set is about to hit a wall.
\nI have been building on the web for over ten years, and the era of building for agents is here. It is time to move from designing pixels to designing protocols.
\nMost developers still think the old way. We build a site, drive traffic to a landing page and hope the user finds the checkout. This is a manual, high-friction process.
\nToday, buyers increasingly use tools like ChatGPT, Gemini or Microsoft Copilot to do the \"work\" of shopping. They ask for a waterproof hiking boot under $150 with specific shipping requirements.
\nThe AI doesn't visit your store to browse. It queries data.
\nIf your store isn't built to be \"read\" by an agent, you simply don't exist in that transaction.
\n\nAsk ChatGPT, Gemini and Copilot for one of your best-selling products the way a real buyer would phrase it. Note whether your store shows up, and whether the price and stock it quotes are correct.\n\nWe have spent years building \"utility\" apps that handle simple automations. But Shopify is moving fast to pull those features into the core.
\nWith tools like Shopify Magic and Sidekick, the basic stuff (copywriting, image editing, simple workflows) is being commoditized.
\nThe real value is moving deeper into the stack. Agents need structured data, real-time inventory logic and a way to run a checkout without a browser window. A product description alone won't do.
\nIf you rely on \"fake\" variants or messy metafields, you break the agent's ability to reason about your store.
\nWhen an agent fails to understand your product structure, the buyer gets \"no results found\" or a hallucinated alternative. That is a lost sale for your client and a failed project for you.
\nAgentic commerce is a system where AI agents take over the discovery, comparison and execution phases of shopping. Shopify is laying the groundwork with three pillars.
\nThe open standard Shopify co-developed with Google for how agents discover, negotiate and transact with any merchant.
\nIt is composable: it defines the conversation, then plugs into MCP for tool access and AP2 for payment authorization. For a hands-on walkthrough, see my Shopify UCP quick-start.
\nUCP's discovery layer. It standardizes products into a universal taxonomy, verifies pricing and inventory in real time, and syndicates that structured data to connected AI surfaces.
\nThe management layer. You configure product data once in the Shopify admin, then distribute and sell across AI channels (ChatGPT, Microsoft Copilot, Google's AI Mode, the Gemini app).
\nAs developers, our job is shifting from visual storefronts to \"agent-ready\" infrastructure.
\n\nIn the agentic era, your structured data is your most important asset. A well-built JSON-LD schema or a clean set of Shopify Metaobjects is now worth more than a fancy hover effect.
\nAgents rely on clarity. They need to know if a product works with another one, what the material breakdown is, and exactly when it will arrive.
\nIf you want to see how I've helped businesses modernize through better architecture, check out some of my previous work.
\nProductVariant records to function.An agent can't admire your layout. It can only read your data.
\n\nExport your catalog and count products that fake variants as separate products or hide specs in the description HTML. That count is the scope of your first \"agent-readiness\" cleanup.\n\nThe Model Context Protocol (MCP) lets us connect AI models to external data sources like the Shopify API. It is the same binding UCP uses to expose commerce tools to agents.
\nPicture an \"inventory agent\" for a merchant. A plain alert fires when stock is low. This agent goes further: it queries recent sales trends via the Admin API, checks supplier lead times and suggests a restock amount, all through a chat interface.
\nWith Laravel and Docker, we can build custom middleware that acts as an MCP server and exposes specific Shopify \"tools\" to an AI agent.
\nThis deep technical work is what separates a senior engineer from a freelancer who only installs themes.
\nTo thrive in the next five years, I recommend focusing on four technical areas.
\nStart offering \"agent-readiness\" audits. Clean up product data, normalize attributes and make sure all metadata is machine-readable. This is the new SEO. A short llms.txt file that points agents at your key pages is an easy first item on that list.
\nLiquid isn't dead, but headless architectures using Hydrogen and Oxygen fit the agentic future better. They force you to think in data and APIs rather than templates.
\nIf you're weighing the trade-offs, I broke them down in Shopify Liquid vs. headless.
\nLearn how to use RAG (retrieval-augmented generation) and tools like pgvector. You can build agents that search thousands of product reviews or support docs to give \"expert\" advice to shoppers.
\nIf you're new to this, start with picking the right RAG stack.
\nShopify classifies agent traffic into three trust tiers: Anonymous, Signed and Token. All three can search the catalog and build carts and checkouts. Only a Token-tier agent whose token has been granted purchase permission can call complete_checkout and finish the order.
Direct checkout also depends on the channel. Copilot, Google's AI Mode and the Gemini app offer native checkout powered by UCP (Google's is still rolling out to more brands). In ChatGPT, the buyer completes the purchase on the merchant's online store inside an in-app browser.
\nThere is also a dedicated Agentic plan for brands on any platform. They list products in Shopify Catalog and sell through agentic storefronts with no monthly fee, paying card rates (from 2.9% + 30¢ USD online) when a product sells.
\nFor the identity side, see UCP agent profiles and buyer-linked tokens for agent checkout.
\nThe change is happening faster than most of us realize. Shopify reports that in Q1 2026:
\n| Signal | \nQ1 2026, year over year | \n
|---|---|
| AI-driven traffic to its stores | \nGrew roughly 8x | \n
| Orders from AI-powered searches | \nClimbed nearly 13x | \n
| New buyers via AI channels | \nOrdering at close to 2x the rate of other channels | \n
More and more of commerce will run through agents. It is a great time to be a developer if you are willing to learn the new protocols. You can read more about me and my background in web applications.
\nIf you want your Shopify catalog and checkout ready for AI agents, here's how I help merchants and teams get agent-ready, or drop a note via contact.
\nAre you building a store that only humans can see, or one that the whole world can buy from?
\nA RAG demo takes an afternoon. A RAG system that users trust takes fixing the seven structural mistakes below.
\n\n- Most RAG failures in production come from retrieval, not from the model: bad chunks, no reranker, no hybrid search and no metadata filters.\n- Treat embeddings like a database schema. Pin the model, version the index and re-index when either changes.\n- Stop vibe-checking. A golden set of 50 to 100 questions scored on faithfulness, relevance and context precision tells you if a change helped.\n\nGetting a RAG (retrieval-augmented generation) demo working is easy. You take a few PDFs, throw them into a vector database like Chroma or Pinecone, and ask a question. It feels like magic.
\nIf you're still choosing your foundation, my guide to picking the right RAG stack covers the trade-offs before you commit.
\nBut shipping RAG to production is where the magic dies.
\nI've seen too many teams launch a feature only to find users getting irrelevant answers, waiting 10 seconds for a response, or getting \"I don't know\" for questions that are clearly in the documentation. When the \"vibe check\" fails at scale, your users lose trust.
\nI've spent the last few years building custom web applications and AI systems, and I've had to fix these same leaks in my own stacks. Here is how to get from \"it works on my machine\" to a production-grade AI system.
\nMost people start with a simple character-based or token-based splitter. You tell the library to \"give me chunks of 500 tokens with a 50-token overlap.\"
\nThis is a mistake.
\nNaive chunking treats your data like raw soup. It might cut a sentence in half, split a table in the middle of a row, or separate a code example from the explanation before it.
\nIf the retriever pulls only one of those halves, the LLM has zero chance of giving a correct answer.
\nI always recommend chunking on the actual structure of the document first. Use headers (H1, H2, H3), paragraphs or Markdown delimiters so related ideas stay together.
\nFor complex data, use recursive character splitting that respects newlines and punctuation before it falls back to raw token counts.
\nYou can compare strategies on your own text with my RAG chunk splitter. Bad chunking is one of the most common reasons your RAG fails in production.
\n\nPaste one of your longest real documents into the [RAG chunk splitter](/tools/rag-chunk-splitter/) and count how many chunks cut a sentence or a table row in half. If it is more than a couple, switch to header-based splitting first.\n\nVector search is great at finding \"roughly similar\" content, but it is not a precision instrument. It relies on cosine similarity, which documents can fool when they share a similar \"vibe\" but don't contain the answer.
\nIf you take the top 5 results from your vector store and shove them into the prompt, you're leaving quality on the table.
\nI look at retrieval as a two-stage process:
\nThe reranker acts like a bouncer at a club. It doesn't care if a document looks \"okay.\" It only lets in the ones that are relevant to the question.
\nVector search finds neighbors. A reranker finds answers.
\nThis is the silent killer. I've seen teams upgrade their embedding model from text-embedding-ada-002 to text-embedding-3-small without re-indexing their entire database.
Suddenly, the vectors generated for new queries don't \"line up\" with the vectors stored in the index, and similarity scores go haywire.
\nEven worse is changing the preprocessing logic (like how you format chunks) while keeping the old vectors.
\nTreat your embedding model like a database schema. If you change the model, you must re-index.
\nI always include the model name and version in the metadata of every index I build. That way I can test a new model side by side without breaking the production flow.
\nMy experience in cloud infrastructure has taught me that consistency beats a \"better\" model that doesn't match its data.
\n\nAdd an `embedding_model` and `embedding_version` field to every chunk you write today, and make your query code refuse to search an index whose version does not match the model it is using.\n\nEveryone wants more context. We see context windows of 128k or even 1M tokens and think, \"great, I'll give the LLM everything!\" This is a trap for two reasons.
\nFeeding tens of thousands of tokens of context into an LLM inflates your time-to-first-token. It can push response times into double-digit seconds.
\nModels still struggle with the \"lost in the middle\" effect. They reliably use information at the start and end of a long context but tend to ignore facts buried in the center.
\nA bigger context window is not the same as memory, and stuffing it rarely helps.
\nI start with a latency budget. If the user expects a response in under 2 seconds, I can't afford to send 20 chunks.
\nI limit retrieval to the top 3 to 5 high-quality chunks and start streaming as soon as the first token is ready.
\nIf you need more data, use a multi-step approach: a cheaper model summarizes the retrieved chunks before the refined info goes to your main model.
\nVector search is bad at finding specific keywords or unique identifiers.
\nIf a user asks for \"error code XF-904,\" a vector search might return documents about general error handling because the \"vibe\" is similar. It might miss the one document that actually mentions \"XF-904.\"
\nI always combine dense vector search with traditional sparse search (like BM25). Blend the two result lists with something like Reciprocal Rank Fusion (RRF).
\nYou get the semantic understanding of vectors and the keyword precision of full-text search. This is non-negotiable for enterprise search or technical documentation.
\nIf keyword precision matters more than you expect, it's worth weighing vector search against graph search for your retrieval strategy.
\n\nIf your RAG system holds documents for different customers, versions or dates, pure vector search will betray you.
\nYou might ask about \"API changes in 2024\" and get results from 2022 because they share similar keywords.
\nRelying on the LLM to \"ignore\" the wrong dates in the context wastes tokens and invites hallucinations.
\nApply filters before the vector search even happens. If you know the user wants \"v2\" of your documentation, filter the query to chunks with version: '2'.
This shrinks the search space and improves accuracy. I use this heavily when building Shopify apps, where data must be strictly siloed by shop ID.
\n\nList the three questions your users ask most and write down which metadata field (version, date, tenant, product) would narrow each one. Add those fields to your index before you tune anything else.\n\nHow do you know your RAG stack is getting better? Most devs ask a few questions, see that the answers look okay, and ship.
\nThis \"vibe-checking\" doesn't work. When you change a prompt or a chunk size, you might improve one answer while breaking ten others you didn't check.
\nI use the Ragas framework or simple LLM-as-a-judge patterns to run automated evals. I keep a \"golden set\" of 50 to 100 questions with ground-truth answers.
\nEvery time I change the architecture, I run the eval and look at three metrics:
\n| Metric | \nQuestion it answers | \n
|---|---|
| Faithfulness | \nIs the answer actually derived from the context? | \n
| Answer relevance | \nDoes it answer the user's question? | \n
| Context precision | \nAre the retrieved chunks actually useful? | \n
Building production RAG is a game of millimeters. It's about cleaning your data, pinning your models and measuring what's happening under the hood.
\n\n- Chunk by document structure, not by raw token count.\n- Retrieve broad, then rerank for precision.\n- Pin the embedding model and re-index when it changes.\n- Keep context small: 3 to 5 good chunks inside a latency budget.\n- Combine vector and keyword search, and filter by metadata first.\n- Score every change against a golden set instead of trusting a vibe check.\n\nIf your RAG pipeline answers well in the demo but drifts in production, here's how I help teams fix retrieval and ship it. You can also reach out if you're hitting a wall with your architecture.
\nWhat's the one thing currently keeping you from hitting that \"deploy\" button?
\nIf your checkout request waits on a PDF, an email and three third-party APIs, more RAM won't save it. Hand that work to a message broker and answer the user right away.
\n\n- Synchronous controllers tie your response time to the slowest secondary task, so one slow API can turn into a 504 for the user.\n- RabbitMQ sits in the middle: producers publish, exchanges route, queues hold and consumers do the work at their own pace.\n- Move one slow task at a time to a queue, scale workers on their own, and make every worker idempotent.\n\nEvery time a user hits \"checkout,\" your server generates a PDF, sends a welcome email, updates inventory and calls three third-party APIs.
\nIf any of those services takes too long to answer, the user sees a 504 gateway timeout.
\nIt starts as a small delay, then becomes a bottleneck. Soon you're adding RAM to a problem bigger hardware can't fix.
\nWhen everything is synchronous, one failure in a secondary task breaks the whole user experience. I've watched dashboards turn red during a marketing spike because the database was busy with background reports and couldn't handle new signups.
\nThe fix is to change how your services talk to each other: decoupling, with a broker like RabbitMQ.
\nIn a typical web app we do too much inside the controller. A request arrives and we finish every related task before sending back \"200 OK.\" That's fine with ten users. For a growing SaaS, it breaks.
\nPicture a coffee shop where the person taking your order also grinds the beans, steams the milk and draws the logo on the cup before taking the next order. The line wraps around the block, because the cashier is tightly coupled to the barista's work.
\nTo scale, the cashier writes the order on a slip and hands it off, and is free for the next customer. That slip is your message. The counter where the slips go is your message broker.
\nRabbitMQ is an open-source message broker that sits between the parts of your system. It stores messages and also routes them by rules you define.
\nWith RabbitMQ in the middle, the web tier only has to say \"this task needs doing.\" That takes milliseconds. The user gets an instant confirmation while workers handle the heavy work when they're ready.
\n\nSudden bursts and noisy neighbors are among the hardest parts of scaling. If an enterprise client uploads a 100,000-row CSV, you don't want the login page to slow down for everyone else.
\nThat's exactly the work message queues for heavy document processing are built to absorb.
\nWith RabbitMQ, those 100,000 rows become 100,000 messages. Workers process them at a steady pace. If the queue grows, you add worker instances instead of scaling the whole application.
\nThat's horizontal scaling in its simplest form: workers are decoupled from the web server, so each scales on its own load.
\nLaravel's queue system ships drivers for the database, Amazon SQS, Redis and Beanstalkd. To talk to RabbitMQ directly, you add the community package vladimir-yuldashev/laravel-queue-rabbitmq.
Your web tier should take the order. Your workers should make the coffee.
\nYou don't have to rewrite the codebase overnight. Pick one slow, low-risk task, like the \"forgot password\" email or an image resize, and move only that to a queue. Then pick the next one.
\n// instead of sending the email directly\n// $emailService->sendWelcome($user);\n\n// dispatch a job to the RabbitMQ-backed queue\nProcessWelcomeEmail::dispatch($user)->onQueue('high-priority');\n\n// the user gets a response right away\nreturn response()->json([\n 'message' => 'welcome! check your inbox soon.',\n]);\nNow if your email provider (SendGrid, Mailgun or similar) has a bad day, your app stays up. The message waits in RabbitMQ until the service is back.
\n\nTime every step inside your slowest controller action. Anything the user doesn't need to see in the response, and that takes more than 100 ms, goes on your list of jobs to queue first.\n\nA broker-first mindset is the first step from a monolith to microservices. Once your monolith publishes events like order.placed or user.registered, other services can listen without you changing the original code.
The system also gets easier to observe and debug. The RabbitMQ management UI shows how many tasks are pending and how fast they're being processed, so you stop guessing why the server is slow.
\nAlready on GCP? The same patterns apply with Google Pub/Sub. Pick the broker that matches your hosting stack, not the trend cycle.
\nScaling a SaaS is about giving your work room to breathe, and a broker gives you that room. If you're pulling slow work off your request path, here's how I help teams design queues and workers that scale, or tell me about it.
\nWhat's the slowest part of your application right now, and could it run as a background job?
\nIf one slow service can kill your whole request, your services are glued together. Events let them work apart.
\n\n- A Pub/Sub topic broadcasts each event once, and every subscription gets its own copy to process at its own pace.\n- Choose push delivery for serverless endpoints and pull delivery when your workers need control over backpressure.\n- Add a dead-letter queue, idempotent consumers and a watch on the unacked message count before you call it production-ready.\n\nBuilding a modern web application usually starts simple. You get a request and you send a response.
\nAs your business grows, that simple flow starts to feel heavy. Maybe you need to send a welcome email, update a CRM and trigger a data warehouse sync all at once.
\nIf you do this synchronously, your users are stuck staring at a loading spinner. If one service fails, the whole request dies, and your system becomes a house of cards.
\nYour application logic is tangled like old headphones in a pocket. Every new feature adds more risk and more latency. You need a way to let your services talk without being glued together.
\nThe solution is event-driven architecture (EDA). In the Google Cloud world, the heart of that architecture is Google Pub/Sub.
\nIt is a globally distributed messaging service that decouples the services that produce events from the services that consume them. If you'd rather self-host the broker, the same patterns translate directly to scaling with RabbitMQ.
\nAt its core, Google Pub/Sub is built on two concepts: topics and subscriptions.
\nI like to think of a topic as a radio station. It broadcasts information out into the void. It doesn't care who is listening or what they do with the music.
\nA subscription represents a stream of messages from a specific topic. The publisher only needs to know about the topic. It doesn't need to know if there are ten consumers or zero.
\nThis is the heart of the shift from synchronous to asynchronous communication. When a user signs up on your site, you publish a UserSignedUp event to a topic.
Your main app is done and returns a success message to the user right away. Meanwhile, subscribers pick up that event and do their jobs in the background.
\n\nFan-out is one of the most useful patterns in Google Pub/Sub. You publish a single message to a topic, and multiple subscriptions each receive a copy.
\nImagine you run an e-commerce store. When an order is placed, three services need to act:
\nInstead of your checkout service calling three different APIs, it sends one message to an order-events topic. Three separate subscriptions (inventory, shipping, analytics) each get their own copy.
They process it at their own pace. If the analytics service is down for maintenance, the shipping label still gets created. The messages wait in the subscription until the service is back online.
\nCheckout publishes once and moves on. Every downstream service works on its own clock.
\nWhen you set up a subscription, you decide how you want to receive messages. Google Pub/Sub gives you two main options: pull and push.
\nPush subscriptions suit serverless architectures. Google Cloud sends the message to a webhook URL you provide.
\nThis fits cloud infrastructure built on Cloud Run or Cloud Functions. It scales automatically and you only pay for what you use. You do have to make sure your endpoint can handle sudden spikes in traffic.
\nWith pull, your consumer service asks Google Pub/Sub for messages when it is ready. This gives you much more control over backpressure: if your worker is busy, it doesn't ask for more work.
\nPull is the usual choice for long-running services or for tools like Laravel's queue workers. It holds up better for heavy processing where you want to fine-tune concurrency, which is the model I use for message queues in document processing.
\n| Delivery | \nHow messages arrive | \nGood fit | \nWatch out for | \n
|---|---|---|---|
| Push | \nPub/Sub calls your webhook URL | \nCloud Run, Cloud Functions, serverless | \nSudden traffic spikes on the URL | \n
| Pull | \nYour worker asks when it is ready | \nLong-running workers, Laravel queue workers | \nYou manage the workers yourself | \n
In a distributed system, things will fail. A database might time out or an external API might be down. If a message can't be processed, you don't want to lose it.
\nThis is where dead-letter queues (DLQs) come in.
\nA DLQ is another topic where Google Pub/Sub sends messages that failed to be acknowledged after a set number of attempts. Instead of retrying forever and clogging your main pipeline, the \"poison\" message is moved aside.
\nI always recommend a DLQ for every important subscription. It acts as a safety net.
\nYou can build a small dashboard or script to inspect the failed messages, fix the underlying issue and replay them. For Laravel jobs, my PHP unserialize tool turns a failed payload back into readable data. That prevents data loss and keeps your system moving.
\nFor the PHP and Laravel ecosystem, integrating Google Pub/Sub is smooth. Laravel ships with solid queue support for Redis and SQS, and the google/cloud-pubsub client lets you tap into GCP's global scale.
This kind of decoupling is what makes the move from a monolith to microservices manageable instead of terrifying.
\nYou can treat Google Pub/Sub as a custom queue driver. Here is how you might publish a message in a typical service class:
\nuse Google\\Cloud\\PubSub\\PubSubClient;\n\n$pubsub = new PubSubClient([\n 'projectId' => 'your-gcp-project-id',\n]);\n\n$topic = $pubsub->topic('user-events');\n\n$topic->publish([\n 'data' => json_encode([\n 'user_id' => 123,\n 'action' => 'signup',\n ]),\n 'attributes' => [\n 'event_type' => 'UserSignedUp',\n 'priority' => 'high',\n ],\n]);\nAttributes let you filter messages at the subscription level. A subscriber can choose to only receive messages where event_type is UserSignedUp.
That saves compute and outbound traffic, because your worker never sees the messages it doesn't care about.
\nSet up monitoring from day one. Google Cloud integrates Cloud Monitoring with Google Pub/Sub.
\nKeep a close eye on your \"unacked message count.\" If this number keeps climbing, your subscribers can't keep up with the producers.
\nGoogle Pub/Sub is cheap at low volumes, but at millions of messages those bytes add up. Use batching on the publisher side to reduce the number of API calls.
\nAlso watch message retention. The default is seven days, and you can set anywhere from 10 minutes up to 31 days. Keeping unacknowledged messages for more than 24 hours adds storage cost, so shorten the window if you don't need a long replay.
\n\nSet an alert on the unacked message count for each subscription, and check whether your retention period is longer than you actually need.\n\n\n- **Start by finding the \"facts\"** in your system (for example `OrderPlaced`) and turn them into events.\n- **Use the fan-out pattern** to keep your services decoupled and focused on one task.\n- **Always add a dead-letter queue** so failures don't block the main pipeline.\n- **Use message attributes for filtering** at the subscription level.\n- **Make your consumers idempotent.** If they receive the same message twice, it doesn't cause errors or double charges.\n\nThis takes more planning up front, and I use it as the backbone of many systems I build because the payoff in stability is worth it. If you want help moving a synchronous flow to events, here's how I help teams design event-driven systems on Pub/Sub and Laravel, or let me know what's stopping you from making the switch.
\nAre you still using synchronous API calls for everything, or have you started moving toward an event-driven flow?
\nYou don't have to choose between overpaying for simplicity and overworking for control. Coolify and Docker give you the managed-platform experience on a server you own.
\n\n- Managed platforms charge per seat and per usage, and they lock your databases, env vars and build pipelines into their ecosystem.\n- Coolify is an open source, self-hosted control plane (think Heroku or Vercel) built on Docker: git-push deploys, one-click databases and automatic SSL.\n- A production setup adds your own Dockerfile (or Nixpacks), off-box database backups and health checks for zero-downtime deploys.\n\nShipping a SaaS is hard enough without the constant worry of a \"surprise\" bill from your hosting provider.
\nI have seen developers start on a managed platform and find that as soon as they add a second team member or cross a bandwidth threshold, their costs jump.
\nYou are stuck choosing between paying a \"convenience tax\" that eats your margins and spending your weekend fighting Nginx configurations and manual SSH commands. It feels like you are either overpaying for simplicity or overworking for control.
\nIt gets worse when you realize most managed platforms are wrappers around the same open source tools you could run yourself. You are paying for a pretty dashboard and an easy git-push flow.
\nWhen you try to leave, you find yourself deep in vendor lock-in. Your databases, environment variables and build pipelines are all tied to a proprietary ecosystem. If the platform goes down or changes its pricing, your business is at the mercy of their support team.
\nCoolify combined with Docker gives you the same developer experience as high-end managed platforms, but on your own infrastructure. You get the one-click deploy feel and a clean dashboard while keeping full control over your servers.
\nIf you want the business case first, I make it in why I ditch expensive cloud providers for self-hosted SaaS. This guide is the hands-on build.
\nEarly on, a $20-per-month plan feels reasonable. But as you grow, those costs multiply instead of adding up.
\nMany platforms now charge per seat. With a team of five engineers, you might pay a hundred dollars a month before you deploy a single line of code.
\nThen come the usage fees. Bandwidth, image optimization and serverless function execution costs are often opaque and hard to predict.
\nI have seen stacks move from managed services to a self-hosted Coolify setup and cut the monthly infrastructure bill by about ninety percent, with the same performance and reliability.
\nWhen you own the server, you own the resources. There are no \"hidden\" charges for extra build minutes or database connections.
\n\nAdd up last month's hosting invoices and split them into seats, usage fees and base plans. The seat and usage lines are what a self-hosted setup removes.\n\nThink of Coolify as an open source, self-hosted version of Heroku or Vercel. It is a control plane that sits on your server and manages everything for you.
\nIt handles your deployments, your reverse proxies, your SSL certificates and your databases. It turns a raw Linux VPS into a full hosting platform.
\nCoolify is built on Docker, so every application you deploy is containerized. Your environment stays consistent across development, staging and production.
\nNo more \"it works on my machine\" excuses. If it runs in a container on your laptop, it will run the same way on your Coolify server.
\nYou keep the git-push workflow. You drop the per-seat bill.
\nDocker is the quiet engine that makes this whole workflow possible. Instead of installing PHP, Node.js or Python directly on your server, you package them into a container image.
\nThat gives SaaS founders several key benefits:
\nI use Docker on every build I ship. It makes handoffs easy: whoever runs the app next doesn't need to worry about the underlying server configuration, only a Docker-compatible environment.
\nCoolify deliberately stays at the single-host Docker layer rather than orchestrating clusters. If you're weighing that trade-off, see Docker vs Kubernetes: containers vs orchestration.
\nGetting started is simple. Coolify's documented minimum is 2 CPU cores, 2 GB of RAM and 10 GB of free disk, plus extra space for Docker images, volumes and backups. I'd budget 4 GB of RAM if you plan to run more than a couple of apps alongside it.
\nI usually recommend Ubuntu for the operating system. Once you have your server, run the single installation command from the Coolify documentation.
\n# the only command you need to bootstrap Coolify\ncurl -fsSL https://cdn.coollabs.io/coolify/install.sh | sudo bash\nThe script installs Docker, sets up the Traefik reverse proxy and launches the Coolify dashboard. Within minutes, you can log in to your own private hosting panel.
\nFrom there, connect your GitHub or GitLab account. Coolify listens for webhooks and triggers a new build whenever you push code to your main branch.
\nIt feels magical. You get the same feedback loop as the big-name platforms: build logs in real time and a preview URL for your feature branches. And you do it all on a cheap VPS.
\nManaging databases is one of the hardest parts of self-hosting. Nobody wants to hand-configure Postgres clusters or worry about backing up Redis instances. Coolify solves this with \"one-click\" services.
\nYou can spin up a Postgres, MySQL, MongoDB or Redis instance in seconds. Coolify generates secure credentials and gives you the connection strings.
\nIt also handles persistent volumes, so your data stays safe when your container restarts or you update the image.
\nPersistent volumes are not a backup strategy, though. Schedule automated dumps and keep them off-box, because replication and backup solve different failure modes.
\nFor a SaaS, I usually recommend a dedicated server for your databases once traffic climbs. Coolify supports multi-server setups: one server acts as your control plane and other servers act as worker nodes where your apps and databases live.
\n(I cover that scaling pattern in more depth in scaling SaaS with advanced Coolify deployment strategies.)
\n\nFor each database you run in Coolify, write down where its last backup lives and when it was taken. If the answer is \"on the same server,\" schedule an off-box dump today.\n\nSecurity shouldn't be an afterthought. Setting up SSL with Let's Encrypt used to mean cron jobs and manual certificates. With Coolify and Traefik, it is fully automated.
\nWhen you point a domain to your server and add it to your app configuration, Coolify requests and installs an SSL certificate. It also handles renewal, so your SaaS is always served over HTTPS without you touching a terminal.
\nCoolify also helps you manage environment variables securely. You don't need to hardcode secrets in your git repository. Define them in the dashboard and they are injected into your containers at runtime.
\nThis is a standard best practice that many developers skip when they are in a rush. Need a strong value for one of those secrets? My secret generator makes one in your browser.
\nIf you want to do this the \"pro\" way, here is how I structure my deployments.
\nCoolify can detect many frameworks like Laravel or Node.js, but I always recommend writing your own Dockerfile. It gives you total control over the build.
\nUse multi-stage builds to shrink your image. That makes deployments faster and saves disk space.
\nIf you don't want to write a Dockerfile, Coolify supports Nixpacks. It is a tool built by Railway that inspects your code and builds an optimized container image, with detection for 20-plus languages.
\nRailway itself has since moved to a successor called Railpack, but Nixpacks remains a solid zero-config option inside Coolify. Coolify also offers Railpack as a beta build pack for Git-based apps.
\nNever deploy without a health check. You want your app to actually respond before the reverse proxy sends traffic to it.
\nCoolify lets you define a health check endpoint. If the check fails, the old container keeps running and the new one isn't promoted. This is the foundation of zero-downtime deployments.
\n\nSelf-hosting isn't for everyone. If you are a solo developer with no interest in learning how a server works, paying the \"convenience tax\" might be worth it.
\nYour time is valuable. If a managed platform saves you five hours of frustration a month, it might pay for itself. My self-hosting cost calculator puts those hours next to the server bill.
\nIf you are building a real business, though, you need to understand your stack. Owning your infrastructure is about more than saving money. It is about autonomy: no matter what happens to a specific provider, you can move your business elsewhere quickly.
\nWhether the app is an e-commerce engine on Shopify or a custom Laravel application, the goal is the same: high performance and long-term stability.
\n\nAdd a `/health` route to your app that checks the database connection, set it as the health check in Coolify, then deploy a deliberately broken build to staging and confirm the old container keeps serving.\n\n\n- **Start with a clean VPS** and install Coolify using the official script.\n- **Containerize your application** with a Dockerfile for full control, or use Nixpacks for zero config.\n- **Use one-click services** for your databases and keep automated backups off the server.\n- **Set up your domain** and let Coolify handle the SSL certificates.\n- **Add health checks** so updates never take the app down.\n\nWhen you stop worrying about the \"how\" of deployment, you can spend more time on the \"what\" of your product. If you want to move your SaaS onto Coolify without losing a weekend to it, here's how I help teams set up self-hosted deployments, or tell me about it directly.
\nWhat is the one thing stopping you from moving your SaaS to a self-hosted setup today?
\nEvery new data source used to mean new glue code for your agent. MCP replaces that glue with one protocol any client can speak.
\n\n- MCP is an open standard that puts a clean boundary between the AI client and the servers that expose your data and actions.\n- It has three primitives: tools (model-controlled actions), resources (application-controlled data) and prompts (user-controlled templates).\n- The server is the gatekeeper, so you can limit an agent to specific tables or read-only access, and swap models without rebuilding integrations.\n\nBuilding context-aware agents is harder than it should be. AI tooling is a mess of fragmented integrations.
\nEvery time I want to give an LLM access to a new data source or tool, I end up writing custom glue code that breaks the moment an API version changes. We force very capable models to peer through a keyhole when they should have a wide-open window into our data.
\nThis fragmentation creates a lot of technical debt. You spend most of your time on plumbing and only a sliver on the agent's actual intelligence.
\nWithout a shared way to pass context, the model often hallucinates because it lacks real-time data. It gets stuck on \"I don't have access to that\" or, worse, \"I'll guess what that data looks like,\" which leads to unreliable output and a poor user experience.
\nThe Model Context Protocol (MCP) changes this. It is an open standard created by Anthropic and donated in December 2025 to the Agentic AI Foundation, a Linux Foundation fund co-founded by Anthropic, Block and OpenAI.
\nIt lets me build agents that connect to any data source through one shared language. If you want the bigger picture, I broke down API vs MCP separately.
\nA recurring hurdle in custom web applications has always been data silos. When I work on complex technical challenges, the goal is usually to make data actionable.
\nTraditional tool use means wiring every schema and function call into each app by hand. With MCP, you define them once in a server, and any client can discover them.
\nMCP acts as a bridge. It draws a clear line between the AI application (the client) and the data sources (the servers).
\nThat separation means I can swap the underlying model without rebuilding the data integration layer. If I move from Claude to another model that supports MCP, the tools and resources stay the same.
\nIt also eases the context window problem. Instead of stuffing a huge document into the prompt, I expose it as an MCP resource.
\nThe model only pulls what it needs, when it needs it. That is more efficient and cheaper, and it gives you agents that know their environment without drowning in it.
\n\nTo build with MCP, start with its three core primitives. They are the building blocks of any context-aware system.
\n| Primitive | \nControlled by | \nWhat it is | \nExample | \n
|---|---|---|---|
| Tools | \nThe model | \nActions that change the world | \nWrite a file, POST to a Shopify API | \n
| Resources | \nThe app | \nRead-only data for grounding | \nProduct docs for a support agent | \n
| Prompts | \nThe user | \nTemplates that guide tasks | \nA standard \"summarize this ticket\" flow | \n
Tools are model-controlled actions. Giving an agent a tool gives it the ability to change the world, like writing a file to disk or making a POST request to a Shopify API. The model decides when to call the tool based on the user's intent.
\nResources are application-controlled data. Think of them as read-only files or database entries the agent can inspect. They give the agent grounding: for a support agent, the product documentation would be a resource it can search and read to answer accurately.
\nPrompts are user-controlled templates that guide the interaction. MCP prompts let me standardize how users work with the agent across platforms, so the model interprets tasks consistently.
\n\nList what your agent needs and sort each item into tool, resource or prompt. Anything read-only that you currently paste into the system prompt is a candidate resource.\n\nI prefer TypeScript for MCP servers because of the mature official SDK. The protocol itself is language-agnostic, with official SDKs for Python, Java, Go, C#, Kotlin and more.
\nVersion 2 of the TypeScript SDK is the stable line, released alongside the 2026-07-28 spec. It ships as split packages (@modelcontextprotocol/server and @modelcontextprotocol/client), and v1 keeps getting fixes for at least six months.
Here is a basic server that exposes a weather tool with the high-level McpServer API. If you have a sample tool response, my JSON to TypeScript and Zod converter gives you the types.
import { McpServer } from \"@modelcontextprotocol/server\";\nimport { StdioServerTransport } from \"@modelcontextprotocol/server/stdio\";\nimport * as z from \"zod/v4\";\n\nconst server = new McpServer({ name: \"weather-server\", version: \"1.0.0\" });\n\nserver.registerTool(\n \"get_weather\",\n {\n description: \"Get the current weather for a location\",\n inputSchema: z.object({ location: z.string() }),\n },\n async ({ location }) => {\n // logic to fetch weather from an API goes here\n return {\n content: [{ type: \"text\", text: `It is sunny in ${location}` }],\n };\n },\n);\n\nasync function main() {\n const transport = new StdioServerTransport();\n await server.connect(transport);\n}\n\nmain();\nThe snippet shows how simple the protocol is. I define the tool and how to handle the call, and the MCP client handles the rest.
\nThis modular approach is what I look for when managing cloud infrastructure or complex backend systems. It is clean and it scales.
\n\nCopy the server above, replace the weather logic with one read-only call to a system you use daily, and connect it to an MCP client to watch the model call it.\n\nSecurity is a major concern when you give an AI agent access to your data. I have seen many setups where API keys are hardcoded or permissions are far too broad.
\nMCP helps with a client-server design where the server controls exactly what is exposed.
\nYou can add fine-grained access control at the server level. For example, an MCP server connected to a database can be limited to specific tables or read-only queries.
\nThat level of control is essential for enterprise applications.
\nDecide what the agent can touch on the server side, and the prompt can't talk it past that line.
\nThe ecosystem has matured fast. MCP has gone from an Anthropic experiment to an industry standard, and the Agentic AI Foundation has support from Google, Microsoft, AWS, Cloudflare and Bloomberg.
\nOn the tooling side, clients like Zed, Cursor, VS Code and Claude Code ship MCP support, so AI assistants write better code with real context. For a hands-on walkthrough, see how I connect my own dev tools through Claude MCP.
\nIf you want to dive into MCP, I recommend these steps:
\nMCP marks a shift away from \"black box\" agents toward transparent, context-aware assistants you can reason about in production. If you're building agents that need real access to your data, here's how I help teams design and ship MCP servers, or drop me a line to swap notes on real-world MCP server design.
\nHow are you planning to use MCP in your next project?
\nCoding used to mean translating intent into syntax, line by line. Agentic workflows flip that: you state the intent, and a loop of tools does the typing.
\n\n- Chat-style AI assistants left you as the manual bridge between the model and your system: copy, paste, test, repeat.\n- Agentic loops (intent, reasoning, action, observation, correction) let the model reach your tools directly, mostly through MCP.\n- The human job becomes architecture and review: small scoped MCP servers, stateful agent tools and a sign-off gate before any action runs.\n\nI've spent the last decade building Laravel applications, managing Docker clusters and fine-tuning Shopify stores.
\nFor most of that time, \"coding\" meant one thing: translating a business requirement into syntax a machine could execute. It was a manual, linear process of writing line by line, debugging stack traces and managing state.
\nRecently, the ground has shifted. We're moving away from \"writing code\" and into \"orchestrating intent.\"
\nThis transition is often playfully called vibe coding. The meme hides a real architectural change: a move from sequential instructions to agentic workflows powered by protocols like MCP (Model Context Protocol).
\nIf you are new to the idea, my intro to vibe coding covers the philosophy and developer-experience side before this post dives into the architecture.
\nThe traditional development lifecycle is full of invisible friction. You have an idea (the \"vibe\"), you break it into tasks, and then you spend 80% of your time fighting syntax, configuration and boilerplate.
\nIn a standard Laravel environment, even a simple feature like an automated reporting tool needs routes, controllers, service classes and database migrations.
\nYou are the compiler. You are the architect. You are the labor.
\nOur cognitive load gets consumed by the \"how\" rather than the \"what.\" We get stuck on PHP version compatibility or Docker networking issues and lose sight of the actual user value.
\nThis manual micromanagement doesn't scale with the demands of modern business.
\nWhen AI first arrived as basic autocomplete, it felt like a shortcut. It wasn't a solution.
\nWe ended up with what I call \"the Copilot paradox\": the AI suggests code, but you still copy-paste it, test it, find the error and feed it back.
\nThe AI is a \"black box\" that doesn't know your system. It doesn't know your database schema, your MCP servers or your deployment status on Coolify.
\nYou are still the manual bridge between the AI's logic and your local environment.
\nThis creates a new kind of fatigue. Instead of writing code, you become a high-speed code reviewer, constantly switching between your editor and a chat window.
\nThat isn't \"vibe coding.\" It's accelerated manual labor.
\n\nFor one day, count how many times you copy code out of a chat window and paste an error back in. Each round trip is a step an agent with tool access could do on its own.\n\nTrue vibe coding shifts your role to high-level system architect.
\nThat becomes possible through agentic workflows: systems that execute tasks in loops instead of only completing text. That jump from a static model to an autonomous actor is exactly the LLM vs AI agent distinction.
\nThe breakthrough here is the Model Context Protocol (MCP), created by Anthropic and donated in December 2025 to the Agentic AI Foundation, a Linux Foundation fund, so it is now vendor-neutral.
\nMCP acts as the \"USB-C port\" for AI applications. Instead of you handing the AI context, the AI uses an MCP client to talk directly to your tools: your PostgreSQL database, your Slack channels or your GitHub repositories.
\n\nIn a traditional chain, you give a prompt and get a result. In an agentic loop, the architecture looks like this:
\nI call this \"intent-based engineering.\" You aren't writing the migration. You are approving the architectural decision.
\nNobody lands here on day one. The jump from chat prompts to loops is a progression I broke into levels of an AI coding workflow, and MCP sits near the top of it.
\nStop typing the migration. Start approving the decision behind it.
\nAs an engineer who values quality, I don't let the \"vibe\" take over without guardrails.
\nThe loop only stays safe if something verifies what comes out of it. That is the whole argument in trust is not a QA strategy. Here is how I structure my agentic stack using Laravel and AI today.
\nI build small, dedicated MCP servers that expose only the tools the AI needs. This keeps the context window clean and the security tight.
\nI go deeper on this in Claude MCP dev tools and on designing context-aware agents with MCP.
\n// Conceptual MCP tool definition in a PHP environment\npublic function defineTools(): array\n{\n return [\n 'get_database_schema' => [\n 'description' => 'Retrieves the structure of the Laravel application tables.',\n 'parameters' => [],\n ],\n 'run_artisan_command' => [\n 'description' => 'Executes an artisan command safely.',\n 'parameters' => ['command' => 'string'],\n ],\n ];\n}\nInstead of one-off chats, I use tools like Cursor, Claude Code or Devin Desktop (the IDE formerly called Windsurf) that keep a stateful connection to my local file system.
\nThe agent can \"see\" the impact of its changes in real time, just like a human developer would.
\nThe most important part of the architecture is the review gate. Even with agentic loops, the human architect must sign off on the \"plan\" before the \"action\" phase.
\nThis keeps the PHP logic on clean architecture principles rather than just \"making it work.\"
\nWhat that gate should actually check is a separate question. I argue for dropping line-by-line reading in favor of specs and automated verification, and I cover approval gates for writes in HITL gates for agent mutations.
\nIf you're a founder or a CTO, the lesson is simple: stop hiring for syntax and start hiring for system design. The technical barrier is collapsing, but the architectural stakes are higher than ever.
\nI now build agent-ready systems, whether it's a complex Shopify integration or a custom SaaS, so the architecture is ready for agents from day one.
\n\nPick one recurring engineering chore (a weekly report, a dependency bump) and write it as an intent plus a done condition. If you can't state the done condition, an agent can't close the loop either.\n\n\n- Chat assistants keep you as the copy-paste bridge. Agentic loops remove that step.\n- MCP is the standard connector between agents and your tools.\n- Keep MCP servers small and scoped to the task.\n- Put a human sign-off between the plan and the action.\n- The code might be generated, but the vision is still yours.\n\nIf you want to move your team from chat prompts to scoped agent loops on Laravel and MCP, here's how I help teams design and ship that stack, or get in touch directly.
\nAre you ready to stop writing code and start orchestrating your intent?
\nWhen a production RAG system gives wrong answers, check retrieval before you blame the model. Fix how you find and rank context first.
\n\n- Vector search finds related ideas but misses exact identifiers like SKUs, error codes and project names, so pure-vector RAG drifts in production.\n- Hybrid search (vectors plus BM25), merged with Reciprocal Rank Fusion, keeps both meaning and exact matches.\n- A cross-encoder re-ranker and regular evaluation (faithfulness, response relevancy, context precision and recall) turn a good demo into a dependable system.\n\nYou built a RAG (Retrieval-Augmented Generation) demo. On your machine, with a handful of PDFs, it looked convincing and the answers felt coherent.
\nThen you pushed it to production.
\nUsers report that the LLM is \"hallucinating\" when the real issue is retrieval. Obvious answers go missing even though they're in the docs. Irrelevant chunks surface because they're semantically close but useless.
\nIf your RAG system feels unreliable, look at retrieval design first. Most production RAG systems fail because they lean too hard on vector search and mistake a strong demo for a dependable system.
\nFor the broader catalogue of ways this goes wrong, see 7 mistakes you're making with your production RAG stack. One pattern keeps showing up in my own work: a demo proves possibility, but production demands precision.
\nIn a demo, semantic resemblance often feels good enough. In production, \"good enough\" is where failures begin.
\nEmbeddings are useful. They map text into vectors so you can retrieve by meaning instead of exact wording. Semantic similarity still differs from retrieval accuracy.
\nVector search is good at finding related concepts and weak at specificity.
\nIf a user searches for \"Project-X-99 deployment logs,\" vector search might return \"Project-A deployment\" or \"logging best practices,\" because they're semantically close. It can miss the exact identifier \"X-99,\" which carries little semantic weight in a high-dimensional space.
\nOnce retrieval drifts, the LLM inherits the drift. It can't reason its way out of missing or irrelevant context. You pay for tokens that produce confident but unhelpful answers, and users lose trust for a reason that sits one layer below the model.
\nCombine semantic retrieval with lexical precision. That combination is hybrid search.
\nBM25 (Best Matching 25, from the Okapi probabilistic retrieval framework) is the standard lexical ranking method behind classic search. It ignores meaning and rewards exact terms by how important they are in a document and across the collection, while accounting for document length and diminishing returns from repeated terms.
\nUser questions are rarely pure meaning or pure keyword. They're usually a mix. If you're still deciding how to structure retrieval in the first place, vector search vs graph search is a useful companion read.
\n| Query type | \nVector search | \nBM25 | \nHybrid | \n
|---|---|---|---|
| \"How do I cancel my plan?\" | \nStrong | \nDecent | \nStrong | \n
| \"Error E-4012 on checkout\" | \nOften misses the code | \nStrong | \nStrong | \n
| \"SKU HB-220 return policy\" | \nOften misses the SKU | \nStrong on the SKU | \nStrong | \n
| \"Ways to speed up the store\" | \nStrong | \nWeak on synonyms | \nStrong | \n
Embeddings capture what a question means. Keywords catch exactly what it names.
\n\nPull 20 real queries from your logs that contain an ID, SKU or error code. Run them through your current retriever and count how many return the exact matching document in the top 5. That number is your case for adding BM25.\n\nRunning two retrieval strategies creates a new question: how do you combine them?
\nA practical answer is Reciprocal Rank Fusion (RRF), introduced by Cormack, Clarke and Büttcher in 2009. It's simple and reliable, and it doesn't pretend that scores from different retrievers are directly comparable.
\nscore = 1 / (k + rank), where rank is the 1-based position in each result list (so the code below adds 1 to a 0-based index). The constant k (60 in the original paper) keeps top-ranked items from dominating too much.Here's the minimal PHP version I drop into a Laravel service:
\nfunction reciprocalRankFusion(array $resultSets, int $k = 60): array\n{\n $scores = [];\n\n foreach ($resultSets as $results) {\n foreach ($results as $rank => $docId) {\n $scores[$docId] = ($scores[$docId] ?? 0.0) + 1 / ($rank + 1 + $k);\n }\n }\n\n arsort($scores);\n\n return $scores;\n}\nA document that is semantically relevant and lexically precise now rises to the top for a reason.
\nHybrid search is a strong foundation, but production RAG usually needs one more layer of judgment: a re-ranker.
\nA re-ranker like Cohere Rerank or BGE-Reranker is a cross-encoder that reads the query and the document together. Relevance is relational: the question is whether this document answers this question, beyond what it happens to contain.
\nThis cuts context stuffing and improves what reaches the model. It's one of the clearest differences between a demo and a production system that behaves consistently. The extra hop adds latency, which is where a Redis and semantic caching layer earns its keep.
\n\nA RAG system can impress in a demo and still be weak in production. Once real users, messy documents and ambiguous queries arrive, weak retrieval turns the LLM into expensive guesswork.
\nBuilding a RAG demo is easy. Building reliable RAG takes a real understanding of retrieval, ranking and context design. If you're still choosing your foundation, picking the right RAG stack walks through the vector database trade-offs. If your production RAG keeps missing, here's how I help teams fix retrieval, ranking and evals, or get in touch with your war story.
\nWhere does your own system still behave like a demo when it should be behaving like production?
\nA big-bang rewrite can kill the business while the engineers play with new toys. Strangle the monolith one route at a time instead.
\n\n- The real problem is coupling, and jumping straight to microservices just moves those dependencies onto the network.\n- The strangler fig pattern puts a routing layer in front, sends one endpoint at a time to a new service, and keeps the monolith as a safety net.\n- Containerize the monolith first, protect new services with an anti-corruption layer, and split only when the monolith hurts more than the complexity tax.\n\nYour monolith is a ticking time bomb, and every feature you add brings the explosion closer.
\nI have seen it happen a dozen times. A startup begins with a clean Laravel or Rails app. It is fast, easy and productive. Then the team grows and the code base swells.
\nSuddenly, a simple change to the checkout logic breaks the authentication system. Deployments that used to take five minutes now take forty.
\nYou are no longer scaling your business. You are managing technical debt.
\nThis is when most developers start dreaming of microservices: every service isolated, every deployment instant. The reality is often a nightmare.
\nDo it wrong and you end up with a distributed monolith. You get all the complexity of networking with none of the benefits of isolation.
\nI skip the \"big bang\" rewrite and use the strangler fig pattern to move from monoliths to microservices without losing my mind or my job.
\nIf you are still deciding whether to split at all, start with monolith vs microservices. This post assumes you have made that call and want the migration playbook.
\nWhen a monolith gets too heavy, the first reaction is to scrap it. I have seen companies spend two years on a rewrite only to ship a product with half the features of the original.
\nThe monolith is a phase. The real problem is coupling.
\nWhen every part of your app knows too much about every other part, you cannot move. If you jump straight into microservices, you will likely port those dependencies into a network layer. Instead of a function call failing, you get a 500 error across a network socket.
\nI prefer a slower, deliberate approach focused on high-value extractions. Look for the parts of the app that hurt the most.
\nIs image processing slowing down the web server? Is the reporting engine locking up the database? Those are your first candidates for microservices.
\n\nList the three parts of your app that cause the most incidents or slow deploys. Pick the one with the fewest dependencies on the rest of the code as your first extraction.\n\nMartin Fowler named this approach after a tree that grows around another tree. It starts as a small vine and eventually replaces the host entirely.
\nIn software, you build new features as services while the old monolith keeps running.
\nThe process starts with an API gateway or a load balancer. I put a routing layer in front: Nginx or a Google Cloud HTTP(S) load balancer with Cloud Armor added for WAF and DDoS protection.
\nIf a request comes in for /api/v1/orders, the URL map sends it to the new service. Everything else goes to the old monolith. (If you're unsure which box does what here, see load balancer vs API gateway.)
This lets me test the new service in production with real traffic while the monolith acts as a safety net. If the new service fails, I flip the routing back.
\nI don't have to migrate everything at once. I can migrate one endpoint at a time.
\nMove one route, watch it under real traffic, and keep the old path one switch away.
\nYou cannot do microservices without Docker. I treat every service as a black box.
\nThe monolith might run on an old version of PHP while the new service is a lean Go binary or a modern Laravel instance. Docker makes that possible.
\nI start by containerizing the monolith. Even if it stays a monolith for another year, the container forces me to define its environment and makes the infrastructure reproducible.
\n# a simplified example of a service container\nFROM php:8.4-fpm\n\nWORKDIR /app\nCOPY . /app\n\nRUN apt-get update && apt-get install -y \\\n libpq-dev \\\n && docker-php-ext-install pdo_pgsql\n\nEXPOSE 9000\nCMD [\"php-fpm\"]\nOnce the monolith is containerized, I can deploy it to a platform like Google Kubernetes Engine (GKE).
\nThis is where microservices pay off. I can scale the order service to fifty instances during a sale while keeping the blog service at two.
\n\nWrite a Dockerfile for your monolith as it is today and get it running locally with the same PHP version as production. That alone removes a class of \"works on my machine\" bugs.\n\nThe hardest part of microservices is the data, not the code. Your monolith has a single database, and your microservices should each have their own. So how do they talk?
\nI use an anti-corruption layer (ACL). When I extract a service, I don't let it reach back into the monolith's database. That would be cheating.
\nInstead, I create an interface. If the new service needs user data, it asks the monolith through a private API or a message queue like Google Pub/Sub.
\nThe new service doesn't care about the legacy app's messy database schema. It only cares about the data it gets through the ACL.
\nWhen the user logic is migrated later, I update the ACL to point to the new user service. For the asynchronous side (letting services react to events instead of calling each other directly), I lean on event-driven Pub/Sub or a RabbitMQ-backed queue.
\nScaling a monolith usually means buying a bigger server. Scaling microservices means managing a fleet of small instances, so I rely on cloud-native tools to manage the complexity.
\nI use Terraform to manage my infrastructure as code. That keeps staging and production identical.
\nIf I need a new database for a service, I define it in code instead of clicking around a dashboard.
\nFor deployments I use tools like GitHub Actions or Coolify. Every service has its own pipeline.
\nIf I update the checkout service, I only deploy the checkout service, without worrying about the rest of the system.
\nI'd be lying if I said this was all sunshine and rainbows. Microservices come with a \"complexity tax\": distributed logging, service discovery and eventual consistency.
\nMy advice is to move to microservices only when the pain of the monolith is greater than the complexity tax. If your team is three people and your app is simple, stay in the monolith. You will move faster.
\nIf you are hitting walls every day and your developers are afraid to touch the code, it is time to start strangling.
\n\n- **Start with an API gateway** to handle routing.\n- **Containerize your monolith first** to normalize the environment.\n- **Use the strangler fig pattern** to migrate one domain at a time.\n- **Build an anti-corruption layer** to keep new services clean.\n- **Invest in infrastructure as code** early on.\n- **Only split when the monolith starts to hurt** your productivity.\n\nMigration takes time. I have spent months on a single extraction to get it right, because the goal is a system that grows with your business, not a count of services. If you're planning that first extraction, here's how I help teams move from a monolith to services without a rewrite, or tell me about your migration.
\nHave you ever tried a \"big bang\" rewrite only to regret it six months later?
\nThe scary moment in a SaaS launch is when you realize you don't know if customer A can see customer B's data.
\n\n- Pick your isolation model early: a shared database with `tenant_id` is cheap, a database per tenant gives the strongest isolation.\n- Never rely on memory to filter tenant data. A global scope trait adds the filter to every query and sets `tenant_id` on create.\n- Keep the web tier stateless and isolate cache and storage per tenant, so you can scale out with Docker and automate tenant setup.\n\nI still remember the panic of my first SaaS launch. I was watching the logs as the third customer signed up, and I realized I had no idea if customer A could see customer B's data.
\nThat realization is a rite of passage for every developer.
\nBuilding a software-as-a-service (SaaS) platform is a big technical challenge, and the main hurdle is almost always data isolation. Every tenant should feel like they have the whole application to themselves.
\nIf you get this wrong early, it will haunt you for a long time. Here is how I approach Laravel multi-tenancy to serve many customers from one codebase while keeping security and scalability intact.
\nMost developers start with a single database and add a user_id or team_id to every table. It works fine for the first ten users. Then the complexity grows.
You add more relationships. You forget a where clause in one obscure controller. Suddenly one customer can see another customer's private invoices.
That is a catastrophic failure. It kills trust and can end your business overnight.
\nPerformance also suffers. As your database grows to millions of rows, the queries get slower.
\nIndexing helps, but it does not solve the underlying problem of data bloat. You need a strategy that isolates data while keeping your infrastructure manageable.
\nWhen I build custom web solutions, I always start by choosing between two paths: a single database or a multi-database setup.
\nThe single database approach uses a shared schema, and every row has a tenant_id. It is cheap to run and easy to update.
I recommend it for startups where costs need to stay low. You can manage thousands of small tenants on a single server this way.
\nThe multi-database approach is the gold standard for enterprise. Every customer gets their own database.
\nThis gives the strongest isolation and makes per-tenant backups and replication straightforward. If one database crashes, the others stay online. I use this for tenants with strict compliance needs.
\n| Strategy | \nIsolation | \nCost and effort | \nGood fit | \n
|---|---|---|---|
| Single database | \nShared schema, tenant_id | \nLow, easy to update | \nStartups, thousands of small tenants | \n
| Multi-database | \nOne database per tenant | \nHigher, more to run | \nEnterprise, strict compliance | \n
I often use the stancl/tenancy package for Laravel. It is a very flexible tool and lets you switch between these strategies as your business grows.
Some of this work is on my work page.
\nThe secret to sleeping well at night is automation. I never rely on my memory to filter data. Instead I use Laravel global scopes.
\nA global scope automatically adds a filter to every query on a model. It makes sure Customer::all() only returns customers for the current tenant, behind the scenes, so you cannot forget it.
If tenant filtering depends on someone remembering it, a leak is only a matter of time.
\nI create a BelongsToTenant trait and apply it to every model that needs isolation. It handles the filtering and sets the tenant_id when a new record is created.
It is a simple solution that prevents most accidental data leaks. Here's what that trait looks like in practice:
\n<?php\n\nnamespace App\\Concerns;\n\nuse App\\Models\\Tenant;\nuse App\\Scopes\\TenantScope;\nuse Illuminate\\Database\\Eloquent\\Relations\\BelongsTo;\n\ntrait BelongsToTenant\n{\n protected static function bootBelongsToTenant(): void\n {\n static::addGlobalScope(new TenantScope());\n\n static::creating(function ($model): void {\n if (! $model->tenant_id && app()->bound('tenant')) {\n $model->tenant_id = app('tenant')->id;\n }\n });\n }\n\n public function tenant(): BelongsTo\n {\n return $this->belongsTo(Tenant::class);\n }\n}\nYou also need to isolate your cache and file storage. If two tenants upload a file named logo.png, they should not overwrite each other.
I configure Laravel to use tenant-specific prefixes for all storage paths. That gives every tenant a true \"sandbox.\"
\n\nWrite one test that creates two tenants, logs in as the first, and asserts that `Model::all()` returns zero rows from the second. Run it for every tenant-scoped model.\n\nYour architecture is only as good as the infrastructure it runs on. I usually deploy my Laravel apps with Docker on cloud providers like Google Cloud or AWS.
\nContainerization is key. It lets me scale the application horizontally: when traffic spikes, I spin up more web server instances.
\nThat only works because the web tier stays stateless. Tenant context is resolved per request, not stored on the box, so the infrastructure stays clean.
\nWhen a single tenant starts pushing serious request volume, I reach for Laravel Octane to handle high traffic before throwing more servers at the problem.
\nI use managed database services like Google Cloud SQL. They handle the heavy lifting of backups and scaling.
\nFor multi-database setups, automated scripts provision a new database whenever a customer signs up. With this \"infrastructure as code\" approach, adding a tenant is a button click away.
\n\nThe final piece is the user interface. Customers who own multiple businesses need a smooth way to move between accounts.
\nI build clean dashboards with Vue, with the frontend talking to the Laravel backend over a GraphQL API for a reactive feel. The tenant switcher is always within reach and shows exactly which context you are working in.
\nI cache the tenant configuration in the frontend to avoid needless API calls. These small details separate a basic app from a polished one.
\nIf you are starting a new project today, here are the steps I recommend:
\nstancl/tenancy because of its flexibility. It handles subdomain routing and database switching out of the box.stancl/tenancy have built-in commands for this.Here's how I help teams architect and ship multi-tenant SaaS on Laravel, and you can always get in touch.
\nAre you building a SaaS or migrating your current app to a multi-tenant structure, and what is the main technical hurdle you are facing right now?
\n\"AI or traditional development?\" is the wrong question. The right one is how much speed, control and risk your business can afford.
\n\n- AI-assisted development wins on speed for known patterns: CRUD apps, admin panels, internal tools and first prototypes.\n- Traditional engineering wins on control when business rules are complex, regulated or expensive to get wrong.\n- Most teams should run a hybrid: AI drafts, humans own architecture, logic and the quality gates before merge.\n\nMost teams frame AI vs traditional development as a binary choice. That framing is the first mistake.
\nI see it a lot. Teams chase AI because it feels faster, or reject it because it feels messy.
\nBoth directions land in the same place: rushed systems with weak foundations, or polished systems that ship too late.
\nThe better move is to understand where each approach wins, where it breaks and where a hybrid gives you the best return. Both work. They solve different problems.
\nAI changes how I build software. Instead of writing every repetitive piece by hand, I use tools that understand context, generate scaffolding, speed up testing and remove a lot of drag from delivery.
\nI dig into the day-to-day of this in my post on vibe coding, where taste and intent matter more than raw typing speed.
\nThis is where AI shines.
\nFor standard workflows, admin panels, CRUD-heavy systems, internal tools and first-pass prototypes, AI can cut a serious amount of time. What used to take weeks can often shrink to days if the scope is clear and the review process is tight.
\nThat speed usually comes from a few places:
\nHow much of that speed you capture depends on where you sit on the AI coding workflow levels. Copy-pasting into a chat window and orchestrating agents against your real codebase are not the same tool.
\nI lean toward AI-heavy workflows when speed matters more than perfect customization on day one. That usually means:
\nIn those cases, AI acts like a power tool. It does not replace the builder. It makes the first cut much faster.
\n\nList the next five features on your roadmap and mark each one \"known pattern\" or \"custom logic.\" Start AI-heavy work only on the first group.\n\nAI is fast at common patterns. It is weaker at deep product nuance, strange business rules and systems that need careful long-term architecture.
\nIf you skip review, you can ship something that looks finished but behaves like a prototype wearing a production costume.
\nThe trade-offs differ between predictive models and generative ones, which I break down in machine learning vs generative AI.
\nSome studies of AI-accelerated codebases report more duplicated code, larger pull requests and faster technical-debt buildup. The upfront speed can quietly shift effort into later debugging and rework.
\nThat is the trap. The fix is treating AI output as a draft, not as truth, the same discipline I argue for in the architectural shift to agentic workflows.
\nAI output is a first draft. Review is what makes it a product.
\nTraditional development is slower, but it gives me tighter control over how the system is shaped.
\nThis is the path I trust most when the business rules are complex, the architecture matters or the cost of failure is high. Every part of the system is designed with intent instead of inferred from a prompt.
\nTraditional development is better when the software needs precision. That matters for:
\nOne underrated benefit of traditional development is predictability.
\nManual design, explicit code reviews, architecture decisions and planned testing give me a clearer picture of trade-offs. It is like building with blueprints instead of assembling furniture from a photo.
\nThat slower process often saves time later because fewer assumptions make it into production.
\nThe downside is obvious. Manual coding, reviews, debugging, refactoring and testing take time.
\nYou need stronger engineering talent, and the discipline to keep standards high when deadlines start squeezing the team. You pay for control in time and cost.
\n| Factor | \nAI-assisted development | \nTraditional development | \n
|---|---|---|
| Upfront speed | \nOften significantly faster | \nStandard industry timelines | \n
| Cost structure | \nCheaper to start, debt-prone later | \nHigher labor cost, steadier | \n
| Team requirements | \nStill needs senior review | \nRequires senior expertise | \n
| Customization level | \nStrong on common patterns | \nUnlimited customization | \n
| Quality assurance | \nFast drafts, human review required | \nManual review from the start | \n
| Risk management | \nVariable, depends on review rigor | \nPredictable risk factors | \n
| Scalability | \nScales output, not judgment | \nScales with team growth | \n
Choose AI when your bottleneck is delivery speed and the work is close to known patterns. That usually applies when:
\nChoose traditional development when the cost of being wrong is higher than the cost of being slower. That usually means:
\nThis is the option I recommend most often. The strongest teams do not treat this like a religion. They use AI where speed helps and switch to traditional engineering where judgment matters.
\n\nThe hybrid model works because it treats AI like a junior accelerator, not like an autopilot.
\n\nWrite a one-line rule for your repo: which folders or modules AI may draft freely, and which ones always need a senior design review first. Put it in your contributing guide.\n\nIf I were introducing AI into an existing team, I would start small:
\nIf the team stays mostly traditional, I would protect the basics:
\nIf the goal is balance, the workflow matters more than the tools:
\nThe teams that will win in 2026 are not the ones that blindly choose AI or reject it.
\nThey know where speed is enough, where control is non-negotiable and where a hybrid gives them speed without chaos.
\nUse AI to remove friction. Use traditional engineering to protect the parts that matter. Combine both when the business needs speed and reliability at the same time.
\n\n- Pick by risk, not by trend: known patterns go AI-first, high-stakes logic stays human-led.\n- AI speed comes with a debt bill if you skip review.\n- Traditional engineering buys control and predictability at the cost of time.\n- A hybrid with clear ownership and quality gates gives most teams the best return.\n\nYour development strategy should match your business goals, not the trend cycle. If you're deciding where AI belongs in your delivery process, here's how I help teams pick and ship the right mix, or reach out to talk it through.
\nIf you had to choose today, which matters more for your next product: speed, control or a hybrid path?
\nOne server running Coolify, your database, three apps and every build is fine for an MVP. For real customers, split the control plane, the builds and the data apart.
\n\n- Run Coolify on its own small server and deploy apps to separate app servers you add over SSH.\n- Rolling updates only protect users when a health check reports real readiness and the app shuts down gracefully.\n- Move builds to a dedicated build server with a registry, keep production data on a managed database, and deploy from CI with a deploy-only token.\n\nYou moved your apps off a messy manual VPS and into Coolify. Everything is in one place. Then traffic spikes.
\nHosting your production database, three web apps and a memory-heavy build on a single $10 DigitalOcean droplet is a recipe for trouble.
\nIt's fine for a side project. With paying customers, you need a plan for what happens when that one server hits 100% CPU, or when a deploy takes the whole site down for five minutes.
\nCoolify can handle larger workloads. You need to set it up the right way.
\nMulti-server nodes, rolling deployments, dedicated build servers and CI wiring. If you're just getting started, my walkthroughs on Coolify and Docker for SaaS hosting and running a fully self-hosted SaaS on Coolify cover the foundations.
\nThe most common mistake I see is keeping everything on one node. When a build starts, it eats CPU and RAM. The web app lags and the database gets starved.
\nRun the Coolify instance on one small server of its own. That's mission control. Then add separate app servers where your containers live.
\nIn Coolify, open Servers, add a server over SSH, and choose which server each resource deploys to. That gives you horizontal scalability: when one server fills up, add another and point the next app there.
\nIt's the same separation I use when building custom web applications, and it keeps one machine from taking everything down.
\nNothing hurts user trust faster than a \"502 Bad Gateway\" every time you push a small CSS fix. Many self-hosted setups kill the old container and then start the new one, and users fall into the gap.
\nFor Nixpacks, Railpack, Static, Dockerfile and Docker Image apps, Coolify starts the replacement container while the old one keeps running. It waits for the new container's health check, then stops and removes the old one. Docker Compose apps don't get this rolling sequence.
\nWithout a health check, Coolify treats a container that started as ready, even if the app inside is still booting.
\n/healthz that returns 200 only when the app can serve real requests./healthz and an interval of about 5 seconds. The check runs inside the container, so the image needs curl or wget.On the default Traefik proxy, unhealthy containers are removed from routing, so traffic stays on the old container until the new one passes.
\n\nCoolify's own docs are clear that rolling updates don't guarantee zero downtime. Both versions run side by side for a moment, so migrations, queues and sessions have to work with old and new code.
\n\nOpen your app's health check settings in Coolify. If the endpoint returns 200 before the database connection is up, change it to check the database and cache first, then redeploy and watch the logs.\n\nDocker builds are heavy. Compiling assets, installing npm packages and building images can push a server's usage through the roof. If that build runs on the server serving your customers, they feel it.
\nMark a high-CPU server as a build server in Coolify. On deploy, Coolify builds the image there, pushes it to a container registry, and the app server pulls the finished image.
\ndocker login on the build server, and on the app server too if the registry is private.Your production server never feels the build. It just pulls a ready-to-run image.
\nKeep your builds away from your traffic and your data away from your compute.
\nCoolify makes \"New Database\" one click, but running production Postgres or MySQL in a container on the same server as your app is risky.
\nFor production I usually recommend a managed database like AWS RDS or Google Cloud SQL, with backups and point-in-time recovery handled for you.
\nIn Coolify you pass the connection string as an environment variable. State stays separate from compute, the same stateless app discipline that makes horizontal scaling possible.
\nFor a professional workflow, code should move from GitHub to production without anyone clicking in the Coolify UI.
\nI prefer GitHub Actions. Coolify's GitHub App integration works well, but Actions let you run tests and linting first and trigger the deploy only when everything passes.
\nCreate an API token with only the deploy permission under Keys & Tokens > API Tokens, and store it as a CI secret. Copy the app's deploy webhook from Configuration > Webhooks. On a self-hosted instance, API access must be turned on in Settings > Advanced.
\nname: deploy to production\non:\n push:\n branches:\n - main\n\njobs:\n deploy:\n runs-on: ubuntu-latest\n steps:\n - name: trigger coolify webhook\n run: |\n curl --fail -X GET \"${{ secrets.COOLIFY_WEBHOOK_URL }}\" \\\n -H \"Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}\"\nPut your test job before this step and broken code never reaches your servers.
\nScaling well means a system that behaves predictably. Whether you're building a Shopify app or a Laravel SaaS, the principles are the same. If you're moving a growing app onto Coolify, here's how I help teams set up multi-server deploys and CI, or drop me a line.
\nHave you ever had a build crash your production server in the middle of the day?
\nHeadless can fix real problems, or hand your team a second system to babysit. Pick the simplest stack that removes your actual bottleneck.
\n\n- Liquid with Online Store 2.0 is still the right default for most growth-stage stores: fast to ship, cheap to maintain and native to Shopify.\n- Headless with Hydrogen and Oxygen can be faster, but only with disciplined frontend work, and it turns your storefront into a software product you own.\n- Decide with four questions: is the theme blocking revenue, can the team own a custom storefront, are integrations truly complex, and is slowness a code or an architecture problem?\n\nYour store is making money, but every change feels like surgery on a moving car.
\nTraffic is up, orders are up and so is the pressure. The theme layer starts to push back: feature requests turn into fragile hacks, app scripts pile up and mobile performance slips with every install.
\nThis is when brands start weighing Shopify Liquid vs. headless. The pitch sounds great: more freedom, better performance, cleaner frontend architecture. It's also where expensive mistakes happen.
\nThe real question is which stack fits your stage, your team and how you actually operate.
\nLiquid is the default for a reason. It's tightly integrated with Shopify, fast to ship and much easier to maintain than a custom headless frontend.
\nFor most growth-stage stores, Liquid gives the best speed to market. Online Store 2.0 made themes far more modular, so a solid theme setup goes a long way before it becomes a blocker.
\nA good Liquid stack is like a well-tuned delivery van. Not flashy, but it moves product reliably, takes updates quickly and doesn't need a pit crew every week.
\nThe problem shows up when business needs outgrow theme constraints. Maybe the storefront needs deeply custom interactions. Maybe product data has to come from an ERP, a PIM and a custom backend at once. Maybe merchandising logic is too complex for theme code to stay clean.
\nThat's usually when headless becomes a serious conversation.
\nPerformance is the most common reason teams look at headless, and milliseconds matter. On e-commerce storefronts, small delays add up to lower conversion, weaker ad efficiency and a worse mobile experience.
\nHydrogen is Shopify's headless framework: a React Router app preconfigured for Shopify's APIs. Oxygen is Shopify's serverless hosting for Hydrogen storefronts at the edge.
\nA well-built Hydrogen stack gives you finer control over rendering, data loading, caching and frontend execution. That opens the door to a lower LCP and a more responsive storefront.
\nIf the team over-fetches Storefront API data, hydrates too much JavaScript or ships a bloated component tree, the custom storefront can end up slower than a decent Liquid theme. That happens more often than people admit.
\nThere's also a middle path. Shopify storefront web components add dynamic, framework-light commerce to a page without a full headless rebuild.
\nA headless storefront is only as fast as the discipline of the team that builds it.
\nGoing headless splits commerce from presentation. Shopify still handles products, checkout and admin workflows. Your custom frontend handles the customer experience.
\nThat sounds clean on paper. In practice, you now own two systems instead of one.
\nYou manage hosting, deployments, caching, GraphQL queries, error handling, observability and integration behavior across services. Every app in the stack has to be checked for API compatibility. If an app only works by injecting snippets into a theme, that convenience is gone.
\nIn Liquid, many features feel plug and play. In headless, reviews, loyalty, search, subscriptions, personalization and analytics can all turn into custom integration work.
\nThe same thinking applies to newer surfaces like agentic commerce on Shopify, where the store has to expose clean, machine-readable APIs whatever stack you pick.
\nLiquid is renting a well-equipped shop. Headless is designing your own building. You get more freedom, and the plumbing, wiring and maintenance become your problem.
\n| Factor | \nLiquid (Online Store 2.0) | \nHeadless (Hydrogen + Oxygen) | \n
|---|---|---|
| Time to launch | \nOften weeks | \nMonths, with a bigger team | \n
| App compatibility | \nMost apps work out of the box | \nEach app needs an API integration | \n
| Performance ceiling | \nGood with clean theme code | \nHigher, if the frontend is disciplined | \n
| Ongoing ownership | \nTheme updates | \nA software product to run | \n
Time and budget usually decide this faster than architecture opinions.
\nA custom Liquid theme can often launch in weeks: faster testing, faster iteration and lower cost. If the business mainly needs merchandising flexibility, better UX and cleaner performance hygiene, Liquid usually gives the better return.
\nHeadless is a different kind of investment. The build takes longer, the team needs stronger frontend engineering and the integration surface is bigger. Maintenance doesn't stop after launch.
\nYou're paying for a redesign and also taking on a software product. Deployments, monitoring, API changes, dependency updates, caching strategy and developer ownership become part of normal operations. Shopify releases a new API version every quarter, and my Shopify API version calendar shows how long the one your storefront uses is supported.
\nFor many stores that trade isn't worth it yet. Freedom is expensive when the business doesn't fully need it.
\n\nI reduce this decision to four practical questions.
\nThe simplest stack that fixes the real bottleneck is usually the best one. Building a spaceship to cross the street is still a bad decision. You can see how this plays out across real Shopify builds I've shipped.
\nChoosing a stack is a long-term commitment. The right answer solves the current bottleneck without creating three new ones. If you're weighing a rebuild, here's how I help Shopify brands choose between Liquid, Hydrogen and the middle path, or drop me a line for a 30-minute architecture sanity check.
\nAre you hitting real technical limits in Shopify, or feeling the pull of a more customizable stack?
\nPick the vector database that fits your scale, your team and the database you already run.
\n\n- Default to pgvector if you already run Postgres: no new infrastructure, and your embeddings sit next to your relational data.\n- Pick Pinecone when you want zero ops at large scale, and Weaviate or Qdrant when hybrid search and filtering decide your quality.\n- Cheap wins matter more than the brand: load data before building the index, normalize vectors and keep metadata lean.\n\nYou built a chatbot. It works on your machine until you feed it 50,000 internal documents. Then it hallucinates, it slows down, and it pulls a report from three years ago when you asked for last week's.
\nNine times out of ten, the weak link is the RAG stack, and usually the vector database underneath it.
\nA Retrieval-Augmented Generation (RAG) system sounds like a weekend project. Past the \"hello world\" stage, you hit the database wall.
\nChoosing the wrong vector store early leads to high latency, rising cloud costs and a painful migration six months later when your data outgrows the setup.
\nI've seen teams freeze in front of the number of options in the AI ecosystem. Pick the right tool for your scale and your team.
\nHere is the 2026 landscape, so you can stop scrolling and start shipping.
\nAn LLM like Claude or GPT knows a lot but remembers nothing about your data. RAG gives it that memory, and your vector database is the librarian. If the librarian is slow or loses books, the model can't do its job.
\nIf you're still deciding whether RAG is the right tool at all, versus baking knowledge into the model, start with my breakdown of RAG vs fine-tuning.
\nIf you already run Postgres for your web applications, pgvector is usually the first stop. It's an extension that adds vector types and indexes to the database you already trust.
\nIt fits well under about 10 million vectors. You keep ACID guarantees and simple backups, and your relational data stays next to your embeddings. No new infra, no new security review.
\nPinecone is a managed, serverless vector database. You don't manage clusters or tune indexes. You send vectors and get results, and you pay for what you use.
\nIt's the usual pick for teams that want to grow toward a billion vectors without hiring a dedicated DevOps engineer.
\nIf your app needs to blend semantic search with classic keyword search, these two lead. Both are built for vector retrieval from the ground up, and both offer open-source and managed cloud versions.
\nWeaviate does hybrid search out of the box. Qdrant, written in Rust, is fast and light on memory.
\n| Option | \nBest for | \nHosting | \nHybrid search | \n
|---|---|---|---|
| pgvector | \nTeams already on Postgres | \nYour Postgres | \nWith Postgres full-text | \n
| Pinecone | \nZero ops at large scale | \nManaged (BYOC on Enterprise) | \nSupported | \n
| Weaviate | \nHybrid search and filtering | \nSelf-hosted or managed | \nBuilt in | \n
| Qdrant | \nFast, memory-efficient search | \nSelf-hosted or managed | \nBuilt in | \n
Pick the vector store your team can operate on a bad night, not the one that wins a benchmark.
\nPick the database that matches your constraints.
\nMost startups don't have a billion vectors. They have a few thousand PDFs. In the sub-1M range, pgvector is almost always the right answer.
\nIf you're building a global legal search engine or a large e-commerce recommendation system, you need the distributed design of Milvus or Pinecone. Don't build a large distributed system for a small amount of data.
\nPure vector search is weak at exact technical terms. Search for \"PHP 8.4 features\" and you might get general PHP articles. Hybrid search blends the meaning of the vector with the precision of a keyword match.
\nIf search quality is your top metric, look at Weaviate or Qdrant. If your data is highly relational, weigh vector search vs graph search before you commit to pure vectors.
\nEvery new piece of infrastructure is another thing that can break at 3 AM.
\nWith a small team, lean on managed services like Pinecone or Zilliz. With a strong infra team that wants to save on cloud margins at high scale, self-hosting Qdrant on Coolify or Kubernetes is the move.
\n\nCount the vectors you have today and estimate the count in 12 months. If both numbers are under a few million and you already run Postgres, install pgvector and build a prototype before you evaluate anything else.\n\nSince I work a lot with custom web development using Laravel, here is what this looks like in practice. Laravel 13 with the AI SDK (laravel/ai) has native vector columns and a whereVectorSimilarTo query method on Postgres with pgvector.
// migration: an HNSW index with cosine distance\n$table->vector('embedding', dimensions: 1536)->index();\n\n// query: pass the user's question as a string and Laravel embeds it for you\n$results = Document::query()\n ->select('content')\n ->whereVectorSimilarTo('embedding', $query, minSimilarity: 0.4)\n ->limit(5)\n ->get();\nUnder the hood this is pgvector's cosine distance operator, <=>. That snippet is the core of a RAG system: find the content, send it to the LLM and get a grounded answer.
On older Laravel versions, the pgvector/pgvector-php package gives you a Vector cast and a nearestNeighbors scope that do the same job.
These will save you weeks of refactoring.
\nVector indexes like HNSW are fast to search but slow to insert into. For a large initial load, insert your vectors first, then create the index. The pgvector docs say the same: it's faster to build the index after loading your data.
\nMake sure your embedding model and your vector database agree. If you use cosine similarity, normalize your vectors. It keeps scores consistent and prevents odd ranking bugs.
\nDon't store the whole JSON document inside the vector store. Store the vector and an ID, and keep the heavy data in your primary database. The index stays small and fast.
\nThese are the cheap wins. For the deeper traps (bad chunking, missing reranking, stale embeddings), see the 7 mistakes you're making with your production RAG stack. To test chunk sizes on your own documents, try my RAG chunk splitter.
\nI've built these systems for startups and established businesses, and this is how I usually guide them:
\nThe right stack is the one that lets you ship your AI features today. If you're choosing one for production, here's how I help teams design and ship RAG systems, or reach out with your war stories.
\nWhat's the hardest problem you've hit with retrieval in your RAG system so far?
\nUsers judge software by how it behaves. With AI writing more of the implementation, your taste and your intent decide whether a product feels right.
\n\n- \"Vibe coding\" (Andrej Karpathy, February 2025) means describing what you want and letting AI write the code; for professional work, he now prefers the term \"agentic engineering.\"\n- Your value moves toward taste: noticing when a screen feels off and steering the model with feedback on the feel as well as the function.\n- Vibe the surface and harden the core: keep a solid, tested foundation (Laravel, Docker) under an exploratory UI layer so the codebase doesn't turn into a ball of mud.\n\nMost developers are obsessed with logic. We spend years on syntax, query optimization and architecture debates, and we build systems that are technically correct but feel hollow.
\nThey work, but they don't sing. That gap is what vibe coding tries to close.
\nYour users care about how the software feels, far more than clever recursion: whether a button responds, whether a layout makes sense, whether the flow is calm or stressful.
\nProducts that win will master that feel, and AI tools make it cheaper to iterate on it than ever.
\nFor a long time I thought my job was to be a logic machine. If I followed every best practice and wrote the most efficient Laravel code possible, the project would succeed.
\nI was wrong.
\nLogic is the foundation, the skeleton that keeps the building standing. Nobody wants to live in a skeleton. People want a home with character and warmth, and in software that character comes from the vibe.
\nWhen we focus only on logic, we get software that does what it says on the tin and leaves users feeling nothing. Or worse, it frustrates them, because nobody thought about the friction of a slow button or a confusing layout.
\nAndrej Karpathy coined \"vibe coding\" in a February 2025 post: \"fully give in to the vibes, embrace exponentials, and forget that the code even exists.\" He framed it as a mode for throwaway weekend projects.
\nIn a February 2026 retrospective, he wrote that programming through LLM agents is becoming a default workflow for professionals, \"except with more oversight and scrutiny,\" and named \"agentic engineering\" as his favorite term for that serious version.
\nI use the broader sense the industry picked up: moving from writing code to curating intent with tools like Cursor and Claude. You describe the feel and behavior you want in natural language, and the AI does the heavy lifting on implementation.
\nYour value as an engineer shifts from typing speed to taste: recognizing when an interface feels \"off\" and knowing how to steer the AI to fix it.
\nThat is really a question of prompt engineering versus context engineering: giving the model enough of the right context to make good calls on its own.
\nOn a Shopify store build, the checkout logic matters. The feel of the checkout (smooth transitions, reassuring feedback, clean typography) is what moves conversions.
\nWhen code gets cheap to write, taste becomes the scarce skill.
\nTo work this way you need tools that stay out of your way, so the gap between an idea and a working screen stays small.
\nCursor changed how I work. Instead of spending twenty minutes on boilerplate for a new Vue component, I describe it in the chat: \"Build me a dashboard widget that feels airy and modern, uses a bento grid layout and gives the user a sense of calm control over their data.\"
\nThe AI generates the code and I review it. If the feel is wrong, instead of fixing it line by line, I talk to the model again: \"This feels too cramped. Give it more white space and make the shadows softer.\"
\nThat's the heart of vibe coding: a high-level conversation about intent.
\n\nSome of you are thinking: \"This sounds like a recipe for a messy, unmaintainable codebase.\"
\nYou're right to worry. If you vibe through a project without discipline, you end up with a ball of mud. Experience matters more here, because someone has to decide what the AI may touch.
\nVibe coding works when you shift your focus to where it pays off. Use your experience to build a solid core, so the vibe layer has something safe to sit on.
\nFor me that core is usually Laravel and Docker. Laravel is built for developer happiness, and it gives me a foundation I trust: authentication, database migrations and API structure.
\nWith that core in place and covered by tests, I can be exploratory with the frontend and the user experience. I vibe code the top layer because I trust the layer beneath it.
\nIn e-commerce, this approach fits well. Shopify has spent years refining how checkout and the admin feel.
\nWhen I do Shopify customization, I write Liquid code and I also try to match the brand's feel. A luxury jewelry brand needs a very different experience from a high-energy fitness store.
\nOne should feel slow, deliberate and expensive. The other should feel fast, punchy and motivating. You get there by obsessing over the details a logic-only developer skips.
\nGenerating a function to sort an array is now a commodity. Building an experience that moves people is rare, and that's where engineers can still stand out. If you want to bring AI-assisted building into your team without losing code quality, here's how I help teams set up AI coding workflows with guardrails.
\nWhat's the one app you use that just \"feels\" right, and what could you borrow from it for your next project?
\nMost developer blogs read like a LinkedIn post had a baby with a Medium article. This one won't.
\n\n- This blog is where I take notes out loud on the Laravel, Shopify and AI work I ship.\n- Expect Laravel internals, AI engineering with Claude and MCP, Shopify Plus patterns and hard-won DevOps lessons.\n- Every code sample is meant to run in production, and you can follow along through RSS with no newsletter or popup.\n\nI'm Anass. I ship Laravel + Shopify + AI for a living, and this is where I take notes out loud.
\nThe patterns I keep reaching for don't show up in tutorials. So I'm writing them down.
\nThink Laravel Octane at scale, RAG done right and agentic commerce.
\nEach of those took real projects to get right. Writing them down helps me, and it might save you a few late nights.
\nOpinions I'll change when proven wrong, backed by code that already runs in production.
\n| What you get | \nThis blog | \nAI-slop blog | \n
|---|---|---|
| Real production code | \nYes | \nNo | \n
| Opinions | \nYes | \nNo | \n
| GPT-written filler | \nNo | \nYes | \n
| Personality | \nYes | \nNo | \n
Every post that needs code gets the syntax treatment: real, runnable, not pseudo-code.
\n// app/Actions/AskClaude.php\nuse Prism\\Prism\\Enums\\Provider;\nuse Prism\\Prism\\Facades\\Prism;\n\n$response = Prism::text()\n ->using(Provider::Anthropic, 'claude-sonnet-5-5')\n ->withSystemPrompt('You are a senior Laravel engineer.')\n ->withPrompt('Refactor this controller for clarity.')\n ->asText();\nThat's it. First post live. More incoming. If you want this kind of work on your own product, here's how I help teams ship Laravel, Shopify and AI features.
\nWhich topic should I write about first: Laravel at scale, RAG, or agentic commerce?
\n