What is an AI co-worker?

Take the name literally, because I do. A co-worker is the person sitting at the desk next to yours. They have their own computer, they are working on their own thing, and when you need them you turn around and ask. That is the whole image.

There is a hierarchy in that picture and I am not going to pretend otherwise. You are the one in charge. You are driving, you know exactly what is going on, and they know you are leading. Not superiority in a bad way, just clear: the work is yours. And at the same time they are genuinely good with the tools they have at their disposal, which is why you asked them in the first place.

The interesting part is what a good colleague does with a request. You say "can you do A" and they do not only do A. They know A needs B and C done first. They know what A is going to mean for the rest of the work. Sometimes they push back: sure, we can do that, but then we have to handle the other side too. That is judgment, and judgment is what separates a colleague from an assistant.

Nearly every AI agent shipping today sits on the other side of that line. They take commands. They are subordinate to us by design, and that is a perfectly reasonable thing to build. It is just not what I wanted. I wanted something that steps all the way into the frame of the app, sees it the way the user sees it and the way the developer built it, and works from inside that.

"Copilot" was the right word for 2023, when the thing being shipped was a chat box that could read your data and write you a paragraph. Three years on the calendar, which in this field is a decade. The word carried an implied promise it never delivered: someone sitting next to you, doing half the work.

Here is my honest read on why that promise stalled.

Most copilots put the AI next to the app instead of inside it, which leaves the work with the user. The AI produces an answer; the human still does the clicking. That is a better search box, not a colleague. The user's job was never "get an explanation", it was "get the thing done".

The ones that do act usually make you build a second, parallel app for the AI. You define actions on your backend, wire them to tools, then keep that surface in sync with your real product forever. Every feature now ships twice. That is the tax that quietly kills these projects six months in, long after the demo got applause.

And the failure modes people blamed on the model were mostly structural. When our own agent got confused about whether it or the user had just navigated, the instinct was "try a better model". We A/B'd a newer model against the old one on exactly that confusion and found no meaningful difference. It was a race in the runtime, not a gap in intelligence. You cannot prompt your way out of a race, and you cannot buy your way out with a bigger model. That finding is why I think of this as a runtime problem rather than a prompting problem, and it is the single most useful thing I learned building it.

A co-worker is a different claim than a copilot. A copilot advises. A co-worker is given a job, works inside the same tools as everyone else, and is accountable to the same permissions.

Being clear about the boundaries is most of the definition. Here is the fast version, the neighbours people put us next to and when each of them is the right call instead:

A co-worker is none of these. It is a runtime that sits inside a product you control, which is what lets it be both useful and safe.

That table is the summary. The tools you will actually weigh us against each get their own page, where the comparison is specific and fair, and I say plainly where they are the better buy. Each one links back to this page.

All of them, side by side: see how we compare.

This is the part people assume is magic, so here is the real mechanism, with the real integration surface from our SDK.

The app tells the co-worker what exists

There is no DOM scraping and no screenshots. Every single thing the co-worker can see or do is something you declared, in code, on purpose. That is a deliberate design choice, and it is the difference between an agent that is safe by construction and one that is safe by hope.

You declare four kinds of thing.

Actions: what it can do. These are the functions you already wrote for your own buttons.

useAction({
  name: "addTask",
  description: "Add a task to the user's list. Use when the user asks to add or create a task.",
  kind: "write",
  parameters: {
    type: "object",
    properties: { title: { type: "string", description: "The text of the task." } },
    required: ["title"],
  },
  run: (args: { title: string }) => addTask(args.title),
});

One operation, two operators: a human clicks it, or the co-worker calls it. Your handler runs either way.

Surface: where the user is. Plus any page-level state you want the co-worker to be able to reason about, and an optional cue for how to behave on this screen.

useSurface({
  label: "Tasks board",                      // the name the co-worker says out loud
  data: { total: tasks.length, remaining },  // your own page state, any shape you choose
  cue: "The user is managing their to-do list; answer 'what's left' from this data.",
});

Elements: things on the page it can see and operate. An element with no actions is see-only. Add actions and it can work the control.

// Operate-only, the minimal interactive form:
useElement({
  name: "view-mode",
  actions: { setKanban: () => setViewMode("kanban"), setList: () => setViewMode("list") },
});

// See-only, read-only state the co-worker can reason about:
useElement({ name: "next-task", state: { nextUp: nextTask?.title ?? "all done" } });

Cues: what matters here. A short piece of guidance that rides on a surface or an element. It is read as data about the page, never as an instruction that can override the co-worker's guardrails.

That is the whole vocabulary. Actions, surface, elements, cues. Integration is annotation, not rewriting: you are labelling the app you already have, one screen at a time, and every screen you have not labelled yet simply stays invisible to the co-worker instead of breaking.

The one server-side piece

There is no AI backend to build, and I want to be precise about that phrase rather than let it inflate. You do not build a voice pipeline, you do not host a model, you do not define a parallel set of backend tools for the AI, and you do not keep any of that in sync with your product. What you do build is one token route, because your secret key has to live somewhere that is not the browser:

import { coworkkitSessionRoute } from "@coworkkit/server/next";
import { auth } from "@/your-auth";

// Derive the user id SERVER-SIDE from your session, never trust the client.
export const POST = coworkkitSessionRoute({
  getUserId: async () => (await auth()).userId,
});

Then the browser side is one wrapper:

import { CoworkkitProvider, type CoworkkitSession } from "@coworkkit/react";

// Fetch a session from YOUR backend, which holds the secret key.
async function getToken(): Promise<CoworkkitSession> {
  const res = await fetch("/api/session", { method: "POST" });
  return res.json();
}

export function App() {
  return (
    <CoworkkitProvider getToken={getToken}>
      <YourApp />
    </CoworkkitProvider>
  );
}

Anyone who tells you an in-app co-worker needs zero backend is selling you something. That route is the backend, all of it, and it is the reason the rest of the model holds together.

What happens during a turn

The user presses talk and speaks. The hosted runtime transcribes, the model decides, and it speaks back. Three details are worth knowing because they are where the design lives:

• The tool catalogue is announced at runtime, not built. When the session starts, the browser publishes the actions currently mounted in your component tree over the session's data channel, and it publishes updates as components mount and unmount. No build step, no code generation, no backend deploy, no registering tools in a dashboard. Navigate to a new screen and the co-worker's abilities change with it, mid-conversation. That is also how "do A" survives A needing B first: it goes where the work is, its abilities change with the screen, and it picks the original job back up.
• When the model calls one of your actions, the call travels back into the browser. It is a request/response over the live session, and the thing that runs on the other end is your own run handler, executing in the user's already-authenticated tab. The result goes back to the model as a tool result and it carries on talking.
• What the user is looking at reaches the model as a tool result, not as an instruction. That sounds like a footnote and it is not. Delivered as instruction-channel text, models read the page state out loud, including fields they invent. Delivered on the channel a model already knows how to synthesize rather than recite, the problem disappears at the root. Again: runtime, not prompt.

Why "as the signed-in user" is an architecture and not a slogan

The co-worker process holds zero credentials to your backend. None. There is no service key, no impersonation token, no "act as user X" primitive anywhere in the design. The only path from the model to your data is through a handler running in that user's own browser session, using whatever auth your app already uses.

Three consequences follow, and they are the reason I chose this shape:

• Your existing authorization is already the authorization layer. Same session, same middleware, same row-level security, same audit trail. There is nothing new for a security reviewer to evaluate.
• The blast radius is structurally bounded. Compromise the agent and you get no direct backend access, because there is nothing to compromise. It cannot act as the wrong user because no primitive exists that could.
• You do not plumb anything. The alternative design, where the AI mints signed tokens your backend has to trust, means new middleware, key rotation, claim mapping and a security review. This design means writing handlers that call your backend the way your UI already does.

The honest cost of this choice: work that outlives the browser tab does not belong in a handler. If the co-worker promises to send a report and the user closes the tab, the handler is gone. The pattern is the same one your app already uses for anything slow, which is to enqueue a server-side job from the handler and let it run as the user through your existing plumbing.

Everything above is one idea wearing four hats: nothing is exposed unless you expose it.

If this were a show, the developer writes the script. You decide which screens the co-worker knows about, which controls it may touch, what matters on each page, and which moves need a human hand on them. Inside that script the co-worker genuinely improvises. It decides what to say, what order to work in, when to ask, when to tell you that you are about to make a mess. It just cannot walk off the set.

That is the opposite of handing an agent your whole product and hoping. Point a general browser agent at a real app, or expose your surface wholesale to whatever agent turns up, and everything is on the table, including the parts you would never have chosen to hand over. It looks great in a demo. It is a wild west in production.

Production has one real criterion: nothing can go wrong in a way nobody sanctioned. Scripted, with room to improvise inside the boundaries you drew, is how you get to keep both.

Something with hands inside your product needs a real safety model, not a disclaimer. Four controls do the work, and I would read this section twice if I were evaluating anyone's in-app agent, ours included.

Every action declares how firmly it is gated.

• open: it just runs. Reads, benign changes.
• soft: the co-worker has to say what it is about to do before it does it.
• hard: a confirmation card appears and the user clicks it before anything runs.

hard is enforced by the SDK in the browser, not by asking the model nicely. The handler is not called. A confirmation is raised, the user taps, and only then does your code run. The summary shown on that card is computed in the browser from the actual arguments and never crosses the wire:

useAction({
  name: "deleteTasks",
  description: "Delete tasks by id.",
  control: "hard",
  confirmationSummary: (args: { ids: string[] }) => `Delete ${args.ids.length} tasks?`,
  run: (args: { ids: string[] }) => removeTasks(args.ids),
});

Touching the UI is gated separately, and the user arms it. Actions that operate on-page elements are what we call Hand mode, and they cannot fire while the user has it switched off, no matter what the model decides to try. That gate lives in the SDK, in the browser, at the point of dispatch.

You can tighten the whole app with one prop, and you get an audit trail for free.

<CoworkkitProvider
  getToken={getToken}
  defaultControl="soft"                          // app-wide baseline for ungated actions
  onActionRecord={(record) => logToYourSiem(record)}
>

Every settled action emits one record: what ran, which gate applied, and how it ended, including the ones that were blocked or cancelled. The SDK emits, you decide where it goes.

And the ceiling is still the user's own permissions. All of the above sits on top of the identity model, which is the actual guarantee. Gates decide what the co-worker may attempt; the signed-in session decides what is even possible. It can never do more than that user could do themselves. Not "should not", by policy or by prompt. Cannot, because there is no other path.

This is a young category and I would rather you find out here than three weeks in.

• The user has to be there, with the app open. This shape is for work done alongside someone. If you need an agent that acts overnight with nobody watching, that is a backend agent, and the honest answer is to queue a job.
• There is no phone line. In-app voice over WebRTC only. If the channel is a phone call, use a telephony-first voice platform, and they are genuinely better at it than we will ever be.
• Web and React today. The identity model and the wire protocol are framework-neutral by design, but the SDK you would install today is React. No mobile SDK.
• Nothing it can see is free. Actions are cheap because they are functions you already have. Screen-level and element-level annotation is per-screen work, and on a large product that is real effort, incremental but real. Anyone promising a co-worker that just knows your whole app is either scraping the DOM or guessing.
• If you cannot change the app, this is the wrong tool. Declaring things requires owning the code. Point a browser agent at it instead and accept the trade.