A copilot that lives inside the software your users already have open.
Chat and copilots scoped to real actions against your own systems, not a general chatbot bolted onto the side of the product.
What this is
This is a chat or copilot interface built into software you already run, your SaaS product, an internal tool, a customer portal, rather than a standalone chatbot living on its own page. The interesting engineering is not the chat window. It is deciding exactly what the assistant is allowed to do and giving it real tools to do it: look up a record, draft a reply, create a ticket, calling your existing APIs instead of guessing at an answer.
The model does not enforce its own limits. If it has API access to cancel a subscription, it will eventually call that when a user vaguely implies they might want one, because a system prompt asking it to be careful is a suggestion, not a boundary. Permission scoping and a confirmation step for anything irreversible have to live in the code around the model. This is the part that goes wrong in builds that only had to survive a demo.
A prompt that works today can quietly stop working after a model update, or after somebody edits an unrelated part of the system prompt. An assistant that ships needs an evaluation set of real conversations it has to keep handling correctly, the same discipline as automated tests for ordinary code, run on every change so a regression is caught before a customer finds it.
If what is needed is answering questions from company documents without taking action, that is usually knowledge search over your own documents on its own, cheaper to build and cheaper to run. If what is needed is turning a flow of incoming documents into structured records, that is document and email processing. This service is for the case where a conversational interface, embedded in your product and calling your own systems, is genuinely the right surface.
What you get
Scoped tool set
A defined list of actions the assistant can call against your APIs, with anything irreversible requiring an explicit confirmation step outside the model.
Conversation UI embedded in your product
The chat surface itself, matching how your users already work rather than a bolted-on widget.
Session and identity handling
The assistant knows who is asking and acts only within that user's own permissions, never a shared service account with access to everything.
Retrieval layer where it needs company knowledge
Often built as its own component, and reusable on its own if the retrieval need outgrows the assistant.
Guardrails and refusal handling
What the assistant declines to do, and what it says when it does, decided deliberately rather than left to the model's default behaviour.
Evaluation set of real conversations
A fixed set the assistant has to keep handling correctly, run against every change so a prompt edit or model upgrade does not silently break something that used to work.
Logging and escalation path
A transcript a human can review, and a way for a conversation to hand off to a person when it should.
When this fits, and when it does not
A good fit
- You have software your users already open every day, and a chat or copilot inside it would save them navigating five screens for something they could just ask for.
- There are real actions to take, not just questions to answer, look something up, draft something, update a record, and you want those wired to your actual systems.
- You can name what the assistant should never do, refund without approval, delete a record, message a customer unprompted, because that list is the actual design work.
- Somebody already tried a quick prompt against an API and it worked in a demo, and the question now is making it hold up outside the demo.
Not a good fit
- What users need is answers pulled from your documents, not actions taken in your systems. Build knowledge search over your own documents on its own; it is simpler, cheaper to run, and there is no tool-permission problem to solve.
- You want the assistant to act with financial or legal consequences fully unsupervised. We will build the workflow, but anything irreversible gets a confirmation step; an assistant that can issue refunds on its own with no human check is not something we will ship.
- The audience is three internal people who could be helped faster with a script or a better filter on the existing tool. A chat interface is not the lazy option here, it is the expensive one.
- You have not decided what the assistant should actually be allowed to do. That is design work worth doing first, and often the honest starting point is an AI feasibility study rather than building against an undefined scope.
How it runs
- 01
Scope the tool set
Exactly what the assistant can call, what needs confirmation, and what it should refuse, written down before any UI gets built.
- 02
Wire to real APIs
Against your actual data, not a mocked demo environment, because permission edge cases only show up against the real system.
- 03
Build the conversation UI
Into your product, with session handling that ties each conversation to a real, permission-scoped user.
- 04
Build the evaluation set
From real conversations, and run the assistant against it until it holds up, not just on the happy path someone demoed.
- 05
Ship with logging and escalation
And hand over the prompts and source, so changes do not require us.
Questions we get
What happens when the underlying model gets updated by the vendor?
The evaluation set catches regressions before your users do. We re-run it against updates and flag what changed rather than assuming nothing did.
Can it take actions, or does it just answer questions?
Both, scoped to whichever tools it is given. If it is answers-only for your case, knowledge search over your own documents alone is usually the simpler and cheaper build.
How do you stop it from doing something a user did not actually ask for?
Permission boundaries and confirmation steps live in the code that wraps the model, not in the prompt. The model is never the place we enforce limits.
Which model do you build on?
Whichever fits the cost, latency and data-handling constraints of the job. That choice, and whether it needs to run on-premise, is usually settled during scoping rather than fixed in advance.
Have a product that could use a copilot?
Describe what your users currently do by hand or by ticket. An engineer reads it and gives you a straight answer on whether an assistant is the right fit for it.