FILE 08Research buildIn-house
The brief
Before an agent touches a client's systems, we want to know where careful design stops it from doing harm. This build stress-tests that.
The approachA working helpdesk agent for a made-up company, with tools, a person approving anything risky, three layers of guardrails, a second AI checking every answer, and a built-in search test.
What we built
- A working IT helpdesk agent for a made-up company, built to stress-test agent design before it reaches a client.
- Finds answers with keyword and synonym search across the knowledge base, and shows the whole knowledge base as a 3D map.
- Uses six tools, and asks a person to approve anything risky before it runs a fix on a simulated laptop.
- Three layers of guardrails, tested with a planted document that tries to give the agent hidden orders.
- A second AI checks every answer against its sources, and a built-in test scores the search: 16 of 16 questions found the right article in the top three.
Deliverables
- Agent with six tools
- Human approval before risky actions
- Three layers of guardrails
- Answer checking by a second model
- Search test: 16 of 16 in the top three