UXR | Interaction Design
KAIRO: Designing interaction patterns for an AI debugging agent
Overview
Project Summary:
KAIRO is a bug capture and resolution tool for game developers. The flagship feature is an AI-powered Root Cause Analysis agent that investigates a reported bug and surfaces what caused it, which files are implicated, and what a fix might look like.
The pitch is compelling. In practice, developers were not buying it. Research showed they didn't understand the agent, didn't trust its output, and had no way to contribute their own context to the analysis. My goal was to explore whether redesigning the interaction around the output, without touching the model itself, could shift how developers experienced and perceived the agent.
Project Type: Applied Design Research
Results: +52% dependability, +61% intuitive use, +34% usefulness ,+13% trust
My Role: Product Designer (Mitacs Research Intern)
Timeline: Apr 2026 - July 2026
Domain: Developer tools | Game development | AI Interaction
Tools & Methods: 1:1 Interviews, trust scale, UEQ+, Figma, HTML/CSS, Claude.
Focus Areas: Human-in-the-loop Design, Interaction Patterns, Perceived Trust, Agent UX, Mixed Methods
WHAT WE HEARD
Game developers already don't trust AI tools. KAIRO gave them a reason to trust it even less.
AI tools make it effortless to get answers quickly. But effortless isn't the same as reliable. The outputs don't tell you how confident the system is, whether the response might be biased, or where its limitations are. That lack of transparency pulls users in two directions. Some overtrust what they're given, accepting outputs at face value without questioning them. Others don't know when they should trust the system at all. Both are design problems. And both come from the same root cause: an interface that was built for ease, not for informed judgment.
PROBLEM
Game developers already don't trust AI tools. KAIRO gave them a reason to trust it even less.
After 1:1 interviews with game developers and level designers, three things stood out:
The agent was invisible
developers didn't realise they could interact with the RCA output at all. They thought it was a static report.
"I had no idea I could ask it anything or give it more context."
One wrong answer broke everything
KAIRO showed a confidence score. One wrong high-confidence answer was enough to kill trust entirely.
"It takes time, gives you this confident answer, and the answer is just wrong. I never trusted it after that."
They were already sceptical
Most participants had studio policies against AI-generated output, or personal rules from past failures.
Their default: I'll verify it myself anyway.
DESIGN STRATEGY
Question I asked myself: I can't make the model more accurate. Can I change how developers interact with it?
I couldn't change the agent's reasoning or output. What I could propose and test was everything around it: when developers were asked to contribute, how the output was framed, and whether the experience felt like collaboration or just delivery.
Two supporting decisions were made based on research findings. The confidence score was removed and replaced with "Most Likely Cause" language, and the output structure and information architecture were reorganised in the proposed designs to better reflect how developers actually read and evaluate a diagnosis.
I proposed two interaction patterns to test against the baseline:
CLASSIC
(current version)
Agent runs. Output appears. Developer reads. No interaction.
STRUCTURED
(human-in-the-loop after)
Output appears first. Developer is then prompted to validate: does this look right? They can confirm, flag a problem, or explore alternative causes.
GUIDED
(human-in-the-loop before)
Before the agent runs, it pauses at two checkpoints. Developer confirms environmental context and flags recent changes. Their input shapes the output before they see it.
All three conditions used the same bug data and the same RCA output. Hence, the scores were a design effect, not a model effect.
THE THREE PATTERNS
How did the three interaction patterns differ?
Classic Interaction
Run RCA >> Agent analyses the bug >> Review output
Structured Interaction
Run RCA >> Agent analyses the bug >> Review output >> Agree /partially agree / disagree with the diagnosis >> Explore alternative causes (partially) >> Add more context (disagree)
Guided Interaction
Run RCA >> Review and edit analysis direction >> Flag recent changes or uncommitted modifications >> Agent analyses the bug >> Review output >> Investigate further if needed
FINDINGS
Guided scored highest. Classic came last on everything.
These are the findings compared to the classic baseline.
Perceived Trust (6-point Trust Scale)
Structured: –13%
Guided: +13%
compared to Classic baseline.
Perceived Overall User Experience (UEQ+)
Structured: +40%
Guided: +48%
compared to Classic baseline.
Dimension-based User Experience (UEQ+)
Dependability
Structured: +34%
Guided: +52%
Intuitive Use
Structured: +57%
Guided: +61%
Usefulness
Structured: +29%
Guided: +34%
compared to Classic baseline.
Preference Rankings
Guided was the first choice for 5 of 7 developers. Classic was the last choice for all 7.
"Guided felt like it would waste less time. It asks before running the heavy process. Structured still felt more intuitive than Classic.”
— Dev A
"I ranked them by how much control I felt I had and how well I could add my own context."
— Dev B
" Guided and Structured were similar to use, both clearly different from Classic. Having options to redirect the investigation was easier than trying to do it in Classic."
— Dev C
" The ability to directly correct the fix in Structured put it ahead for me. Classic felt loose, not like a purposeful tool.."
— Dev D
REFLECTION
The interaction is the product. Especially with AI!
Interaction shapes perception, not just accuracy
When developers contributed context before the output arrived, the same result felt more credible. The model didn't change. The interaction did.
Timing matters more than I expected
Structured improved intuitive use and usefulness but underperformed on trust. Post-output validation reads as friction, not collaboration. When you step in changes how you feel about the output.
Users want control over the output, not just visibility into it
Being shown a result and being asked to shape one are fundamentally different experiences. Developers didn't just want to see what the agent concluded. They wanted to feel like their input had a role in getting there.