Building for humans — user research with AI
Meeting the people behind the software you're building.
And how to stay human while leveraging AI.
These last weeks we were in Norfolk and Norwich, my home city.
It gave me the chance to make a 'physical' appearance with the team I'm building software for.
Ten user interviews later...
- I've mapped over a hundred problems and fixes.
- Understood workflows better than AI agents could ever predict.
- And importantly... I didn't write a line of notes or read a single transcript.
Welcome to building for humans by doing user research with AI!
Family time, eating rocks, beaches and seal trips!
Worklife balance: interviewing the real people at the Norwich office
BUT! Customers don't know what they want!
The Steve Jobs line. Every founder quotes it.
Me argueing this with Steve Jobs.
Entirely right!
Customers do not know what they want.
A lot use it as a reason to skip talking to customers.
Easily done, building is fun, talking to people is slow when AI can drive builds much faster...
But that's not what we're asking in user interviews.
We're watching behaviour.
How they use what we've designed.
What works. What doesn't.
Testing a hypothesis, the way a scientist would.
This gives us what AI loves...
Context!
For the longer argument on why talk to users at all, I wrote an essay: Why do user research when AI could make the decisions for you.
Why context... or better put... data.
Last email: the most beneficial thing for my Dad's practice has been connected data. Context-enabled AI on the latest models.
User interviews are the same idea, pointed at people.
This, for me, is where AGI already exists.
AI can explore, define and analyse thousands of words at once.
It evaluates and cross references better than I can.
Hours of recordings. Hundreds of lines of transcript, with the screen video to go with it.
How to get the context.
Create a hypothesis - what do you want to test with users.
A script, if you want one. This time I didn't.
Bonus: a real interface. I branched our codebase and asked Claude:
"Create a real deployment of our onboarding interface and user flows with dummy data, off our real database, that interacts exactly like the real thing, that I can use for testing."
The interview harness: our real onboarding screens with fictional clients.
Then I went to the office and watched people use it.
Their real workflows, on a safe copy of the real thing.
We just clicked record.
Both the transcript and the screen.
Analysing.
Simple prompts. Frontier models. My current preference is Fable 5.1 for the deep reasoning.
One document for the core strategy. Another for the bugs and small fixes — that one went straight to a second agent to work through.
How to use video call transcriptions and video recordings? I built a kit, open source: github.com/charlieellington/user-testing-kit
It reads the transcript, pulls a still from the recording at every timestamp that matters, and writes each finding with the quote, the time and the frame.
One finding, start to end: the frame the AI pulled, the words said at that second, and the problem it became.
Findings.
The biggest one was onboarding. My sister's job (yes! she works for the practice too!). It's very fiddly right now.
Proposals and a client's onboarding experience can be generated in minutes.
Both the client proposal and the client's onboarding are built via an MCP in claude.ai, into a live interface and front-end links.
However, complex onboardings — multiple companies, directors, self-assessments and more — are not checked and the Ai was getting the clients screens wrong.
My manifesto on building this: fast AI work, slow human checks.
That needs to be built for onboarding.
So we're going to build "Onboard Design" into the sales workflow. AI builds what it thinks is right from the proposal and the Companies House data on the company. The team checks it, and can customise the experience in the interface (or tell Claude to customise it via the MCP).
User Research -> Sketching a solution, AI refines it from the user interviews analysis before we build.
Results.
This gives you an in-depth strategy for what to build next. Or how to improve a product, an internal process, and much more.
Doing this human part is what makes more niche solutions possible. More quality software.
Last email, the idea of software becoming more democratised resonated with a few of you.
User interviewing is one lego brick into building personalised software, not just for you, but for others.
Being human as a lego character.
What's next.
Do your own interviews:
User interview essay and guide: ellington.design/user-testing
AI transcript and video analysis kit (open source, MIT): github.com/charlieellington/user-testing-kit
Or hire me.
A Zebra Sprint is exactly this loop:
Hypothesis — what should we build.
Prototype — build the real solution.
Test — interview users.
Iterate, then release.
See the results from interviews
See what I shared with the team: https://www.triplebottomline.uk/team-update
Much love,
Charlie
P.S. Reply and tell me how you're keeping your AI use human first. I love getting mail!
This is email eight. Previous: ellington.design/emails.