AI Client Review System for a Coaching Business
Coaches were collecting weekly client check-ins they no longer had time to read, so progress signals and early warning signs slipped through. The team needed to stay on top of every client without hiring more coaches.
Industry
Online coaching business (health and performance coaching)
Role
Systems consultant — workflow design, AI integration, automation build, documentation
Impact
Coaches went from a growing backlog of unread check-ins to reviewing their full roster in a short daily pass, with at-risk clients surfaced the same day they submit.
01 - Problem
The problem
This coaching business was built on close attention to every client, anchored by a weekly check-in: wins, struggles, adherence, energy, anything on their mind. It worked when the roster was small. By the time I came in, several coaches each carried dozens of clients, and the check-ins had become a liability — coaches were skimming, falling behind, or reading responses days late. Data clients faithfully submitted every week was barely used.
The cost was concrete. Clients who wrote that they were struggling sometimes heard nothing until their next call. Coaches felt the guilt of knowing the signals sat unread. The founder's worry was churn — clients who drift usually say so in a check-in before they cancel, and nobody was catching those in time. My constraint was clear: AI never replies to clients or makes coaching calls. The check-in is a trust channel. The job was narrower — turn unread long-form responses into something a coach can act on in minutes, with a human decision at the end of every path.
02 - System Flow
How the system moves
Client submits check-in
Weekly Typeform with client ID
Submission triggers scenario
Typeform webhook into Make
Claude summarizes response
Structured prompt, JSON output
Signals scored and validated
Progress and risk flags in Make
Summary lands in dashboard
Notion database per coach
At-risk clients trigger alerts
Slack message to assigned coach
Coach reviews and decides
Human reads, acts, marks reviewed
03 - Build
What I built
Restructured check-in form in Typeform
I rebuilt the check-in before touching automation. The old form was open text boxes — hard for humans and AI to work with consistently. The new Typeform mixes structured fields (adherence, energy, a self-rating) with the open-ended questions clients like answering, and carries a hidden client ID so every submission matches cleanly to a record. Structure before automation made everything downstream reliable.
Summarization pipeline in Make and Claude
A Make scenario picks up each submission via webhook and sends it to Claude with a tightly scoped prompt: summarize the week, list concrete wins and struggles, and assign progress and risk flags against criteria the coaches defined with me — not criteria the AI invents. Claude returns structured JSON, and Make validates it before anything moves. On a shape mismatch it retries once, then routes to an error channel with the original response untouched.
Coach-facing dashboard in Notion
Summaries land in a Notion operations dashboard: one row per check-in, filtered views per coach, flagged clients on top. Every row links to the full original response one click away — a summary is a starting point, not a substitute for the client's own words. A status field (new, reviewed, action taken) gives the team a shared picture of what's handled.
Risk alerts in Slack
When a check-in is flagged at-risk — declining self-ratings, quit language, a missed-week pattern — the assigned coach gets a Slack alert with the summary, the flag reason, and a link to the Notion record. Alerts are deliberately rare and specific; a channel that fires constantly gets muted within a week. Sensitive language bypasses summary politeness and tells the coach to read the full response now.
Documentation and handover
I delivered a runbook covering the workflow automation end to end: what each Make module does, how to adjust the prompt and flag criteria, what the error channel means, what to check when something looks off. The prompt lives in a versioned doc with a change log so the team can tune it without losing what worked. The business owns this system after I leave — it doesn't rent my attention.
04 - Impact
Business impact
Every check-in now gets read — by AI first, then a coach — instead of a growing share never being opened.
Reviewing a full roster went from an afternoon of reading that rarely happened to a short daily pass through the dashboard.
At-risk clients surface the same day they submit, so coaches reach out while the message is fresh.
Coaches walk into calls already briefed on the last few weeks, which clients read as attentiveness.
The founder has a single reporting view of check-in volume, flag rates, and review status across all coaches, where before there was none.
Structured check-in data is now a clean asset the business can build on, instead of text scattered across an inbox.
05 - Edge Cases
What had to be handled
Empty or one-word check-ins aren't summarized into false confidence — they're tagged low-signal, since a client who suddenly writes almost nothing is itself a pattern worth a coach's eyes.
Malformed AI output (JSON that fails Make validation) triggers one retry, then routes to an error channel with the raw submission intact, so a bad model response never silently drops a check-in.
Crisis-adjacent or health-and-safety language skips the normal queue and alerts the coach to read the full original response immediately — the AI summary is never the only thing a human sees.
Duplicate submissions in a week are matched on client ID and week number; the latest stays active, earlier ones are archived not deleted, so nothing a client wrote is lost.
Submissions that fail to match a client record (ID typo, test entry) land in an unmatched-triage view for a human to resolve instead of guessing.
If the Claude API is down or rate-limited, Make queues and retries on a schedule, and the original response still lands in Notion first — the dashboard degrades to unsummarized entries, not missing ones.
06 - Improvements
What I would improve next
Add a per-client trend view so coaches see trajectory across weeks — three slightly-down check-ins matter more than one bad one, and spotting that still relies on coach memory.
Build a one-click feedback loop where coaches mark a flag as wrong, and use those corrections to tune the flag criteria and prompt instead of adjusting on anecdote.
Send a weekly Slack digest per coach covering their roster's flags and unreviewed check-ins, as a safety net for anything missed in the daily pass.
If volume keeps growing, move long-term storage out of Notion into a proper database with a lightweight reporting dashboard — Notion is the right internal tool at this scale, but it has a ceiling.
Related capabilities