Skip to content
PROOF48
All answers
ai

What to actually check when you review a week of AI assistant conversations

Sort the week into three piles, the same three PROOF48's AI assistant page uses: answered, escalated and unanswered. Read the answered pile for confident wrong facts, check that a person really replied to each escalation, and turn every unanswered question into a new answer or a standing handoff rule. Treat the first month as learning.

Founder, PROOF48Updated 2026-10-295 min read

01Start with the answered pile, not the problems

Most people open the escalations first, because that is where the drama is. Start with the answered pile instead. Those are the conversations nobody saw while they happened, so a wrong price, an old opening time or a policy you changed last month went out with full confidence and no one flagged it. Read a handful with your current price list and hours beside you, and compare line by line. The other two piles show the assistant admitting what it does not know. A wrong answer in the answered pile is the failure that can cost you a customer. How many you read is your call. Our AI assistant page says you see every conversation. In our view, read the escalated and unanswered ones in full and sample this pile.

  • Keep your current price list, hours and policies open beside the conversations
  • Check every number, date and promise the assistant gave
  • Look for answers that were right last season and are wrong now
  • Note any answer you would not have given yourself, even if it is not wrong

02Escalated: did a person actually pick it up

An escalation is not finished when the assistant hands over. It is finished when a person has replied. Open each escalated conversation and ask four things. Did someone answer? How long did the customer wait? Did the person have to ask the customer to repeat what was already written? Was it the right person, a manager for a complaint and not whoever was nearest? Our AI assistant page says you decide what triggers a handoff, such as the type of topic or the time of day. So a conversation that escalated for a reason you did not intend is a rule to change, not a one-off. In our view, a pile of escalations about one topic means that topic should go to a person from the first message, and the assistant should say so plainly.

  • Did a person reply, and how long after the handoff?
  • Did the customer have to repeat anything?
  • Was it the right person for that kind of question?
  • Which trigger fired, and did you want it to?

03Unanswered: every question gets one of three outcomes

Each unanswered question is a decision. If the answer is a stable fact, such as a delivery area or a cancellation rule, add it so the assistant can answer next time. If the answer needs judgment, such as a complaint or an unusual request, make it a standing handoff topic. If it has nothing to do with your business, leave it. Look for repeats before you act: one odd question is noise, the same question from five people is a gap. Our AI assistant page says the first month is learning and that escalations drop after that. We publish no figure for how much or how fast, because we have not measured it.

  • A stable fact: add the answer
  • Needs judgment: make it a standing handoff topic
  • Off topic: leave it
  • The same question more than once: treat it as a gap first

04Make the review someone's job

The review fails the way every weekly task fails: nobody owns it. Pick one person who knows the business rules, usually the owner or a manager, and give them a fixed slot in the week. Conversations hold customer names and contact details, so decide who may read them. Our post on chatbot data covers the contract questions. Keep a one-line log with the date, what you changed and what you want to watch. Next week, check last week's change first: did the same question come back?

  • One named reviewer who knows prices, hours and rules
  • A fixed slot in the week
  • A one-line log of what changed
  • Last week's change is the first thing to check

05Where PROOF48 is the wrong choice

PROOF48's AUTOMATE plan costs 2,997 EUR a month and lists an AI website chatbot that qualifies and books, AI DM and email auto-responders, booking automation with a CRM and two custom automations a month. It does not cover WhatsApp or phone. Our AI assistant page describes the weekly review as ongoing work after launch, but it only works if somebody reads. If nobody in your business will own it, we are the wrong choice, and so is any other vendor. We are also the wrong choice if every conversation needs a human judgment call. We publish no figure for how many conversations a client sees in a week, how long a review takes or how often escalations drop, because we have not measured them.

  • AUTOMATE, 2,997 EUR a month: website chatbot, DM and email auto-responders, booking automation, CRM
  • WhatsApp and phone are not covered
  • Wrong fit: nobody will own the weekly review
  • Wrong fit: every conversation needs a human judgment call
From a build we shipped

No assistant to show, so here is the nearest build

None of our case studies is an AI assistant, so we have no real conversations to show you. Our AI assistant page says client deployments are underway and will appear there when they are live and documented. The nearest build is Santorini Hotel in Eforie Nord: a booking enquiry flow, and reception, restaurant and complaints phone lines published separately on the site and in its structured data. That is not a chatbot and it proves nothing about reviewing one. It does show the habit behind the review: sort what comes in by who should handle it. We publish no booking or occupancy figures for the hotel.

See the build: Santorini Hotel

Questions, answered

How often should I review AI assistant conversations?

Our AI assistant page describes a weekly review. In the first month, which that page calls a month of learning, looking more often is your call.

Do I have to read every conversation?

The page says you see every conversation, sorted by category. How many you read is up to you. We would read the escalated and unanswered ones in full and sample the answered ones.

Who should do the review?

Someone who knows your prices, hours and rules well enough to spot a wrong answer, usually the owner or a manager.

Does PROOF48 publish how often its assistants escalate?

No. Our AI assistant page says escalations drop after the first month. We have not measured by how much and publish no figure.

Where PROOF48 fits

A weekly review of AI assistant conversations is three jobs: read the answered pile for wrong facts, confirm that a person replied to every escalation, and turn each unanswered question into a new answer or a standing handoff rule. Give the review one owner and a fixed slot. PROOF48's AUTOMATE plan, 2,997 EUR a month, includes a website chatbot and does not cover WhatsApp or phone.

· Founder, PROOF48

Cosmin has worked in online marketing and software development since 2018, and has built with AI every day since January 2023. He still writes code from scratch daily, in Python, React, Rust and C#. PROOF48 ships business websites in 48 hours and invoices afterwards, so every opinion on this blog was paid for in real builds, not in theory.

Related articles