Every business owner has had this moment. The dashboard says a key figure jumped last month. Your gut says it didn't. Now you have two problems: the number might be wrong, and even if it is right, you no longer trust the screen.
Most dashboards are built to look finished, with rows of cards and big numbers and green and red arrows. Very few are built to be checked. We think that is backwards, because a number you can't verify is a guess with good typography.
Most of what I believe about this came out of building the analytics layer for a med-spa's payroll and operations app.
What the owner asked for
The owner already had a reporting tool. What he asked for was different. He didn't want another grid of KPIs (key performance indicators, the handful of figures a business watches). He wanted something that reads the numbers and explains what moved and what matters, ranked by how much money each change is worth.
That is a harder brief than it sounds. To rank changes by dollar impact, every underlying number has to be right, and the owner has to believe it is right. So the first phase of the work went into trust. Charts could wait.
Start from the client's own definitions
The quickest way to lose a client's trust is to calculate a figure differently from the tool they already use. So we treated their existing analytics tool as a metric dictionary. For each figure, we wrote down how they define it, then recalculated it ourselves from the raw data in their scheduling platform's API.
Then came the checksum. A checksum here just means comparing our total with a total we already know is right. We took two months of the client's own figures and compared them with ours. Scheduled hours came within about 1.5 percent.
Where figures didn't line up, we left the formula alone and sent the client plain-English questions instead, such as:
- Should consults count toward a nurse's booked hours, or not?
- When a refund is issued, should it reduce the month of the visit or the month of the refund?
- Do staff treatments belong in these totals at all?
Tuning the formula would have produced a closer match faster. It would also have hidden a real difference in how the business thinks about its own numbers, and in the end the questions were worth more to the owner than a tidy match.
Showing the work
Each card on the dashboard has a "more…" link that opens a short explanation in plain language:
- what the number means
- how it is calculated, written as a formula a person can follow
- which records it includes and which it leaves out
- what period it covers, and when it was last refreshed
A simplified explainer reads something like this:
Revenue per scheduled hour
= service revenue (excluding refunds and staff treatments)
/ scheduled nurse hours
Period: last 7 days. Refreshed nightly from the scheduling API.It sounds small. In practice it settles most arguments about the dashboard before they start. If the owner disagrees with a number, he can read the definition and say "that's not how I'd count it", and that is a conversation we can act on.
The explainer tells you how a number should be calculated. The debug view shows how it actually was. The request behind it was simple: "a debug link if you will so we can test against known values."
For any card, the debug view lists the records that produced the number: each visit, each shift, each line that was included, and each line that was excluded and why. That makes the most useful check possible. Pick a day you know well, work it out by hand, and compare.
We use the debug view ourselves before anything ships. When a calculation changes, we re-run it against a set of days with known answers. If a known value moves and it wasn't supposed to, something broke, and we find out before the client does.
Raw data for any day
The last layer is the simplest. For any day, you can download the raw data the app received from the API, untouched. If the debug view shows the work, the raw export is the problem exactly as it was handed in.
That file separates our bugs from source bugs. If a figure looks wrong, it tells you whether the API sent bad data or we calculated badly. It also lets someone else check: an accountant or a second developer can verify a number without needing to understand our code.
It had already paid for itself before the dashboard existed. When we corrected how the app sorted staff treatments, mobile visits, consults and refunds, the raw exports showed exactly which records were landing in the wrong bucket. Our rule on that job was to change code only after the raw JSON confirmed the field values, never by guessing what an API field means.
Ranking by dollar impact, carefully
Once the numbers were trusted, the story layer could be built on top. Phase one is rules-based. It compares this period with the last, works out what each change is worth in dollars, and ranks the changes. An optional second phase adds a written narrative from a language model, but only on top of numbers that have already passed the checks above. A model that explains wrong numbers fluently is worse than no explanation at all.
One rule from that work applies everywhere: ignore big percentages on tiny bases. A service that went from two bookings to four is "up 100 percent" and worth almost nothing. Ranking by dollars instead of percentages keeps a dashboard from shouting about noise.
We also added a development tab inside the app, visible to the client. It lists the scope, the phases and the open questions. When a metric is waiting on a client decision, the tab says so. That keeps questions from getting lost in email threads, and it makes it obvious why a particular number isn't on the dashboard yet.
What we do on every dashboard now
None of this is expensive compared with building the dashboard itself, so it has become the default on our projects.
- Define every metric in writing, using the client's own language.
- Recalculate from raw data, then checksum against figures the client already trusts.
- When numbers don't match, ask. Don't tune the formula until it agrees.
- Give every card an explainer and a debug view.
- Let anyone download the raw data for a given day.
- Rank changes by money, not by percentage.
- Keep a list of known values and re-test them after every change.