Observability for a website and an order funnel

Context

As Growth Tech Lead I led the six engineers who owned bizee.com and the order funnel, where every lead and every order arrives. The team used an agent harness I built in Claude Code: hand it a ticket and it plans, implements, deploys and runs a post-release check in production.

Problem

Releases caused errors in visitors’ browsers: 339 JavaScript errors a day.

The data that could show a bad release lived in separate tools. Elastic RUM had field data from real visitors. Google Search Console had search data. Sentry had errors.

A single number could also mislead. The site-wide LCP percentile showed a slowdown that U.S. visitors were not seeing. About a fifth of page loads came from outside the U.S. Their 75th-percentile LCP was almost twice the U.S. one.

What I did

I built a 33-panel Kibana dashboard on Elastic RUM field data plus daily health checks. The dashboard covers Core Web Vitals from real visitors on the website and the order funnel, synthetic monitors, backend latency, external dependencies and errors. One panel sets website page loads against order starts.

On the LCP question, the slow countries showed no self-identified bots. Excluding bots barely moved the numbers. Bizee sells U.S. business formation, so I set the dashboard’s default to U.S. traffic, with any other country one selection away.

A Claude Code skill I wrote ran the daily sanity check of the website and the order funnel. It triangulates Kibana RUM field data, Google Search Console and Sentry, so no single source decides whether a release is healthy. Deploy gates with automatic rollback checked for de-indexed pages and broken images before users saw them.

Outcome

With the errors visible on the dashboard and in the daily sanity check, we turned them into tickets and worked through them sprint by sprint. Server errors fell about 85%. Release-induced JavaScript errors went from 339 a day to zero after the site moved to content-hashed asset file names.

A release that failed a gate rolled back on its own. The daily check read three sources before calling a release healthy.

Stack