← Create New Paste

System Design Interview: Designing a Page View Counter and Web Analytics Service

3 views
"Design a service that counts page views for millions of websites and shows dashboards" sounds simple, but it is a good system design interview question because it forces you to discuss high-volume writes, aggregation, approximate counting and time-series storage. 1. Requirements Functional: record a view event when a page loads; show total views and unique visitors per page over time ranges (last hour, day, 30 days); show top pages. Non-functional: ingest maybe 1 million events per second at peak, dashboards a few seconds to a minute behind real time, and the tracking script must never slow the customer's website. 2. Estimate At 1M events per second with roughly 200 bytes each, raw ingestion is about 200 MB per second, or 17 TB per day. You will not query raw events for dashboards; you will pre-aggregate. 3. Ingestion path - A tiny JavaScript snippet sends a beacon (navigator.sendBeacon or a 1x1 request) to a collector endpoint behind a CDN or load balancer. - Collectors are stateless; they validate, add server timestamp and geo info, and write to a durable log such as Kafka, partitioned by site_id. - Respond immediately; never make the page wait on downstream processing. 4. Aggregation A stream processor (Flink, Spark Streaming or a consumer group) reads the log and builds rollups per (site_id, page, minute): view count and a HyperLogLog sketch for unique visitors. HyperLogLog is the key point: exact unique counts across arbitrary time ranges are expensive, while HLL gives around 1 percent error in a few kilobytes and sketches can be merged to answer "unique visitors this week" from daily sketches. Minute rollups are later compacted into hourly and daily rollups to keep storage and query costs down. 5. Storage - Rollups in a time-series or columnar store (ClickHouse, Druid, or Cassandra with time-bucketed keys). - Raw events in cheap object storage for reprocessing and ad-hoc analysis. - A cache for popular dashboard queries. 6. Top pages Maintain a per-site top-K using a count-min sketch plus a min-heap per time window, or compute top pages from hourly rollups with a periodic batch job if exactness matters more than freshness. 7. Correctness and edge cases - Duplicates from retries: include an event id and deduplicate within a short window in the stream processor. - Late events: allow a lateness window (for example 10 minutes) and update rollups; anything later goes to batch correction. - Bots: filter known user agents and abnormal rates before counting. - Privacy: hash or truncate IPs, honor consent, and avoid storing personal data you do not need. 8. Trade-offs to mention Lambda (stream plus batch correction) versus pure streaming; exact versus approximate unique counts; minute-level freshness versus cost. A clear answer walks through requirements, numbers, ingestion, aggregation with HyperLogLog, storage and failure handling. If you want support while the interviewer probes deeper, Linkjob AI (https://linkjob.ai) offers real-time help during job interviews and technical interviews, including system design rounds.
Created Nov 17, 2026 09:34 Expires Nov 17, 2027 09:34