Why web apps feel slow: measuring server latency with percentiles

Questions

1. 1
   Not answered. Your service's p99 latency for the last hour is 250 ms. What does that number mean?
2. 2
   Not answered. Over the same hour, Server A reports a p99 of 100 ms and Server B reports a p99 of 300 ms. If you combine their traffic, what is the combined p99?
3. 3
   Not answered. A dashboard shows a p99 for each of the last 24 hours. The hourly values range from 200 ms to 900 ms, and their average is 350 ms. Is the daily p99 350 ms?
4. 4
   Not answered. Which of these numbers can you combine across two sources (two servers, or two time windows) by simply adding them?
5. 5
   Not answered. Your monitoring tool stores latency in histogram buckets (0 to 10 ms, 10 to 25 ms, 25 to 50 ms, and so on) instead of storing every request. What's true about a p99 computed from those buckets?
6. 6
   Not answered. Why is the slowest request you've seen (the maximum latency) worth watching, but usually a bad target for a service-level objective (SLO)?
7. 7
   Not answered. You want an accurate p99 across all 200 servers in your fleet. Which approaches give a correct answer?
8. 8
   Not answered. Latency data is usually right-skewed: nothing can be faster than zero, but a long tail of slow requests stretches to the right. What does that mean for the average (mean) compared to the median?
9. 9
   Not answered. Most requests to a service hit a warm cache and finish in about 5 ms. The rest miss the cache and take about 200 ms. Why does a single percentile like p90 describe this service poorly?
10. 10
    Not answered. This week, p50 latency rose from 40 ms to 90 ms, but p99 stayed at 800 ms. What kind of problem does that usually point to?
11. 11
    Not answered. p50 latency stays at 40 ms, but p99 climbs from 300 ms to 1,200 ms. Which statements are true?
12. 12
    Not answered. The chart below is a CDF (cumulative distribution function). Latency runs along the bottom, and the share of requests finished by that latency runs up the side, with lines marking p50, p95, and p99. What does a CDF make easier to read than a regular histogram?
13. 13
    Not answered. Which of these often cause a web service's latency to have two peaks instead of one?
14. 14
    Not answered. Loading a page makes 100 backend calls at the same time. Each call has a 1% chance of being slow (over its p99), independent of the others. What percentage of page loads hit at least one slow call?
15. 15
    Not answered. When a page waits on N backend calls running at the same time, the page takes as long as the slowest call, not the average one. What happens as N grows?
16. 16
    Not answered. Why does a system's tail latency usually depend on its slowest dependency rather than its average one?
17. 17
    Not answered. A hedged request sends a duplicate request after a short delay and uses whichever response comes back first. Which statements about hedging and retries are true?
18. 18
    Not answered. Same setup, but now the page makes 20 backend calls instead of 100. Each still has an independent 1% chance of being slow. What percentage of page loads hit at least one slow call?
19. 19
    Not answered. Back to 100 backend calls. Hedging now cuts each call's chance of being slow from 1% to 0.1%. What percentage of page loads hit at least one slow call?
20. 20
    Not answered. Little's Law connects three averages: the number of requests in a system (L), the rate at which requests arrive (λ), and the time each request spends in the system (W). Which equation is it?
21. 21
    Not answered. A service handles 200 requests per second on average (λ) and has 50 requests in progress at any moment on average (L). Using Little's Law, how long does each request spend in the system on average (W), in milliseconds?
22. 22
    Not answered. A service handles 500 requests per second on average (λ), and each request takes 100 ms on average (W). Using Little's Law, how many requests are in progress at any moment on average (L)?
23. 23
    Not answered. The chart below plots the average time requests wait in a queue against how busy the server is (utilization, ρ), with points at 50%, 80%, 90%, and 95%. What happens to wait time as utilization rises from 80% to 90% to 95%?
24. 24
    Not answered. Using ρ / (1 − ρ) as a relative measure of queueing delay (higher means longer waits), what is its value at ρ = 0.95?
25. 25
    Not answered. A system has exactly enough capacity for its average traffic and nothing more. Why does it tend to break down, not just slow down a little, when traffic reaches about 2× the average?
26. 26
    Not answered. Two systems are both 80% busy on average and have the same average time per request. In one, request times vary a lot: some take 1 ms, others 500 ms. In the other, every request takes about the same time. Which statements are true?
27. 27
    Not answered. Ten servers each have their own queue. Alternatively, the same ten servers share one queue, and the next free server takes the next waiting request. With the same total traffic and the same server speed, which setup has shorter average waits, and why?
28. 28
    Not answered. An app can group several writes into one database call. Bigger batches mean fewer, more efficient calls (higher throughput), but each item waits longer for its batch to fill (higher latency). Where is the best batch size usually?
29. 29
    Not answered. The timeline below compares the evenly spaced requests a load tester planned to send with the requests it actually sent while the service froze for a while. Based on that gap, what is coordinated omission?
30. 30
    Not answered. A load tester waits for each response before sending the next request. What does coordinated omission do to the latency percentiles it reports?
31. 31
    Not answered. For the same request, the server logs 40 ms of processing time, but the browser measures 180 ms from the click to the response. What does that 140 ms gap most likely contain?
32. 32
    Not answered. Why does a latency SLO need both a time window, like "over any rolling 30 days", and a target, like "p99 under 500 ms", instead of just a number?
33. 33
    Not answered. An SLO promises 99.9% availability over a rolling 30-day window. How many minutes of downtime does that allow in those 30 days?