Debugging with distributed traces: waterfall shapes, context propagation & sampling
A distributed trace shows one request's path through every service it touched. Once you learn to read its shape, you can often spot the bug before you open a single log. This quiz shows you span waterfalls and asks what went wrong: N+1 staircases, stragglers, retry storms, clock skew, and broken handoffs. The last questions cover sampling, which decides whether the trace you need gets kept at all.
Questions
- Not answered. In a trace waterfall, Span B is drawn indented inside Span A and starts after A begins. What does that mean?
- Not answered. A trace's root span lasts 800 ms. Its two direct children last 150 ms and 200 ms. What can you say about the other 450 ms?
- Not answered. A parent span is much wider than its only child, with large stretches of unexplained time on both sides. What does that usually mean?
- Not answered. Two sibling spans share a parent, and there's a gap between them with nothing else in the trace. What could cause the gap?
- Not answered. Why might a trace waterfall and a service's log timestamps disagree about when something happened?
- Not answered. What's the difference between a span attribute and a span event?
- Not answered. A parent span holds many identical child spans of the same length, running one after another and filling most of the parent. What does that usually mean?
- Not answered. Two traces of the same endpoint and code path look very different. One has a staircase of database queries, the other a single short span. What's the most likely reason?
- Not answered. The three child spans below don't depend on each other, and none needs another's output. Yet the trace shows them running one after another. What's going on?
- Not answered. You fix the problem above by starting all three calls before awaiting any of them. What should the parent's duration be closest to now?
- Not answered. A service fans out to five independent calls. Four return in about 50 ms and one takes 310 ms. Why is the parent span 310 ms wide?
- Not answered. A parent span fans out to many independent children and waits for all of them. Which statements are true?
- Not answered. The waterfall shows the same child span repeated several times, with a longer gap before each repeat. What does this usually mean?
- Not answered. A child span seems to start 30 ms before its parent. The child's service is healthy, and its logs show it started only after it received the request. What's the most likely explanation?
- Not answered. You expect a trace to continue from
checkout-apiintoshipping-service. Instead, the trace ends atcheckout-api, and a separate trace starts inshipping-serviceat about the same moment. What's the most likely cause? - Not answered. Which of these, seen in a tracing system, point to broken context propagation somewhere in the request path?
- Not answered. A span named
resize-imagetakes 480 ms of a 500 ms parent, and a developer concludes that resizing is slow. In the code, the span wraps both the resize and a network upload of the result. What does this teach about span durations? - Not answered. The
http.requestspan ends at 120 ms. Asend-emailspan for the same request starts at 150 ms, drawn in its own row and connected by a link instead of nested inside. What does this usually mean? - Not answered. Which of these would make you suspect a span is async handoff work, rather than blocking work the parent waited on?
- Not answered. Six database query spans in one trace share a pattern: a long, equal wait before each query runs, then a short burst of real work. What does that equal wait suggest?
- Not answered. A team samples 1% of traces with head-based sampling. To keep every trace for one customer id, they add that filter to the sampler. Why does this often not work?
- Not answered. Tail-based sampling decides whether to keep a trace only after the trace has finished. What's its main cost?
- Not answered. In an incident review, someone says 'we have no traces for the outage.' What's the most likely reason?
- Not answered. You need to know how often an error code occurred across all traffic last week, by hour. Which kind of telemetry answers this directly?
- Not answered. A user id is fine as a span attribute but dangerous as a metric label. Which of these explain why?
- Not answered. Which of these questions are best answered by reading an individual trace, rather than metrics or logs?
- Not answered. A parent span fans out to 5 independent calls. Each call has a 2% chance of being slow, meaning a 98% chance of finishing within its p99 threshold. What's the probability, in whole percent, that at least one of the 5 is slow?
- Not answered. A trace shows three independent calls made one after another: 120 ms, 260 ms, and 90 ms, for a parent of 470 ms. If they ran in parallel instead, about how many milliseconds would the parent save?
- Not answered. A parent waits on 4 independent children that take 45 ms, 52 ms, 310 ms, and 60 ms. The parent does no work of its own. How long is the parent span, in milliseconds?