What they're testing
Whether you've investigated an incident with them, or only know the tool names.
The short answer~30 seconds
Metrics answer "is something abnormal" — aggregated, cheap, long retention, what you alert on. Traces answer "where did the time go in ONE request" — contextual, usually sampled, how you find bottlenecks across services. Logs answer "exactly what happened at this step" — the most detailed and the most expensive to store and search. The natural investigation flows metric alerts, trace localises, log confirms.
The long answer
The most expensive mistake is using logs to do a metric's job: writing a line per request and counting them with a search query. It works at small scale and becomes absurd at large — you pay per gigabyte of storage for a number a counter provides at nearly zero cost. Conversely, using metrics to find a root cause dead-ends, because metrics have discarded per-request context.
What multiplies the value of all three is correlation: a trace id on every log line, and metric exemplars pointing at a sample trace. Then from a spike on a chart you jump straight to the request that caused it and then to its logs, instead of guessing by timestamp. That's the biggest practical argument for OpenTelemetry over three disconnected tools.
On structured logging: emit JSON with consistent fields rather than free-form strings. It sounds like ceremony and it's the difference between filtering user_id=123 in a second and writing a regex during an incident. And keep a budget: logging everything at debug level in production is the fastest way to make the observability bill exceed the compute bill.
What they'll ask next
?What should you alert on?
Symptoms users feel, not causes. "Error rate exceeds the SLO" is a useful alert; "CPU above 80%" wakes you at 3am for something that may be entirely fine. The four golden signals — latency, traffic, errors, saturation — are a good starting set.
?What trace sampling rate is sensible?
Tail-based sampling if your pipeline supports it: keep every erroring or slow trace, plus a few percent of normal ones. A fixed 1% head sample misses precisely the requests you most need to see.
These lose points
- Logging personal data or tokens. Logs are typically retained longer and read by more people than the database.