How to Define SLAs and SLOs That Actually Work

Someone asked me in an interview recently what p99 latency I'd target for a product that had gotten slow. I almost gave them a number.

It would have sounded confident. It also would have been made up. You don't actually know the right target until you've talked to the people who feel the slowness and the people who'd have to pay for the fix. I said that instead, and it turned into the more interesting conversation.

That question is really about SLAs and SLOs, and most teams get them wrong in the same few ways. Here's how I set them, after doing it on teams where the numbers actually cost something.

Get the three letters straight

SLI, SLO, and SLA get used like they mean the same thing. They don't, and the difference changes what you do on Monday.

An SLI is the thing you measure. p95 latency on the booking endpoint. The share of requests that don't error. Availability over a rolling 28 days. It's just a number.

An SLO is the target you hold yourselves to for that number. "p95 under 300ms, 99% of the time." It's internal, a promise the team makes to itself.

An SLA is the promise you make to someone outside, usually a customer, often in a contract, usually with a penalty if you break it.

The order they come in matters. You measure the SLI, set an internal bar with the SLO, and only then make an external promise with the SLA. And that external promise should always be weaker than the internal one. More on why in a second.

Start from what the user feels, not the dashboard

The most common mistake is setting objectives around things that are easy to measure instead of things people feel. CPU under 70% is easy to graph. It's also not what your user notices. They notice whether the page loaded, whether the booking went through, whether the calendar showed the right slots.

So start with the one or two flows that actually matter for your product, and write the SLI around what the user goes through in that flow. For a calendar, it's whether a practitioner can open their day and book a slot without waiting. Everything else is a supporting metric, not an objective you defend.

Measure it from where the user sits, too, when you can. Server-side latency looks fine right up until the network and the frontend add their share. The number that matters is the one the person actually waited through.

Keep the promise looser than the target

If your internal target is 99.9% and you promise customers 99.9%, you've left yourself nothing. The first bad week and you're in breach, paying credits, on a call explaining yourself.

Set the external promise below the internal one on purpose. Run to 99.9% internally, promise 99.5% externally. That gap is your buffer. It's what lets you have a rough week without it turning into a contract problem. Teams that skip this either quietly lower their real ambition to protect the promise, or spend every week firefighting to defend a number they set too tight.

Percentiles, and the real cost of more nines

Use percentiles, not averages. An average of 200ms can sit on top of a p99 of four seconds, and the p99 is the customer who's done with you. p95 and p99 describe the tail, and the tail is where the pain lives.

Here's the part that interview question was really poking at. Every extra nine, and every step from p95 to p99 to p99.9, costs more than the last. Often much more. Going from p95 to p99 might mean rewriting how you cache. Going from p99 to p99.9 might mean re-architecting a whole path. So the target isn't taste or ambition. It's a spending decision. You pick the tightest number the users genuinely feel and the business will genuinely fund, and not one notch tighter. Chasing nines nobody asked for is how a whole quarter vanishes into work no customer could perceive.

The error budget makes the whole thing useful

This is the idea that made SLOs click for me, and it comes straight from Google's SRE practice. If your SLO is 99.9% availability, then 0.1% is your error budget. That's not a failure you apologize for. It's the amount of unreliability you're allowed to spend.

What makes it useful is that it turns reliability from an argument into a number. Budget healthy? You can ship faster and take more risk. Budget spent? The team slows down and pays into stability until it recovers. Product and engineering stop debating whether to prioritize features or reliability, because the budget answers it for them. That's a far better conversation than two people trading opinions in a planning meeting.

Set the number with the room, not at your desk

This is the thing I actually said in that interview. You don't set an SLO alone. Engineering knows what each nine costs to build. Product and business know what the customer impact is worth. The right target sits where those two meet, and you only find it by putting both in the room.

So treat it less like picking a number and more like a short negotiation. Bring the current baseline. Bring a rough cost for tightening it by a step. Let product bring the customer stakes. Land on the number everyone agrees is worth the effort, write it down, and look at it again in a quarter. An SLO that engineering sets by itself gets ignored by everyone who didn't help set it.

An SLO you don't act on is just a chart

This is the one people skip. If breaching your SLO doesn't change what the team does that week, it isn't an objective. It's decoration.

Decide the response before you need it. Does red mean the team stops feature work until it's fixed? Does it page someone at 2am? Does it start a conversation with the customer before they start one with you? Write that next to the target. The number only has teeth if there's a "then what" attached to it.

Where to start

None of this needs a platform team or a reliability org to begin. Pick one flow that matters. Measure what the user feels going through it. Set a target you can defend to both an engineer and a customer, leave yourself a buffer under the promise, and decide what you'll do the week you miss it. That's a working SLO, and it's more than most teams actually have.

And when someone asks what number you'll hit, it's fine to tell them you'll find it with the people who feel the slowness and the people who'll pay to fix it. That isn't dodging the question. On a real team, that is the answer.