"OK, I got my Lead Time data in a histogram. I heard that then I use the 85th percentile to determine my SLA. Good to go?"
🛑 Hold your horses! Keep in mind that you're in the world of "small samples", which means that the percentiles you calculate from the data you've collected may differ from the percentile lines in your actual distribution, and often by a large margin of error. So, for example, your data may tell you that the 85% percentile is "12", but in reality that line may be at 8, or 43, or 20... or 12.
🚧 That said, before you jump to the conclusion that what you need is more data, keep in mind that for an asymmetric ("non-gaussian") distribution (such as Lead Time), you will need large data sets (as in hundreds of data points) before it starts to converge. And for really skewed distributions, the number of data points may need to be really, really, large. So, solving the problem by collecting more data becomes impractical; you may need many months just to get enough data points, and by then chances are that your process has already shifted and the data is not representative any more.
But there's a more fundamental problem with the use of the 85th percentile as the basis for SLAs. To be accepted, your SLAs must be something your Customer finds fit-for-purpose, and that's a criteria they decide; it doesn't come from your histogram.
So, choose your SLAs based on your understanding of your delivery capability and your Customer's fitness criteria (not by just using math).
Rather than spending a lot of time collecting lots of data to get an "accurate" reading, work with the data you have, use it to understand your process; talk to your Customers to understand their fitness criteria, and use both things to introduce concrete improvements to your delivery process.
PS: Troy Magennis recently wrote about the (lack of) significance of the 85th percentile, and expanded on the origins of it becoming "folklore". Comments by David Anderson and Alexei Zheglov added further history and nuance. You can find the post here.
Comments