I’ve been working in an Electrical and Electronic Engineering department for more than three years now, but I admit that it’s only quite recently that I find myself properly understanding the difference between latency and throughput. These are terms that are used all the time when talking about an electronic circuit. The distinction between them is a bit subtle because both “low-latency” and “high-throughput” can be used to describe a circuit that performs well.
Latency and throughput are attributes of any sort of flow – whether that’s a flow of groceries moving along a conveyor belt, or raw materials turning into goods as they flow through a mass-production factory, water flowing through a hose pipe, or data packets flowing through a computer network, or signals flowing through an electronic circuit. In this post, I’ll use the commuters flowing up an escalator on the London Tube as my example.
Here are the basic definitions.
- The latency of the escalator is the time it takes for a single commuter to get from the bottom of the escalator to the top.
- The throughput of the escalator is the number of commuters that step off the top of the escalator per minute.
It’s interesting to note that commuters care only about minimising latency (because their objective is to make their journey as short as possible), but the Tube operator cares only about maximising throughput (because their objective is to get as many passengers through the station as quickly as possible).
Here are some changes that could be made to the escalator, and how they would affect its latency and throughput.
- If the escalator were sped up, then its latency would decrease and its throughput would increase.
- If the escalator were widened, so that more commuters could fit onto each step, then its latency would not change and its throughput would increase.
- If the escalator were made shorter (and hence steeper in order to make the same vertical ascent), then its latency would decrease and its throughput would not change.
It’s interesting to note that some of those changes affect only latency, some affect only throughput, and some affect both simultaneously. I think it is this intertwining between latency and throughput that confused me until recently, but hopefully this post has cleared all that up now!
Edit. Just after writing this post I came across this text in an article I was reading in the EURASIP Journal on Image and Video Processing …
… which is a good example of the confusion between latency and throughput. (The authors are defining ‘latency’ here, not ‘throughput’.)