DaveWentzel.com All Things Data
Queuing Theory and Knee of the Curve
Queuing theory states that utilization has a direct effect on queues, and because q's are directly related to response time (in fact, q length is part of the response time equation), utilization thus has a direct effect on response time.
Supermarket analogy...no q, good response time and service time at 3am. At 5pm a longer q with worse response time and service time to complete the task. In the latter the utilization of the cahier was much higher, which had a direct effect on q's and therefore on your overall wait or response time.
When we look at the world of computers we find that a CPU whose utilization in a steady state is above 75% has issues. Q's grow exponentially in such a state. Linear growth, the even, incremental growth of utilization, is preferable. Linear growth does not take palce in CPUs with utilization factors over 75%.
Graph of Linear Utilization Growth here
Graph of exponential Q length vs utilization growth here
In such a system, a point is reached at which the growth becomes exponential, rising geometrically and straight up to infinity. This is known as the asymptotic point or the knee of the curve.
Notice that the curve in the second graph is the same as the q length graph. This is why you never want to run your CPUs in a steady state of over 75% utilization. Short periods are fine, but the longer the processors are run at this utilization the more negative impact you will see in terms of q lengths and response time.
The relationship amoung utilization, qing length, and response time is one of the most important in sizing and one you should consider when you select the number and speed of CPUs for your system.
For example, you are sizing a system and you calculate that your system will produce anticipated total processor utilization factors of 180%. It would be better to buy 3 CPUs that will run at about 60%, keeping the utilization 15% under the knee of the curve, than to have 2 CPUs running at about 90%, which would make the utilization 15% over the knee of the curve.
The knee of the curve also applies to other system components, although the knee of the curve will be different.
Disks = 85%
For example, a 9GB disk should not store more than 7.6GB of data at any given time. Observing this limit allows for growth and more importantly, helps keep down response times. A full capacity disk will have longer seek times, adding to your overall response time.
If a disk has an IO capability of 70/sec you would not want a constant IO arrival rate of more than 60. You will then have a reserve capacity for peak utilization periods.