ANSWER:

It is a matter of interpretation. You are not doing anything wrong. Here is how i interpret the graph. It is a distribution plot of x vs y where :

• y is the response time in ms,
• x is the number of requests returned within y ms cumulatively.

Statements based on distribution of request for node(alone) would be :

• 7000 of 10000 requests were returned within ~14 ms
• 2000 of remaining 3000 requests were returned within 14-18 ms
• 800 of remaining 1000 requests were returned within 18-22 ms
• Remaining 200 were returned within 35 ms

But when analysing cumulatively it would be :

• 7000 of 10000 requests were returned within 14 ms
• 9000 of 10000 requests were returned within 18 ms
• 9800 of 10000 requests were returned within 22 ms
• All were returned within 35 ms

Cluster configuration will show faster results than single node as expected.

For the nginx results there will be a fixed amount of delay in processing/load balancing. Looking at the figure it would be about 9-10 ms delay, which is as expected. This does not mean that nginx configuration is a bad performer. It does returns most requests within 30 ms (small difference from other configurations).

You gave concurrent requests as 1000 in Apache Bench, which does not require much load balancing to do as even single node is handling that much load easily. Giving more concurrent connections/heavier processing per request will blow up the graph for single node(it will rise early for lesser number of requests). Nginx sacrifices small processing time to allow more concurrent requests to be handled while still keeping response time reasonable.

user568109 from http://serverfault.com/questions/508007