Latency and the Impact on Enterprise Application Performance

Latency is today the least understood but most important factor in application performance.

Network performance is measured by two totally independent parameters: bandwidth (measured in megabytes per second) and round trip latency (measured in milliseconds).

While the average user is more aware of how bandwidth impacts performance, in the world of broadband connectivity, it’s latency that is actually the bottleneck for the performance of most enterprise applications.

Latency as a Business-Critical Factor

Fast-responding applications lead to better productivity. As network performance improves on a global basis, users today are highly sensitive to even small latencies. A prime example is Amazon, which found that every 100 milliseconds (0.1 seconds) increase in the load time of its website decreased their total sales by one percent.

Your organization likely also has seen increased bounce rates when the website hasn’t loaded at top speed. And if you personally have ever used a laggy application that hesitates when performing specific operations, you know how extremely disconcerting - and frustrating - it can be.

The latency problem is made worse by “chatty” application protocols or naive programs that were designed by failing to take latency into account.

A Real-World Example

Consider a simple straightforward script that deletes a folder. The script recursively deletes all the files in the folder, and then deletes the folder. Deleting a file is a very quick exercise, around one millisecond (1 ms).

Now, let’s say we have a folder with 10,000 files.

The script running over a local file server would perform quite quickly, deleting the 10,000 files in 10 seconds.

But the script would not perform as well if it were running over a file server located in a remote location with a round trip latency of 80 ms (a typical latency for distributed organizations in the U.S. reaching from coast to coast).

Now the operation takes 1 ms but there is another 80 ms round trip latency on top of it. Therefore, the same 10 second deletion process would take… 13.5 minutes!

That’s 81X slower. Talk about a monster productivity killer.Latency Solutions

What could an organization do to overcome the effects of latency on this deletion script? One solution would be to parallelize the script so that it deletes multiple files at once. Instead of a single thread, we could use 10 threads, which would result in a 10X performance gain and a decrease of the total time spent to 1.3 minutes. Still not great.

Continue reading this article →