Storage admins inherently understand the “need for speed” in storage: IOPS and latency.  Sophisticated admins now routinely track their IOPS growth alongside their capacity growth, understanding that scaling both these dimensions together in their storage systems is non-trivial, and that sometimes capacity and IOPS growth diverge. Admins watch the IOPS load grow in their environment, and try to predict the coming wall where their current storage architecture simply can’t scale to meet the performance needs of their applications.

However, often missed in this performance growth forecasting is a hugely significant compounding factor, a factor that is dramatically accelerating the demands on performance storage: the increasing randomization of I/O — a trend we’ve taken to calling the I/O blender.

What is causing increased randomization?  The primary culprit is consolidation.  Consolidation can take many forms: server virtualization, desktop virtualization, database consolidation, or the adoption of cloud computing models.  While cloud computing may be the buzzword du jour,  in fact the move toward improving IT efficiency through consolidation has been ongoing for well into the past decade. As applications and systems are consolidated and virtualized, dedicated storage pipes and dedicated disks for specific workloads go by the wayside. But a hard disk can only serve one master at a time; while it is serving one I/O for one application in a virtualized architecture, it is keeping another, or likely many others, waiting. Unfortunately for your disks, most of your top IT initiatives today involve consolidation, and hence randomization of I/O:

1. Virtual server infrastructure. A single application and OS image do a lot of work to serialize I/O to behave nicely on disk. Years of tweaking by the OS, volume manager, file system and database layers have all yielded a highly-optimized I/O stream that seeks to avoid disk trashing and contention when possible. Now take 10-20 of those “well-behaved” virtual machines and run them on a single physical server and storage volume. The result? The I/O Blender.
2. Virtual desktop infrastructure. While virtual servers present severe I/O challenges, the random I/O issues with virtual desktops can be even worse. Several factors compound virtual desktop storage challenges: extreme consolidation, highly-variable I/O and cost pressure. VDI environments are extremely consolidated with tens or hundreds of users often share a single storage I/O pipe, driving up the random I/O quotient. But because VDI implementations have to compete on a dollar-for-dollar basis with physical desktops, there is little room for expensive storage infrastructures to meet this I/O demand. To further complicate things, VDI I/O workloads are “lumpy,” where many users often perform I/O intensive actions at the same time (e.g., morning logins, patch applications, antivirus scans, etc.). All these factors create significant I/O challenges and frustrated end-users. Again, it’s the I/O blender.
3. Database consolidation (analytic and OLTP,  SQL and NoSQL workloads). Databases are getting bigger and doing more tasks simultaneously — such as when “real-time” analytics workloads share the same online data with traditional OLTP (historically the principal driver of random I/O). Interestingly, even analytic workloads that are designed to be highly sequential in their use of disk tend to become randomized whenever more processing requests are intermixed. As data grows and DBAs look toward consolidation versus partitioning, the I/O blender effect is multiplied.

Fortunately, flash memory has arrived as a powerful weapon to combat both the dramatic growth and increased randomization of I/O. But it’s important to note that flash is a technology and not a solution. Flash, like its predecessor disk, has its own set of idiosyncrasies and challenges that must be managed. Pure Storage is working to close this gap and make leveraging flash to solve the issues of the I/O blender easy and efficient. Stay tuned!