The SSD is Key to Economic Flash Arrays
Robin Harris of StorageMojo penned an interesting post yesterday, asking Are SSD-based Arrays a Bad Idea?
The premise of his piece was that the traditional 2.5”/3.5” HDD disk form factor that SSDs use today is largely a byproduct of rotating hard disk history, and if you were starting fresh to build an all-flash array, you just wouldn’t use that: “Think: if NAND flash storage arrays were being developed today, what is the chance that we’d put the flash into little bricks and then plug a bunch of them into a backplane? So why do it now?”
This post caught our eye for an obvious reason: Pure Storage did start “fresh” to build an all-flash enterprise storage array, and we did decide to use the SSD form factor, after quite exhaustive looks at all the other options. Quite simply, we found that SSDs are the most efficient and economic building blocks from which to build a flash array. Let’s explore why.
Definitions: flash appliances vs. flash arrays
First off to level-set, there are three major approaches to all-flash storage devices:
- Build an All-Flash Appliance using raw NAND chips. In this approach, raw NAND chips are procured, and the flash appliance vendor builds 100% custom hardware to handle functions of the flash controller (flash management, wear leveling, garbage collection, error correction, etc.), typically using custom ASICs or FPGAs. This approach is certainly a viable one, and is present in the market from vendors such as Texas Memory Systems and Violin Memory.
- Retrofit an existing HDD-era Disk Array with flash SSDs, essentially unplugging HDDs and plugging-in SSDs. Recent “all-flash” versions of EMC’s VNX and VMAX are examples of this approach, however even the vendors who offer such solutions are quick to market these as transitionary, as these arrays were fundamentally built for HDDs, and lack significant optimization for flash.
- Purpose-build an All-Flash Array with SSDs. Several next-generation array vendors, including Pure Storage, have taken this approach, leveraging the off-the-shelf industry-standard economics of SSDs, but optimizing their software and system design from the ground-up for Flash. This category is ultimately a sliding scale…some vendors do a lot of work to be purely optimized for flash (pun intended), while others leverage many HDD-centric technologies and approaches to get to market faster, and thus slide down the scale towards #2.
Since approach #2 is viewed by most as a stop-gap approach (even vendors who sell such devices), this post will debate the relative merits of approach #1 vs. #3.
Why SSDs are the most efficient architecture for building an all-flash array
We believe that the SSD packaging of flash is the best way to build a flash array, delivering massive performance while balancing efficiency, agility, and most importantly cost. Here’s why:
1. It’s all about industry-standard economics. To bet on a raw-flash architecture, a vendor has to make a bet on 100% custom architectures…and be committed to out-engineering the broader market and ecosystem. Pure Storage made the opposite bet: we bet that the way to get ahead was to bet on the well-proven x86 architecture, and to bet with the “sweet spot” of the flash market: industry-standard flash packaging (i.e. the SSD). Greater than 95% of flash goes into consumer applications (sticks, cameras, phones, tablets, and laptops), and the consumer flash ecosystem will always be on the fore-front of pushing advances in cost/capacity. Consumer flash just has to be managed well to add levels of resiliency, longevity, and consistency in performance, and that’s where our unique IP comes in in packaging industry-standard SSDs as an enterprise array.
2. Integrated controller = rapid innovation. The second thing to realize is that the heart of this conversation is all about who makes the flash controller. The flash controller is the chip that sits close to the flash and does the heavy lifting around managing the raw flash at the chip level: writing data, managing deletion and garbage collection, and handling all-important error correction. When an all-flash appliance vendor buys raw flash, they also write their own flash controller, generally in an FPGA. While this used to be straightforward with SLC flash, as flash lithography dips below 20nm and MLC or even three-level cell designs become common, the level of sophistication of the flash controller sky-rockets. And, when the flash changes every 9-12 months, you have to do it all over again and re-write the controller every release. For this reason you are already seeing a vertical integration happening in the flash market, and we believe that long-term the multi-billion dollar flash fabs themselves will own the controller layer. So quickly the “advantage” that you get by building your own controller turns into a rather heavy boat anchor, holding-back quick product cycle turns. At Pure Storage we want to be able to constantly keep up with the innovation in the flash market to be able to deliver bigger, faster, and at a lower price every single year…and betting on high-volume integrated controllers in SSDs is the only way to achieve that.
3. Hot-swappable components are key for availability. While vendors in the fully-integrated appliance approach can get to some impressive densities, what they sacrifice to get there is serviceability. They can deliver appliances with redundant controllers, I/O cards, and flash modules, but hot-swapping them becomes difficult or impossible given their dense proprietary hardware designs. What good is a “highly-available” array that then needs to be powered-down to swap-out the failed controller? Or a design that ships flash on DIMMs that can’t be replaced from the front of the device? Some vendors have suggested that they can slide their device out of the rack, pop the top, and then swap controllers or flash DIMMs while the array is running…but that’s just a little too reminiscent of playing Operation for us….bzzzzz….you lose (your data). Pure Storage bet on industry-standard form factors to provide online serviceability without downtime: stateless hot-swap controllers, separate expandable storage shelves, and hot-swap drive carriers – all of which can be serviced without downtime. Pure Storage also employs data reduction techniques (inline deduplication, compression, and pattern removal) to deliver data storage densities that exceed these custom “sealed box” flash approaches with traditional hot-swap components.
The SSD is here to stay…but the form factor will certainly change
Hopefully the points above make it clear why we believe that the industry-standard SSD is the right flash building block to bet on….but we do think one thing will change quickly: how that SSD is packaged. If you open a standard 3.5” or 2.5” SSD, one thing becomes immediately apparent: there’s a lot of air inside. Apple has already shown the PC/laptop world that to get to small form-factor designs, the standard-sized SSD has got to go, so there is currently an industry debate on what the next-generation low-profile SSD form factor will be. Whatever it will be, it will become industry-standard, and we’ll use it.
Net Net: The SSD is the key to economic flash arrays
Net net, by using the industry-standard SSD as the building block of our all-flash enterprise array, Pure Storage has proven that the all-flash storage can be delivered with break-through performance, at a cost below traditional spinning disk. In that end, that’s the ultimate existence proof: all the custom-hardware flash appliances have failed to reduce cost enough to be mainstream, while Pure Storage has pioneered an approach of marrying industry-standard flash SSDs with proprietary software delivering resiliency and data reduction…the result is an all-flash array that anyone can afford. Drop us a line if you’d like to give one a spin (so to speak…it doesn’t actually spin ).