VAST Data changes things radically

VAST Data, just went out of stealth, promises are ambitious with the plan to shake the file storage market segment with a radical new approach i.e delivering a full flash high-end NAS at a lower TCO than HDD.

The company founded in January 2016 has raised so far $80 in 2 rounds from TPG, Norwest Venture Partners, Delll Technologies Capital, Goldman Sachs and 83North. Founders came from different areas, the CEO Renen Hallak, was most recently at EMC and before at XtremIO and Jeff Denworth, VP Products, came from CTera Networks and before DDN and Shachar Fienblit, CTO, was CTO at Kaminario. Interesting we met Jeff and Shachar during the IT Press Tour in Israel in November 2016 and they both worked at that time respectively for CTera and Kaminario. So there is magic here as they claimed to be co-founders at a time they work for other companies, I probably don't get all details on the genesis of the company.

The team wishes to revolutionize the storage and breaks the storage tier approach with only 1 tier full flash. It is illustrated by the famous pyramid, shown below, with a hierarchy based on latency or capacity.

But to build such product you need some hardware components associated with key software features. Here first elements are persistent memory with Intel Optane coupled with a Flash/QLC back-end, a high speed fabric with Ethernet or Infiniband to transport the NVMe protocol, plus a wide erasure coding and aggressive reduction to finally destroy the HDD TCO. VAST Data chooses to offer the product as a highly scalable file server exposing NFS and S3.

With one tier all flash, the vision of VAST Data is to offer an universal storage. This is perfectly aligned with the concept I introduced in January 2012 with the U3 model as an Universal, Unified and Ubiquitous storage. Now the difficulty could be to run some applications on NFS or S3 and not on a block device via a disk file system. But it seems that the product will be able to run applications natively within VAST's servers and we'll get details on that in the future.

In term of architecture, the team adopts a disaggregated model that allows independent scalability for access servers and the storage servers, named Element Store. The file service runs on access servers as Docker containers, fully stateless, so very easy to deploy, to scale and move. These servers are not aware of other in the cluster and they don't need to. All data are stored on storage servers composed of Optane and QLC and addressable by any front-end server.

This architecture named DASE for DisAggregated Share Everything goes beyond classic shared-nothing model that finally limits scalability. Such implementation are good for capacity need but can't sustain the performance demand. The Single File System Image is guaranteed by the absence of cache coherency between access servers and their direct connection to storage servers that finally store all state, locks, metadata and of course data.

To extend life of components such QLC, data placement are paramount and the team spent a lot of time developing and optimizing new algorithms. And this is crucial as VAST Data commits for a 10 years warranty for QLC with a 10% support and maintenance per year.

All access servers belong to the same global unique namespace exposed via NFSv3, NFS over RDMA or S3.

The product go to market model relies on 3 options:

• full appliance for both layers access and storage,
• a mix of hw for the storage layer and software for the access service,
• and finally a pure software model with of course restrictions like size of the cluster, it has to be big, business case and deal size. When the cluster get very big, you can be flexible...

For the hardware, VAST Data sells 2U chassis with 4 nodes and 8 x 50Gb Ethernet ports. For the NVMe enclosure, it represents 675TB of flash raw, 18TB of Optane with 4 x 100Gb Eth/IB ports in a 2U chassis as well. So everything is pretty compact. Any combination can be built: 1 access chassis + 1 enclosure to deliver 30GB/s, 250,000 IOPS, 1M metadata operations per second for 1.2PB with 2:1 reduction ratio. For capacity need, you can double the enclosure, still delivering the same performance level for 2PB of capacity. And you can now double the access layer to double performance and offer now 60GB/s, 500,000 IOPS and 2M metadata operations per second.

For data services, VAST Data provides data reduction - deduplication and compression - and protection with wide erasure coding with a super high utilization ratio, snapshots, quotas, POSIX ACLs and LDAP integration.

The team did a very good job for snapshot as they're implemented as indirect-on-write mode, they are continuous, fine-grained, writable, clonable and they don't impact performance.

The product doesn't offer yet replication across cluster, SMB protocol and multiple file system so multi-tenancy could be seen as limited.

This is just brilliant, we'll see now how the market will react in term of adoption and what competition will say...