Last year, same place, same Vision conference, I spoke with
Mike Palmer, former SVP and GM for datacenter products and now EVP and chief product officer of
Veritas Technologies LLC, about
Object Storage and the huge lack of solution from the leader of Data Management and the need to build something.
Software-Defined Storage is almost synonymous of Veritas as the company is recognized for the leading platform independent open system file system and volume manager since early 90’s.
But even when the pioneers of object storage launched their company around 2005, I mean here Caringo and Cleversafe, Veritas didn’t investigate in that direction. Really lack of anticipation and vision? Not sure as Symantec owned the company at that time for more than a decade and finally froze some developments and initiatives.
Announcement and context
One year later and just 16 months following the split between Symantec and Veritas, Mike Palmer on stage announced
Veritas Cloud Storage (VCS), the company’s answer to the need of high capacity and high resiliency dedicated to unstructured data, I mean here the object storage product from the software giant we expect for several years.
Interesting to notice that the term “object storage” is not mentioned in the
press release that announced the product as the term generates very often mixed feelings.
Product management did a great job except for one thing: the name of product. We understand that object and cloud could be exchanged in that case but as a common practice many people name product by acronym. NetBackup is known as NBU, Backup Exec as BE or Veritas Cluster Server as VCS. And this one clashes with Veritas Cloud Storage. I have to admit that InfoScale product line has introduced a new name InfoScale Availability to replace Cluster Server. So let consider this as a detail, we have to also adapt ourselves.
The new product manager for the product is Chad Thibodeau who has spent a few years at Cleversafe, the object storage leader by far.
Product detail
Also named on stage the “
ZetaScale” product, VCS is a pure Veritas development with hopefully several similar concepts and ideas we find in other object storage products but also many differentiators.
It runs on commodity hardware, bare metal, virtualization or even with container leveraging standard components and Linux OS. Every node is deployed with a local file system (ext2, XFS or VxFS among others) for internal drives and every drive is independent.
The philosophy chosen by the engineering team is based on consistent hashing with a columnar data store backend and the data pivot size is 64MB.
In term of data protection, metadata are protected by replication and data by replication or erasure coding. In that case, it’s based on Reed Solomon implemented with the Jerasure library.
For access methods, the product is very rich, probably one of the richest on the market, with NFS and SMB, S3, Swift plus a http REST API, MQTT and CoAP (for IoT), JDBC connector for BI and plugins for Apache projects such HDFS, Kafka and Storm.
The design offers a choice between eventual or strong consistency models when you configure the product.
VCS provides also a workflow engine and implements several features such data indexing (hopefully), classification, analytics and categorization without the need to add any third party product on top of it.
Market climate
With more than 20 vendors trying to find a niche and a position for their product, the object storage market segment has real difficulties to grow and exist. Recently, even backed by famous investments firms or entities such A16Z, Dell Ventures and Mayfield Fund among others, two companies died,
Coho Data and
Formation Data Systems, event that confirmed a probably too important number of actors for a limited number of business opportunities.
It’s probably not even a market but more a technology that helps to address limits of “classic” architectures and reduce significantly the cost to store data. It’s not a product either like a NAS even if some companies were built on that and try to push that approach. Remember Data Domain, as the first company in enterprise-class data deduplication, they were able to build a product from a feature. And it's not an architecture also like SANs are, where you select components, design a network and finally deliver an orchestrated connected IT environment.
Object storage vendors have also created their own difficulties surfing and leveraging the SDS and commodity approach. The result is all products are very similar and suffer from lack of differentiators. Imagine a blind test, like we do for yogurts, and you will be surprised that users can't make any difference among offerings. It's true today when you ask for S3 access, erasure coding, geo-aware, commodity based… finally lots of them can do the work. Is it what we call a maturity plateau? Or an other sign of tough times?
In that aspect open source plays its role and has put pressure on commercial offerings. In two words, why pay for software you can download? So differentiators and ecosystem are key. Probably use cases could be the last opportunity to survive, potentially oem as well if you find the right partner, as large vendors now offer object storage, it was not the case a few years ago. I invite you to read 2 articles about that
1 and
2.
Finally, as Veritas is already a leader in several data management segments, the market penetration for this product should be pretty straightforward as the natural extension should be one of the messages, something like up-sell, cross-sell.... For sure, for new projects and existing Veritas customers, VCS will be the de facto and default choice, no need to check elsewhere. For new customers, the battle is on but as the leader in data management with a huge portfolio effect, the game should be over pretty soon as well, we already saw that from other vendors. The product should soon skyrocket. Independent small object storage vendors will have some difficulties to survive especially for ones who have just one product and no real differentiators. Let’s start counting…
ProphetStor Data Services (www.prophetstor.com), an emerging player in Software-Defined Storage, just announced the StellarFlash Arrays product line composed of All-Flash (AFA) and Hybrid Flash arrays (HFA) running on commodity servers.
