Torus: Because the world needs another distributed storage software solution

The Storage Architect Last week the CoreOS project announced the release of Torus*, a scale-out storage platform for container clusters built and orchestrated by Kubernetes.

At this point in the project’s development, the software isn’t production ready and still being heavily developed. Over the last few weeks, scale-out storage solutions have been presented at Storage Field Day 10 and been part of the discussions at the recent SolidFire Analyst’s Day. With so many solutions already on the market, do we really need another one?

The CoreOS blog post page justifies the need to develop a new scale-out storage platform, claiming “traditional” storage solutions don’t meet the needs of large clusters of “inexpensive” machines. Its key tenet is to follow the self-proclaimed “Google Infrastructure for Everyone Else” or GIFEE approach to application development. It’s a fair point that older storage arrays and appliances can’t meet the needs of modern application deployment. Storage provisioning was slow and cumbersome by modern standards and was done because the hardware wasn’t as commoditised as it is today. Solutions were bespoke and managed in a structured manner to deliver reliability and high availability.

However, we have plenty of solutions that can meet the requirements of today’s applications. Platforms like SolidFire are already fully driven by API and will no doubt directly support containers (and can do via OpenStack and Cinder). At SFD10, Hedvig demonstrated fully working Docker integration, as did Datera with Docker Swarm (link here). There are numerous others out there too.

It’s somewhat disingenuous to dismiss the entire storage market and imply that none of the solutions available today can work successfully with container-based application deployment. The principles behind Torus are: extensibility, ease of use, correctness and scalability (check out the article for a definition of what these terms mean in this context). I would argue that almost every (scale-out) storage platform today can match those attributes as well as being further down the line in their development.

What I can’t see in any of the details of Torus so far is a discussion on the way throughput and performance will be managed. In fact the GitHub page for the project states specifically that “speed” is a secondary objective at this point in time. Presumably “speed” refers to both throughput and performance, and in reality if it isn’t designed in from the beginning, retrofitting it isn’t really an option.

The Architect’s View

Torus, to me, looks like another “not invented here” project that wants to re-invent something that’s already out there and working. Now, I’m all for competition, and it could end up being a good prospect, but claiming none of today’s solutions are suitable is just plain wrong.

The more worrying thing for CoreOS is that the project could start to head down the OpenStack model, where more and more side projects are created that divert time and attention away from fully developing the core products. A better approach would have been to develop storage standards as part of the Open Container Initiative and allow existing storage providers to adapt or develop their solutions to an open and agreed standard.

I think CoreOS does want to control and manage the entire platform and become, piece by piece, another OpenStack. Over time Torus will probably be mostly deployed with CoreOS but I can’t see it getting widespread traction, not with so many other competing solutions out there.