Storage infrastructure wakeup call

The storage solutions you run today face many challenges, of which the explosive growth in data volumes is just one. In a recent survey of Register readers, we asked what currently stresses your storage systems, and what needs to be done to ensure your internal customers get the services they need. The overriding conclusion was that changes in business and technical demands mean you will almost certainly have to modify how you build and manage your storage environment in the future.

Doing things the same old way probably isn't going to cut it.

Turning to specifics, confirmation from the survey that rapid data growth is a significant issue for most is hardly news. Arguably, of more interest are the other sources of stress which are often overlooked, but acknowledged when prompted for (Figure 1).

Figure 1 - Storage infrastructures are under pressure

Impact of virtualisation

A particular issue highlighted by three quarters of our respondents is the impact of infrastructure virtualisation. Many organisations now run an expanding range of business services, even ‘business critical’ applications, in virtualised environments, so this pressure is unlikely to fade away anytime soon.

One requirement here is concerned with the storage implications of optimally managing virtual machine (VM) images. This could be in a development, test or devops environment where allocating virtual resources needs to be dealt with quickly and efficiently. But there’s also the problem of live environments – web/app server farms, virtual desktop estates, and so on - where being able to respond rapidly to new requirements and deal with tactical issues such as boot storms can lead to a bottleneck of the storage infrastructure if you are not careful.

There is then the challenge of keeping the storage side of things working well as VMs move around the IT infrastructure. Working with storage arrays that may not have been designed with virtualisation in mind takes effort. But storage solutions are being developed to tackle these challenges, with software approaches such as the newly announced VMware VSAN 6 and Virtual Volumes now available. Meanwhile Microsoft is developing Storage Spaces and its associated administration tools and OpenStack is expanding its storage management capabilities.

Flexibility and responsiveness

Another factor impacting storage systems is the recognition that they need to be more flexible when it comes to meeting service level requirements of business services. Gone are the days when an application’s service requirements were defined at project inception and these were fixed for its entire lifetime. Instead there is a recognition that the speed and protection requirements of data do change over time, sometimes quite rapidly. In order to ensure SLAs are met without having large amounts of under-utilised resources, it is essential that storage systems can be managed more dynamically as business requirements vary.

New and changing application needs

Building on this, new and changing application needs, acknowledged by three out of four of our respondents, includes a number of factors which impact on storage. These range from escalating transaction throughput and the need to handle lots more multi-media as a result of ‘digital business’, through to the impact of enhanced workforce collaboration (again giving rise to more unstructured data). Data generated by potential activity in the Internet of Things (IoT) domain could also become a factor with the need, for example, to capture fast-moving data feeds from sensors and other equipment.

And we mustn't, of course, forget the growing appetite among business users for more data and insights to help run the business on a continuous basis, which drives more activity in the area of analytics – even ‘big data’. The increasing demand for real time or near real time analytics places additional pressures on storage.

Escalating business expectations

In addition to such tangible emerging requirements, the survey highlighted that softer, people related challenges are also likely to cause a rethink of how your storage infrastructure is built and operated.

For example, almost 80% of respondents say changing business expectations for always on / anywhere access to data is creating pressure. This reflects the fact that IT systems, and the storage underpinning them, are now such a fundamental part of the business operations infrastructure.

Greater emphasis on risk management

However, the most highly visible non-technical challenge highlighted by the survey is around the security and governance of data. In particular, a huge percentage of respondents, over four out of every five, report an increase in senior exec emphasis on the management of risk.

It’s probably fair to say that the attention being paid to ‘data protection’ and ‘security’ has never been as high as it is today. Barely a day goes by without another story of data leakage hitting IT websites such as The Register. Some of these stories concern data protection lapses big enough to capture the attention of the national and international press. The thought of having to explain how a major incident like that happens is enough to keep any IT professional awake.

One of the biggest headaches here is dealing with data fragmentation. Managing and protecting information as it becomes increasingly distributed across central and departmental systems, even the cloud, challenges many of the assumptions underpinning traditional approaches.

Time to modernise?

Against the background we have been discussing, it is unlikely that building and operating storage systems in the way you have done in the past will be sustainable in the future. More of the same just won’t suffice in terms of cost, management, security or flexibility. Storage technologies have evolved rapidly in recent years, however, so there are many options to enhance the infrastructure and tooling you have in place today. This is just as well, as few organisations can afford a complete storage infrastructure ‘rip and replace’ and so have to find a way to bring in new platforms that complement and extend existing capabilities.

Want to know more?

For more ideas on this, we would encourage you to download the full report, which is available here.