We live in a world of rapidly moving technology trends, and fast-paced consolidation. What will the data centre look like in the coming years, as we begin to deploy these new concepts? The signs are that there is plenty of room for improvement in the way that we manage our equipment and vendors are inventing ways to help us.
We have spent the past few years consolidating servers to virtualise our computing capacity. Some of us have also virtualised storage through the use of storage area networks (SANs). A few people are also starting to address data centre networking, using high-speed, lossless ethernet to run everything from fibre channel through to iSCSI, and virtualised server traffic over the same backbone.
But the real magic will happen when we begin to put all these things together. For example, storage management tools have focused largely on the management of physical units and their support of various logical unit numbers (LUNs) in the past. They haven't particularly addressed virtualised servers. Similarly, the hypervisor management software used to shunt virtual machines around hasn't been developed with storage management in mind. More often than not, administrators will buy separate tools to accomplish these tasks well.
And then, there are the links that tie them together. Network management software focuses more on maintaining link quality, rather than acknowledging the other two pillars of the data centre environment.
Atop all this sits the application portfolio, which is what the computing infrastructure is there to support in the first place. And yet, for the most part, the datacentre infrastructure isn't intimately aware of application performance. Just ask the datacentre manager for a service level agreement on email delivery or screen response time, and see what he says.
The consolidated data centre will change this. The idea is that all these pillars become highly aware of the others. Storage resource management software will begin to understand the data that particular computer servers need to maintain performance. Management of the virtualised servers will be tied intimately to application performance. And it will be done, ideally, in an environment where everything has been consolidated, and where inefficiencies and over-provisioning problems have been driven out of the system.
It's a nice idea. Cisco has been working on it extensively as part of its UCS initiative (and annoying server vendors in the process, by muscling into their markets). Conversely, server experts such as HP (with its acquisition of 3Com and 3PAR) and Dell (with its $1 billion acquisition of Compellent, and other purchases such as Equalogic) are muscling into the networking and storage markets. And IBM? Well, IBM has pretty much two of everything.
The reason that these vendors are growing through acquisition and fleshing out their portfolios is because they want to be able to offer all parts of the data centre portfolio to their customers, and they want to be able to integrate them effectively to add value.
Some vendors have gaps in their portfolios. Cisco's home-grown storage presence is non-existent, for example. But in such cases, they are serving the market through partnerships. Cisco and EMC have been in bed together for some time. This gives Cisco access to EMC's storage and virtualisation expertise, while EMC gets to embrace Cisco's networking prowess.
This idea of a consolidated, high-performance data centre, in which storage, servers, virtualised operating systems and applications all talk to each other over a single, standardised high-speed transport layer is utopian, but deals and partnerships such as these show that the vendors are committed to making it happen. It will take a while to emerge, especially as cash-constrained companies still reeling from the financial crisis are unwilling to rip and replace legacy equipment.
IT professionals working in these environments would do well to watch for this coming trend. It will change the required skill sets necessary to tackle administrative tasks in tomorrow's data centre. Will your CV reflect what is needed?
Yesterday I attended Cloudstock, a pre-conference event put on by Salesforce.com as part of its Dreamforce event in San Francisco.
One of the speakers was Ryan Dahl, the author of node.js. He is smart and opinionated and gave a though-provoking introduction to his project. He has a couple of core arguments. The first is that I/O is slow, and therefore all I/O calls should be non-blocking.
The point here is that access to the disk or the network is hundreds of times slower than access to memory. A few milliseconds may not matter much in a single-user desktop application, but if you are coding a server with many concurrent users it becomes important.
You see this problem magnified many times in web pages that allow calls to remote servers to block the loading of a page. Web developers have learned to use callbacks for things like database queries that can take a while to return. Clients like Flash and Silverlight have no other mechanism for web service calls; doing these via callbacks is built in.
Dahl's second point is that threading does not scale well. He showed a slide comparing the performance of Apache to Nginx as the number of concurrent connections increases. It showed Apache's memory usage growing while that of Nginx hardly changed. Dahl said that Apache uses a separate thread for each connection, but Nginx uses an event model, which is why it performs better under heavy load.
Dahl says threads consume too many resources and complicate coding because you have to worry about synchronisation. He adds that Nginx will "probably take the place of Apache" because of its better performance.
Node.js is a binding for V8 that runs on Mac, Linux, Solaris, or with a bit of effort Windows. You would typically use it for real-time server applications that need to support many concurrent users. Dahl actually coded a chat server during the session, though our attempts to connect were defeated by the firewall.
In an aside, Dahl calls HTTP "the worst protocol on earth". He adds that we are now stuck with HTTP/1.1 for ever. Node.js uses Transfer-Encoding chunked to overcome some of its limitations.
Another of his points is that we are too much tied to web servers. "There's all these weird concepts in web stacks, why not just speak HTTP to things?"
Node.js has great performance, but it is low-level and further wrappers and libaries are appearing to make it more productive to use. What might it become in future? After the talk had ended, Dahl took some further questions and admitted that he would love to see it become the next PHP.
A great session; and whether or not you agree with all Dahl's remarks they are a welcome reminder that despite all the effort that has gone into the computing platforms we use constantly, there are things that could be done much better.
Node.js is in effect public domain software, says Dahl. It is not fully done yet, but in 6 months or so the API will be stable. Well worth a look. So, for that matter, is Nginx.