Recent comments by VMware's Mark Thiele and Cisco's Chris Hoff are illuminating what the future of the data center will look like and the issues that will need to be addressed.

Read more...

VMware's Thiele articulates a new dynamic vision of the data center with no center.  The result is unprecedented economics, flexibility and the return of the strategic importance of the network within IT.

Read more...

Operational efficiency in the cloud comes in part from automation and orchestration as well as from the outsourcing of management and maintenance of the hardware. While you can’t achieve the latter without cloud or hosting externally, you can realize a lot of the same efficiencies in a traditional architecture just by leveraging existing collaborative capabilities of infrastructure 2.0.

Glenn Gruber of Software Industry Insights in “Who’ll Be the First to Offer Cash for Infrastructure” (which is a great read in general) says: 

And for those who are thinking about evaluating a private cloud strategy (although a cloud on your own infrastructure in my view is not a cloud), they may be able to get more value out of virtualization and other strategies, but they lose the operational efficiencies and instant scalability to match spikes in demand that the Cloud has to offer.

I won’t argue the private cloud comment. Really, I won’t because what’s important is the reminder that cloud isn’t the only way to realize operational efficiencies in data center architectures. This is important because not all applications are going to be able to move to “cloud” – whether internal or external. Some applications are just not suited to a shared environment for many reasons including aging technology and the cost to essentially “rip and replace” the systems. But improving the efficiency of delivering those applications is something we can do whether we’re “going cloud” or not.

It’s easy to forget amidst the hype (or maybe the air is just really thin up here in the clouds) that there are other ways of doing things that can be just as beneficial to achieving operational efficiency. It’s true that some operational efficiencies associated with cloud can’t be achieved without, well, a cloud; on-demand scalability is part and parcel of the definition of a cloud and without cloud that capability – and its operational benefits – simply can’t be realized. But there are many other efficiencies – particularly those associated with automation and orchestration and integration – that can be realized by just about any architecture. 

• Automation and orchestration aren’t just for cloud
  The operational efficiencies that come from automating and orchestrating processes in the data center usually associated with cloud and virtualization can be applied to any architectural model, provided the infrastructure is capable of being automated. The reduction of errors associated with manual operations and the time-savings can provide real operational benefits in terms of time recovery and financial efficiency. Whether via scripting or a standards-based control plane API, automation of mundane operational tasks (take down a server for maintenance, bring it up afterwards) can reduce the burden on staff by offloading the tasks to automation systems.
  
  Infrastructure 2.0 isn’t specific to cloud or virtualization; it’s an evolutionary step in infrastructure that provides the framework through which automation and orchestration can be achieved. It’s the foundation for a dynamic infrastructure, the definition of which does not preclude the use of its capabilities in architectures other than cloud.
  
• Feedback loops that include applications make (smarter) real-time decisions possible
  The integration of status, capacity, and performance information from applications with the rest of the infrastructure means security, performance, and availability focused infrastructure can make better decisions in real-time regarding how best to distribute requests across resources. If one application instance is nearing capacity or processing a particularly intense request, that information can be relayed to the infrastructure such that the next request to the application is directed to another more capable instance. The ability to incorporate feedback from applications and across the infrastructure means decisions made by individual components are made within the context of the entire data center, not just its immediate network environment.
  
  This integration is really part of automation and orchestration, as it implies that if actionable data is shared with the infrastructure subsequently the infrastructure will be able to use that information to make decisions without human intervention. And because it’s making decisions based on current information rather than on a static configuration the entire data center executes more efficiently by leveraging the resources available to meet demand as well as business requirements.
  
  This is one of the core tenets of Infrastructure 2.0: collaborative feedback that results in actionable data upon which intelligent systems can make smarter decisions.
  

Cloud computing models offer greater operational efficiencies  because the physical tasks associated with management and maintenance are completely removed from the shoulders of IT in addition to the more flexible use of resources and the inherent benefits of automation and orchestration. While you won’t be able to skip the monthly maintenance on your servers if you aren’t going to a “public” cloud model, you can apply some of the architectural lessons learned from cloud to traditional architectures to realize many of the same operational benefits.

Infrastructure 2.0 requires collaboration. Collaboration requires the ability to communicate. The ability to communicate requires integration. But how that integration will happen may shape the future of infrastructure and network architecture.

There is a growing recognition of the basic problems associated with the rapid rate of change inherent in on-demand architectures (cloud) and the complexity that comes from virtualized data centers. Challenges such as IP address and application management, visibility, and last but not least, integration.

Yes, that most dreaded of all technology concepts has finally come to the network.

The answer to the growing challenges of managing rapid change is automation and orchestration, but in order to build such solutions there is required the ability to integrate infrastructure – both with other infrastructure solutions and with the management systems and platforms that will actual control the orchestration of the data center. The need for this infrastructure integration is rising in awareness. That’s a Good Thing. But questions remain regarding how that integration should be achieved; what form should it take?

While traditional EAI (enterprise application integration) technology originally took the form of API-based integration – that is, libraries that included functions that could be invoked to execute specific functions – in later years with the advent of SOA and Web Services metadata-based integration patterns became much more popular. Metadata-based integration reduced the cost to create, maintain, and support integration libraries for the vendors and insulated customers from changes and the nitty-gritty details.

But then Web 2.0 and social networking became all the rage and integration between those sites reverted to the traditional API-based method with a slight twist. Rather than rely upon completely proprietary data formats, i.e. created specifically for the application, they began to offer both JSON (JavaScript Object Notation) and XML formats to exchange data. While not completely interoperable – the data itself is not compatible across applications – the format is, at least across platforms, languages, and implementations.

Infrastructure 2.0 needs to look at what has – and hasn’t – worked in the application space, and learn from it to lay a solid but extensible foundation for the future of infrastructure integration.

DIFFERENT STROKES FOR DIFFERENT FOLKS

Infrastructure solutions today use a variety of mechanisms to collaborate. The primary purpose has been to allow third-party development of management applications for specific applications or platforms, though there has also been a smattering of enterprise usage for specific integration purposes with data center management systems. Infrastructure today has also generally accepted as a standard format XML, though whether that’s via a RESTful API or a SOAPy API is very much dependent on the vendor’s view of the world and what its typical users demand.

There are a lot of infrastructure solutions (and even more announced/coming after VMWorld this year) that are API-enabled. The thing is that just like Web 2.0 and social networking APIs no two APIs are the same. That means configuring a VLAN on a Cisco switch or an F5 BIG-IP or an HP ProCurve Switch is a completely different process. The API calls themselves, the data required, the process – each is unique to the product. This complicates application portability across clouds (or data centers) because the orchestration and automation that enables a dynamic infrastructure is implicitly tightly-coupled to the infrastructure. That’s okay for the cloud provider, because they’re probably – like you – standardizing on certain vendors so it isn’t going to be a problem for them. And the granularity offered by the various APIs provides them with the ability to build out automation and orchestration solutions that are tailored to their environment. The more that can be automated the more simplified the provisioning process, which in turn offers value to its customers and the ability to differentiate in the market.

But when you try to take an application that may require services from security, acceleration, load balancing, IPS, IDS, firewall, and other infrastructure support solutions from one environment to another, that’s where the differences in the API becomes apparent – and problematic. You can’t automate the migration via the API because the product in one environment may be different than the next, and therefore such a method would be useless. The answer is, of course, to somehow just share the configuration data, but today that is just as tightly coupled to products as APIs.

There’s no standard way to share the metadata – the configuration - that describes those requirements across vendor lines. When you request configuration data from product B it’s completely different from that of product A, and neither one can completely understand the other. So what’s needed is a standardized but extensible metadata format – and a way to share/consume that metadata across clouds and data centers. That’s the concept behind constructing a mechanism through which metastructure data can be published, shared, and consumed, anyway.

NOT MUTUALLY EXCLUSIVE METHODS OF INTEGRATION

When it comes down to it the use of metadata and APIs to integrate and collaborate in a dynamic infrastructure is not an either-or proposition. On the contrary, in fact, both will be critical to the success of infrastructure 2.0 to solve the challenges associated with implementing a truly dynamic infrastructure.

APIs will be necessary to specifically automate and orchestrate data center operational and business processes while metastructure hubs will be necessary for portability, upgrades, and reconfiguration efforts. While it certainly appears, at least at first glance, that metastructure hubs and the metadata integration approach would work well for both design (configuration, a.k.a. governance) and run time dynamism, metadata integration does not enforce any order of operations. Can’t and shouldn’t enforce any order of operations. Infrastructure interested in certain events or data subscribe to a topic or channel and receive (or pull, depending on the model) updates at varying rates. Processes generally require that certain tasks be complete ere the next one begins, and thus require more control. That control comes from an API and a management system capable of executing specific automations across the infrastructure in the specified order at the specified time under specified conditions.

Though it might appear at first that the there are two competing methods of integration to enable the dynamic infrastructure, nothing could be further from the truth. Both metadata integration and API-based integration will be required to build out a truly portable, dynamic infrastructure. And if we look at what’s happened in the web application space, we see that it, too, has compromised on a combination of metadata (standardizing on XML and JSON) and APIs to enable the cross-application sharing of data and functions that essentially today make up the “social networking web”.

Interestingly enough it seems to be working for Web 2.0. Hopefully we’ll see the same kind of success and adoption if we enable similar integration mechanisms for Infrastructure 2.0.

Nemertes research presented during yesterday's webinar on virtualization and the network showed that flexibility is now a bigger driver for virtualization than capex savings.

Read more...