When And Why OpenStack Needs A Cloud Management Platform

At Red Hat we are seeing more and more organizations choosing OpenStack for the next step in their cloud journey. Very often, this transformation journey is marked by three main evolutive stages:

  1. Build a server virtualization environment for scale-up workloads
  2. Extend the server virtualization environment with an Infrastructure-as-a-Service (IaaS) cloud for scale-out workloads
  3. Unify and enforce enterprise-grade governance for both server virtualization and IaaS cloud environments

Different companies stop at different stages of this maturity model, depending on the business needs and the maturity of their IT organization. As the environments in stage 1 and stage 2 grow in size and complexity, companies can reach an operational scale that requires more sophisticated management tools than the ones provided out of the box by server virtualization and IaaS cloud engines.

A Cloud Management Platform (CMP) offers an additional layer to govern a complex server virtualization environment or IaaS cloud as needed by a large-scale end user organization.

In fact, despite OpenStack being a powerful and flexible IaaS cloud engine, doesn’t offer a wide range of management capabilities that some organizations may be looking for, such as:

  • Capacity & Performance Management
  • Configuration & Change Management
  • Chargeback
  • Orchestration

OpenStack does a great job in providing the instrumentation for the aforementioned capabilities – think the metering APIs that OpenStack Telemetry (Ceilometer) offers or the orchestration templates that you can define with OpenStack Orchestration (Heat) – but the management tools that it provides on top of that instrumentation don’t meet the needs of every organization.

To better understand why a CMP is so important at a certain operational scale, let’s use an  analogy: professional property renting.

When you think about the management tools that IT organizations use at each stage of our maturity model, think of:

The Virtual Infrastructure Manager for the amateur landlord

As we said before, at this stage an organization has in place a server virtualization environment and its management console like, for example, Red Hat Enterprise Virtualization Management. The organization is an amateur landlord.

Let’s say that you own one or more apartments that you want to rent. All of them are ideally located in the same city but different in size, finishes, prestige of the location, etc. You want to rent them as long as you can, carefully selecting the best possible occupant for each. You want to keep things simple: long term, fixed price contracts, personally track every change in each apartment and, if something bad happens, you personally work with the occupant to determine responsibility and find a solution.

Your apartments are unique, lovely, hand cared for, just like VMs in a server virtualization environment.

However, you don’t get the most from your properties because this simple, not-automated, way to do business is slow rather than agile, reactive rather than proactive, and with an unbalanced level of attention dedicated to each asset. For example, if one of your tenants starts acting unpredictably and against the law, evicting him can become a nightmare, distracting you from managing all other apartments. In another example, if a growth opportunity knocks at the door, you need time to carefully plan a new property acquisition, select tenants, etc., and this will likely make you lose the opportunity window.

This way of doing business is perfectly fine and sufficient as long as your ambitions as landlord (or your scalability needs as IT organization) remain contained. If your ambition/needs grow, maybe due to a highly competitive market, you need better tools to manage your property portfolio (or your application portfolio) in a more efficient and operationally scalable way.

The IaaS Cloud Manager for the Airbnb-enabled landlord

At this stage an organization has in place an Infrastructure as a Service (IaaS) engine like, for example, Red Hat Enterprise Linux OpenStack Platform. The organization is an Airbnb-enabled landlord.

If the number of apartments you want/need to manage grows, maybe due to early success and increasing market demand, you feel the need for a tool like Airbnb. Airbnb maximizes your capability to address the market demand and minimize the friction in the renting process in many ways. It offers a wonderfully designed website that lists your properties on a map, showing photos of the rooms and furniture, giving guidance about the services around the apartments, and providing a complete booking service that your potential tenants can use in a self-service way.

Airbnb enables you to easily manage different contract options (monthly, weekly, daily), rent a single room or the entire apartment, open and close the calendar for availability instantaneously and, more important, gives you the flexibility to change your mind whenever you want (and offers up to $1M host protection insurance). Airbnb exposes a rating for each property, encouraging landlords to offer a consistent experience for every apartment. Services like Airbnb can help the real estate market grow by increasing competition, pushing landlords to invest more in their properties as revenues come in quicker and in a more frictionless way.

In the same way, OpenStack offers to your lines of business a self-service portal that they can leverage to self-provision what they need, gives you the flexibility to build instance flavours offering different lease times, amount of resources, pre-baked images and grants you the flexibility to introduce or retire those flavours as needed. The usage model encourages users to standardize the OS/Middleware offering, consequently increasing the predictability and efficiency in terms of maintenance, hardware resources, purchasing, etc.

Landlords embrace tools like Airbnb to manage their properties because they want to be agile and catch new business opportunities. To do so, they accept to cut their emotional bond with each individual apartment. IT departments are driven by similar logic, and accept to move from pet-VM to cattle-instances.

The CMP for the professional property manager

At this stage an organization may have deployed a Cloud Management Platform (CMP) like, for example, Red Hat CloudForms, to govern both the server virtualization environment and IaaS cloud. The organization is a professional property manager.

Let’s say that the agility offered by a tool such as Airbnb makes you feel confident to serve hundreds or even thousands of tenants and manage many more properties. This last step in your career as a landlord introduces a completely new set of needs and the complexity is so high that you cannot do everything by yourself. At this point, a tool like Airbnb can’t fulfill all your needs because it’s not designed to serve landlords at scale:

  • managing bookings, cancellations and changes at scale can’t be made with a spreadsheet, you need a professional booking system. You need some level of automation to manage your capacity and at the same time supervise the performance of each property.
  • for each tenant you need to inventory the stay, consumptions, reimbursement, etc., and offer transparent billing. This requires a professional chargeback process.
  • for every booking of every property you need to arrange cleaning, supplies, accesses, etc. When the numbers start rising this can become a massive effort, impossible to be manually fulfilled. You need to orchestrate all the external services connected to your estates: professional cleaning service for both the property and bed linens, for example; suppliers of things like soap, toilet paper, coffee etc.; someone who distribute the keys; and so on.
  • every time a tenant leaves you have to check everything is OK. You need to plan minor and major maintenance activities, changes and improvements for every single property, and even the opportunity to buy new ones!

Operational Burden

Exactly like in our analogy, a CMP introduces a set of critical management capabilities to enhance and augment what OpenStack can do out of the box. Additionally, and critically, a CMP can unify the self-service provisioning experiences across both the server virtualization environment and the IaaS cloud that it manages side by side.

Cloud Management Platform
Following these principles, a CMP like Red Hat CloudForms has capacity planning capabilities that enable IT organizations to know which OpenStack availability zone has enough resources to deploy new instances. For example capacity planning can tell you that a single instance of a web server with 2 vCPUs and 2GB of memory can be safely deployed on zone A but if you plan to scale it out at certain point in time zone B is a better choice provide the amount of additional resources needed.

It provides Performance Analysis capabilities to monitor and forecast the utilization of instances, hosts and providers. For example they can track the average load of the physical hosts over time suggesting the moment to add more hardware to support the increasing demand of resources.

In combination with Ansible (which Red Hat recently acquired), CloudForms offers automation capabilities allowing administrators to create orchestration and configuration workflows for the deployment, setup, and retirement of a certain instances. For example, the deployment of a web server hosting a public website will require your firewall to open a number of ports, and your router to setup a NAT on a public IP to grant access to the Internet audience.

Moreover, CloudForm’s change management and policy enforcement capabilities will keep in compliance the entire environment, tracking modifications and enforcing specific configurations or patch installation on instances and hosts. For example if one of the tenants configures an instance, in its domain, opening a potential security breach CloudForms will automatically restore a safe state.

Last but not least, CloudForm’s chargeback capabilities allow IT organizations to charge OpenStack instances allocation and usage based on a number of different criteria. For example you can account the utilization of a specific instance by minutes, hours, days or a fixed price depending on the kind of workload is going to support.

So, in summary: some organizations may find the management engines coming out of the box with traditional server virtualization or Infrastructure as a Service engines a perfect fit for their business needs. However, for those organizations planning to build a large-scale enterprise-grade private or hybrid cloud, a CMP offers a governance layer that allows them to reach an operational scale that would be impossible to manage otherwise.

Massimo Ferrari
Management Strategy Director