Virtual Machines, Containers, and now LXD: What’s best for me?

First there were virtual machine instances running on bare metal or para-virtual hypervisors, then came containers allowing for better utilization of virtual machine instances, and now we have the Linux Container Daemon (LXD) pronounced “lex-dee”. As the tag line goes “The new hypervisor isn’t a hypervisor, and it’s much, much faster”; one quickly realizes that hyperboles are not where your troubles end! Instead your choices for virtualization just got trickier to sort out.

So here are some simplifying rules to get going quickly:

1. If you have the luxury of hosting your application entirely on the public cloud, stick to the good old fashioned VMs managed using the auto-scale functionality provided by the cloud platform of your choice.

2. If you have the luxury of creating your private cloud using paid software such as the vCloud suite, stick to the good old fashioned VMs managed using the components of the suite such as vRealize.

3. If you are stuck with the free bare metal hypervisors such as vSphere ESXi or Hyper-V(not entirely free) or with paravirtual hypervisors such as KVM then get the right DevOps skills on your team and use Docker containers managed using Fig for development and Chef for production environments.

4. If you are the brave spirit capable of dealing with the hardcore hardware and low level kernel configuration matters opt for Metal As A Service (MAAS) offering from Canonical combined with Juju.

5. If you are lucky to have the hardware setup exactly as required for running OpenStack/CloudStack and have the chops to customize the provided management apps then by all means rock your world by replicating most of the basic AWS offerings within your datacenter.

6. Finally, if you are truly bored of your mind by all the mundane options listed above and have the distinct air of “Been there – Done that” around you with the appropriate management support within your back pocket venture in to the brand new entrants such as LXD!

Of course you might also just be lucky enough to be in my position: Make recommendations and then sit back to enjoy the show! And, of course, come back to this blog for more …

Developer Workstations – The untapped Private Cloud

The “Infrastructure as a Service”(IaaS) model for Cloud Computing has matured significantly in the recent years with a large number of providers offering a variety of public, private, and hybrid clouds at very competitive prices. However, adoption of the “Platform as a Service”(PaaS) model is still lagging. The primary reason for this is quite easy to understand: most developers and delivery managers are least impacted by the IaaS model and hence they can continue developing and delivering software the same way as before. Transitioning to a PaaS model on the other hand may involve significant changes to the software development and delivery process and thus represents a significant risk to the overall success of the project.

Challenges to PaaS adoption in Development Teams

One of the key challenge in adopting a PaaS model is the cost of making the target platform available to the entire development team for all functions. Instead, the typical use case is to implement Continuous Integration using PaaS offerings from vendors such as CloudMunch, OpenShift, Heroku, etc. But this still limits the developers from being able to have access to all the features of the target platform and thereby constraining innovation to a small set of “experts”. Clearly, lack of infrastructure should not be a limiting factor for increased innovation and productivity.

Underutilization of Developer Workstations

Due to the exponential growth in computing power combined with ever decreasing cost of hardware, every developer typically is provided with either a laptop or a desktop that has sufficient computing power to host an application server as well as a database and any other required software locally for their own private use. Doing so allows for a number of benefits from telecommuting to ease of adoption of the Agile development methodology. Furthermore, lack of consistent high speed broadband access, either due to poor infrastructure in developing countries or due to network congestion over the airways in developed nations, limits access to shared enterprise PaaS resources for developers who are rarely tethered to a desk. On the other hand synchronization through a centralized code repository requires very little bandwidth. Finally, these local environments will eventually start varying from each other and more importantly from the target environment thereby introducing the risk that submissions from different team members may not play well together in the CI environment.

Cloud Infrastructure Management Frameworks to the rescue …

Private clouds based on open source software such as OpenStack or CloudStack may offer a solution wherein each developer workstation is a virtual machine host on to which an image of the target platform can be launched on demand. These images can be auto generated as a part of the daily CI cycle. A developer thus would have access to the overall platform so as to either try out new features or make configuration changes to resolve certain issues. The same infrastructure can also be leveraged to provide additional computing resources for the CI cycle or for load testing.

But not without some additional innovation

At present, the primary constraint of the open source cloud management frameworks is the homogeneity of the host hardware. This severely limits the use of developer workstations as hosts. Furthermore, the use of bare metal hypervisors is not feasible and the most popular operating systems for developer workstations are not ideally suitable for hosting type 2 hypervisors. Instead, as a first step, it is recommended that an approach based on desktop virtualization products such as VirtualBox or VMware Player be adopted. As a part of the daily CI build a new version of target environment can be packaged as an appliance and be made available to developers for download. Thus, in addition to getting the latest code at the start of the day, developers can also get the latest target environment and fire it up on their workstations. Additionally, some basic agent software can be developed to allow the developer to add their local guest OS instance to a resource pool for use in intensive computing tasks such as load testing. Simultaneously, open source frameworks can be extended to allow for a more heterogeneous mix of hosts.

This blog was originally published at on January 10, 2013