Removing the Operating System Barrier with Platform as a Service (PaaS) – a guest post from Adron Hall
Welcome to the first in a series of articles that will dive into Open Source Software (OSS) available for cloud computing. In this post, I’m going to focus this series on a particular OSS Platform as a Service (PaaS) called CloudFoundry. VMware initiated this product with wide OSS industry support. By far this PaaS offering is one of the most open, but also provides some of the widest support of any PaaS product on the market. CloudFoundry provides many technology frameworks to build with, while preventing lock-in to any single infrastructure platform, hardware provider or vendor.
What Does a PaaS Do For Your Software Development Efforts?
Before modern PaaS, software development had a traditional hosting and development practice. This was exemplified in highly coupled applications developed by a task based waterfall process. In many environments these practices are still heavily used.
Traditional Highly Coupled Applications
Most developers have seen this traditional application architecture. With a front-end client of some type and a back-end database it stands out as a textbook example of application design. Even the modern web applications like ASP.NET or PHP web apps are often modeled around client server or server heavy architecture of some sort. The most common today is a web client of HTML, JavaScript, and CSS that is rendered or communicates with a server side that provides data storage in a relational database.
Highly Coupled Environments
Of these highly coupled application architectures, a software team usually sets up an environment that mimics the servers and tools they need. Sometimes this works well, sometimes it works horribly. These environments are broken into subsets that need to be duplicated, but often are not. The environments usually consist of development, quality assurance (QA) or testing, user acceptance testing (UAT), and production. Development teams spend significant resources to maintain, build, and remove such environments.
In these enterprises, the software is created and moves through each of these environments toward production. Sometimes there is some automation, but often not, causing a very time intensive, costly, and error prone progression of steps toward production.
Hopefully the application reaches a production environment without any significant errors, but that is often not the case. Clients, either internal to the business or external, then use this environment acting as a last tier of testing through actual use. For the more traditional environments this process can take several hours, days, and sometimes even weeks!
Because of this error prone environment, it takes hours of maintenance, often multiple team members spending time away from actual software development to ensure the stability of the environment. In the end, the environment still has problems, fails, needs rebooting, etc. This environment is not designed for failure or for resiliency. This just begins the long list of problems with traditional hosting environments and traditional development services. I’m sure you could add a list of your own.
Alright? How About a Solution? Enter PaaS!
Wouldn’t it be sweet if operations, software developers, and the business had something they could develop with in a kind of virtual space? Imagine a space that is not highly coupled to a hardware machine or a single server, database, or other single point of failure. Imagine an environment that could be auto-scaled, geographically dispersed, and could insure 5 9’s of uptime. Just think of a space that could be virtualized so it could shift as needed, so that it could provide a simple path to deploy to, yet connect and sit a top whatever infrastructure a business would find best for this space. Well this type of software and service exists, rolled into an idea and product referred to as PaaS. With CloudFoundry, that’s exactly what the OSS Community has created!
OSS Infrastructure as a Service Overview
It is important to note the currently available offerings in the OSS Community related to cloud computing for the Infrastructure as a Service (IaaS) Layer. Creating an OSS PaaS relies on a large subset of technological building blocks to deploy, automate, virtualize, and manage infrastructure. Some of these efforts have spawned the following products and many more:
- OpenStack: The OSS community has been working on an offering with a number of companies called OpenStack. OpenStack provides an OSS suite that can be used to manage, deploy, and build out cloud infrastructure from Dell, HP, Amazon Web Services, Rackspace, or a number of other providers.
- CrowBar: Crowbar was developed by the Dell CloudEdge Solutions Team as a OpenStack installer but has evolved as a much broader function tool. It is grown into a wrapper for Opscode Chef Server.
- Chef: Chef is an open source software systems integration framework, built to bring the benefits of configuration management to infrastructure.
- Puppet: Puppet is an open source software system for automating system administration tasks for systems infrastructure.
Although these tools have made management of infrastructure exponentially easier, there are still many gaps to be bridged in actual development and deployment of software solutions. PaaS functions to manage the infrastructure resources to enable the developer to focus on building applications.
OSS Platform as a Service: CloudFoundry
To maintain a solid, high quality, and powerful open source software stack from infrastructure to the PaaS, CloudFoundry steps in to fill the gap for the OSS community. Over the last few months, an increasing number of supporters, contributors, and advocates of CloudFoundry have been formed to expand the platforms supported and capabilities of CloudFoundry.
CloudFoundry provides support for many platform frameworks. The list below will have additions very soon since the ongoing work by many groups is enabling more frameworks everyday.
- PHP
- Rails (Ruby)
- Sinatra (Ruby)
- Python
- Grails (Java)
- Spring (Java)
- Node.js (JavaScript)
- Scala
The infrastructure service CloudFoundry provides support for varies widely. Just like the frameworks listed above, the infrastructure list will be changing since groups will be adding support rapidly.
- OpenStack (Which includes any of the systems supporting OpenStack such as Windows Azure, Rackspace, etc.)
- Rackspace
- Windows Azure
- Amazon Web Services (AWS)
- CloudFoundry (the actual provider)
- ActiveState
- AppFog
- SpringSource
Some other tools that CloudFoundry PaaS supports include multiple NoSQL, relational databases, caching products, and more.
- RabbitMQ
- vFabric Postgresql
- MongoDB
- Redis
For the Microsoft Enterprise, OSS IronFoundry Adds Support for .NET
.NET has been excluded from the feature list of CloudFoundry until IronFoundry was released on December 13th. IronFoundry is a fork of the CloudFoundry Solution that enables .NET to run with any CloudFoundry Environment. This draws .NET into a level of parity with other platforms like Ruby on Rails, PHP, and Java for open source PAAS compatibility.
Now that we’re all on the same page with OSS, PaaS, and what we all can get out of such a radical shift in when, how, and where we develop software systems get ready for a deep dive. I’m going to jump into installation, configuration, and deployment of CloudFoundry and IronFoundry. After I cover the material to get up and running we’ll take a spin around node.js, .NET, Ruby on Rails, MongoDB, Redis, or maybe some other technologies to deploy, but you’ll have to stay tuned in and keep reading to know which ones I’ll cover!
This is the first of five guest posts from writer Adron Hall. Adron is a jovial, TDD, BDD, get things done well software architect, engineer, coder and jack of all trades. He runs the gamut of dev stack from Ruby on Rails to Node.js to.NET (though his favorite these days is Ruby with a growing admiration for the chaos of JavaScript). He’s passionate about DevOps and loves to get involved in hackathons, user groups, and other tech community events. He tries diligently to keep improving and writes about his adventures at his blog Composite Code.
Sign up here. It's free, so why not?
Great introduction to PaaS and the gap that CloudFoundry is set out to fill. I am looking forward to see how to get this stuff up and running!
Adron Hall Reply:
January 11th, 2012 at 4:50 pm
Sooner than later eh! Will have the 2nd part of the series up real soon!
Posted: 11 January 2012 at 12:41 pm by Jeff Schumacher
Heroku founder James Lindenbaum sees this shift ushering in polyglot developers. I can see pent up enterprise demand emerging and disrupting if the PaaS providers can also provision a single billing statement. Rather than receive bills from each service provider (a la the out of control US health-care billing practices). http://cliveboulton.com/post/12240351564/web-stack-layers-shift-into-cloud-services
Adron Hall Reply:
January 11th, 2012 at 4:51 pm
Polyglots, as are, are starting to filter into the mono-language Enterprises too. It’ll be interesting to see how and when more enterprises start making the shift. Between OpenStack and CloudFoundry ( & IronFoundry ) the reasons to not use PaaS and at least IaaS in the enterprise is disappearing, even for the paranoid. I’m looking forward to seeing this start to play out in the markets!
Posted: 11 January 2012 at 1:00 pm by clive boulton
Adron,
I am interested in learning how CloudFoundry support an Oracle or IBM database. Does CloudFoundry contain an Enterprise Service Bus, Business Process Management Engine, or Identity Management server? If not, how would I add those infrastructure components into my environment?
Mike Reply:
January 12th, 2012 at 10:36 am
Chris.
I think i looked up how to connect to Oracle/IBM/etc. I believe you can. They will just be external to the cloud (which will work with a private or micro cloud). And since they are not OSS, the only way for them to be in CloudFoundry as a service would be for IBM or Oracle to do it (that is my guess).
As for ESB, if your ESB can run on Tomcat, it probably can be deployed as Spring or Java app. (http://blogs.mulesoft.org/galaxy-on-ec2-via-cloud-foundry-in-10-minutes/). I would like to see one as a service too so it could just be picked as a service.
The same holds true for BPM.
As for Identity Management – http://support.cloudfoundry.com/entries/20197687-pluggable-authentication-mechanism-for-cloudcontroller
Additionally – possible ApacheDS – http://directory.apache.org/
Adron Hall Reply:
January 13th, 2012 at 6:39 am
I too, am curious how some of those databases will be supported. Part of the problem with those is they’re built so far from the conceptual idea of horizontal scalability that makes it extremely difficult to offer legitimate scaling to things like DB2, Oracle, etc. It breaks their staunch vertical scaling roots (Thus the reasons behind NoSQL in the first place). I’m sure a half way can be drawn between these two different ideals for RDBMSs similar to what Microsoft has made with SQL Azure and AWS with mySQL and RDS. Matter of fact, knowing how important relational databases are, I hope to enable some of that myself (time for some OSS CloudFoundry/IronFoundry coding!).
As for ESB, BPM, and the lot of enterprise packages, that’s a whole different topic of discussion. I hope to approach it in the near future too. Since PaaS is focused on “Applications” and ESB & BPM are basically applications, there isn’t a huge reason to not have them available. At this point in time, I gather that the primary applications, tools, and other elements are web focused since smaller companies are pushing the innovation of this technology. I imagine not long from now, you’ll see these things starting to crop up. As I said above, I’d sure love to help providing these types of things!
Cheers!
Mike Reply:
January 13th, 2012 at 8:00 am
I am going to try to reply again.
ClouldFoundry is open source. DB2, Oracle are not. So if the they allow it or take and build their version of CloudFoundry, it will be there. I am pretty sure though that you can connect to an external database, as long as you can access it. That will be tough in the public cloud.
ESB & BPM – if they run as Java or Ruby (etc) web apps, they should work. Mule will run in CF (i found a link – google for mule Cloudfoundry). So at some point I am sure they will just run as services.
JasperForge runs in CF as an app and they are working on getting running as a service. So there will be BI.
As for identity management – according to the link i found, they are working on it. In the mean time, you might look at ApachDS.
(i had a much better reply yesterday, but it never showed up).
Mark Reply:
January 14th, 2012 at 7:51 am
As a followup, MuleSoft has their own cloud based ESB. Search (google) for iPaaS. For some reason I cannot post links.
While i can see a BPM running in your cloud instance, I think it is less likely that you would have an ESB in it unless you have LOTS of apps in your instance. But since ESB is more of a architecture than an actual product, you could have something you call an ESB running in your instance.
Ezra Reply:
January 13th, 2012 at 11:26 pm
In order to use your Oracle or IBM database with a cloud foundry install all you need to do is use our service provider SDK which is included in the open source codebase. In fact all the current services that come with cloud foundry(mysql, redis, mongodb,etc,etc) use the same basinal framework to implement the service provider gateway.
So if you were to use Cloud Foundry in your Enterprise behind the firewall in your own data centers, all you would need to do is write a service provider gateway that gives access to your Oracle or IBM database. This is actually fairly easy, it is a “fill in the blacks” sinatra app that you have to run in front of your “service”(in this care your oracle or IBM database) that responds to certain urls and also sends certain commands to the cloud controller in order to register itself with a particular cloud foundry install.
Cloud Foundry itself dog foods this same API for al the services that ship with it. If you conform to this interface and write the sinatra “Service provider gateway” code then you will be able to see your oracle or IBM database(or whatever other service you want to expose to your cloud foundry install) and you will be able to create/bind/destroy/etc service instances of the oracle or IBM database just like you can out of the box with redis, mongodb, mysql and rabbitmq.
Here is a pointer to the code where the current service provider gateways live so you can see how easy it is to write one of these things:
https://github.com/cloudfoundry/vcap-services/blob/master/redis/lib/redis_service/redis_node.rb
That is the main guts of the Redis service provided with cloud foundry out of the box. You basically need to implement provisioning/deleting, binding and giving credentials to bind for whatever service you want to advertise into the cloud.
Your daemon also needs to send a registration packet of a certain JSON format when it boots up with a shared secret token that is shared out of band between the cloud foundry install and the service provider so that not just any old service can just show up in your cloud.
All of this is fairly straightforward and is one of the 3 main planes of extensibility built into cloud foundry, the other 2 “planes” are runtimes and frameworks as well as underlying IaaS or hosting infrastructure that cloud foundry runs on. So CF is easy to extend in these 3 vectors and that has been one of the main project goals since the beginning.
Cheers–
Ezra
Adron Hall Reply:
January 14th, 2012 at 2:03 pm
Hey Ezra, thanks for the response. This is some great additional information. With comments like these I’ll be adding a little extra to subsequent parts of this series. Again – thanks! Great information.
Posted: 12 January 2012 at 4:04 am by Chris Haddad
I cannot find the link for external databases. It might have been in a video. I found an older one that says they plan on it.
I could not find a link for BPM. But i would bet that JBoss is getting jBPM to run on OpenShift (if it does not already). Which, again, should then run on CloudFoundry.
Also found links where people where working on getting Alfresco working on OpenShift. I would think if it works there, it would work on CloudFoundry. Then there would be a ECM.
I guess the “issue” with the things I posted is that they are Java based. Which should not be an issue for most of these things.
Posted: 13 January 2012 at 8:21 am by Mike
I’m curious to know how does the Azure integration works.
Is there a public reference for this?
Adron Hall Reply:
January 13th, 2012 at 1:59 pm
At this time there isn’t a whole lot of documentation. However this will be changing as I am aware of at least a half dozen different companies that are working to put this together for Windows Azure.
Nati Shalom Reply:
January 14th, 2012 at 7:17 am
That’s what i thought. I believe that you should put a disclaimer on that regard as from your post one may think that all this work is done and available already.
You should note that Azure is a PaaS on its own so it would be interesting to see why how the two fits together.
Mark Reply:
January 14th, 2012 at 7:44 am
I was wondering the same thing. A PaaS running on a PaaS? I can see some sort of integration, but not CF on Azure. I can see it CF running on whatever Azure is running on. Except, sadly, Microsoft suffers from NIHS. But the did drop they product for Hadoop. So, maybe … .
Chris Haddad Reply:
January 14th, 2012 at 9:27 pm
Mark, The PaaS on PaaS situation has me confused as well. The ecosystem surrounding Cloud Foundry demonstrates how PaaS, the middle level between SaaS and IaaS is actually a multi-layered market space. A way to unwind the recursive relationship between Cloud Foundry and ecosystem partners is to first start calling the technology a ‘cloud-enabled platform’, and PaaS is an instantiation of the cloud-enabled platform delivered as a service. The CloudFoundry ecosystem partners (e.g. AppFog, Stackato, Uhuru, Tier3) seem to be competing on ease of use enhancements, bundled technology (e.g. language support, cache support, database support), or managed hosting.
I too get confused when I hear AppFog or Stackato described as PaaS’ running on top of Cloud Foundry, which is also marketed as a PaaS. It becomes non-trivial to characterize the differences between offerings. In fact, Stackato is now promoting their offering as “The cloud platform for creating your private PaaS” and AppFog defines their offering as “a cloud-based hosting service for your favorite web application stack”. Under a more defined and consistent nomenclature, Stackato could be considered a cloud-enabled application platform build on top of Cloud Foundry, and AppFog considered as a Platform as a Service built on top of Cloud Foundry.
But the last sentence doesn’t describe ‘What is Cloud Foundry?’ Cloud Foundry self-describes their offering as “an open platform as a service, providing a choice of clouds, developer frameworks and application services”. But the open source project is not a service; it’s a cloud aware application execution framework, which sit underneath a traditional application platform and application frameworks (e.g. .NET, Java, Sinatra, Tomcat, Rails). Cloud Foundry capabilities are actually of little direct use to an application developer; unless during the time when the developer is building servers or deploying an application. The Cloud Foundry core does not help developers design, code, or test applications (where hopefully they spend a majority of their time).
Cloud Foundry and ecosystem partners are focused at a different abstraction level when compared with traditional, integrated application platform suites (i.e. IBM WebSphere, Oracle SOA Suite, Tibco ActiveMatrix, or WSO2 Carbon). Maybe someone will coin a better name to describe Cloud Foundry’s place in the PaaS space.
Adron Hall Reply:
January 14th, 2012 at 2:07 pm
Well Windows Azure is inclusive of a LOT of things right. It is Microsoft’s effort for PaaS, but under that umbrella they’re including changes and additions that would be included under IaaS, but they’re not calling it that. I for one, don’t intend to mislabel Microsoft’s work even if they do.
However, one can indeed get up and running with IronFoundry, if they use the virtual machines available in Windows Azure. I didn’t say much else beyond that because of the huge pricing differences between Windows Azure around this and what it would cost with anyone else. In the future though, the extremely near future, these capabilities will be growing/changing and possibly, I’ve heard that pricing will be getting a review at Azure.
We shall see. As for a disclaimer, consider this it. The blog entry merely states what you can, absolutely, do right now with Windows Azure. It is a PaaS but it also offers some IaaS capabilities.
Cheers, and thanks for reading!
Posted: 13 January 2012 at 1:42 pm by Nati Shalom