Libcloud, a unified interface to the cloud
Abstract
libcloud is a pure python client library for interacting with many of the popular cloud server providers. It was created to make it easy for developers to build products that work between any of the services that it supports.
Proposal
- Provide unified API for manipulating servers instances across many hosting providers who provide an API to manipulate instances. Current API includes: list, reboot, create, destroy, list images, list sizes.
- (future) Provide utilities for manipulating and creating server images in many formats. (See the independent Stacklet project for ideas)
- (future) Provide unified API for storing large objects on popular hosting provider storage APIs.
Background
While there are some projects to create open standards for interoperability within the cloud, most have failed to gain widespread adoption. Libcloud takes the approach of exposing a unified API to cover multiple vendor's APIs, and in the future to support standard APIs, assuming they become prevalent.
Rationale
There is a strong need in the developing cloud infrastructure for a community supported, high quality, and vendor independent tool set for managing servers and their resources. When new servers are just an API call away, traditional infrastructure models are changing quickly. Having a good library built around Apache's values and tradition will enable new server infrastructure to evolve much more quickly.
Initial Goals
Libcloud is an existing open source project, with patches from many different contributors. We view the moving to Apache as a way to improve this community, and look into future APIs around creating server images and large object storage.
Current Status
Libcloud is already open source under the ASL 2.0:
Meritocracy
Libcloud has involvement from members of both the ASF and other open source projects. Communication is driven by both IRC and E-Mail lists.
Community
Currently libcloud has several contributors, but not a large user community other than a few companies. We would like to increase our userbase as part of the incubator process.
Core Developers
Alex Polvi who wrote most of the original code is familiar with open source from working at OSUOSL and at Mozilla. Tom Davis drove much of the re factoring of the initial code base. Jed Smith, Ivan Meredith, Jeremy Orem, Jerry Chen and Paul Querna (ASF member) have all contributed mainly to developing provider specific drivers.
Alignment
Currently there are not many Apache communities involved with cloud computing or python based infrastructure. We believe introducing such a community is a good thing for the Apache Software Foundation.
Known Risks
Orphaned products
libcloud is being used actively by Cloudkick to develop services. It is a core part of Cloudkick's ongoing infrastructure improvements.
Inexperience with Open Source
libcloud was open sourced in July 2009, during OSCON. Core contributors include former employees of Mozilla and an ASF member.
Homogeneous Developers
Much of the initial development was done by Cloudkick, but much of the core design was re-factored by the community, and many of the drivers for each provider have been contributed by 3rd parties.
Reliance on Salaried Developers
The majority, but not all, of the developers are paid by their employer to work on libcloud at this time.
Relationships with Other Apache Products
Libcloud doesn't share many attributes with existing Apache projects due to it being in Python and addressing a new need.
A Excessive Fascination with the Apache Brand
Libcloud project seeks to build a last community around cloud API interoperability, and is not fascinated with any short term gains of being associated with the Apache Brand.
Documentation
- http://www.informationweek.com/cloud-computing/blog/archives/2009/01/the_urgent_issu.html
- http://broadcast.oreilly.com/2009/09/cloud-api-wars---where-is-the.html
- http://blogs.gartner.com/lydia_leong/2009/08/27/are-multiple-cloud-apis-bad/
- http://deltacloud.org/
Initial Source
Initial source is contained completely inside the libcloud github repository.
External Dependencies
- zope.interface
- For Python =< 2.5, a JSON Parser such as SimpleJSON is required.
Cryptography
Uses standard Python APIs for SSL/HTTPS.
Required Resources
Mailing lists
- libcloud-dev
- libcloud-commits
- libcloud-private
Subversion
Issue Tracking
- JIRA (LCLD)
Initial Committers
- Alex Polvi <alex polvi.net>
- Dan Di Spaltro <dan.dispaltro gmail.com>
- Ivan Meredith <ivan ivan.net.nz>
- Jed Smith <jsmith linode.com>
- Jeremy Orem <jeremy.orem gmail.com>
- Jerry Chen <jerry apache.org>
- Logan Welliver <logan cloudkick.com>
- Paul Querna <pquerna apache.org>
- Tom Davis <tom dislocatedday.com>
Sponsors
Champion
- Paul Querna
Nominated Mentors
- Gavin McDonald
- Jean-Frederic Clere
- Ant Elder
- Carlos Sanchez
Sponsoring Entity
- Incubator PMC