Abstract

Cloudpods is a unified multi-cloud/hybrid-cloud IaaS platform that provides unified APIs to access heterogenous IaaS resources across a broad range of cloud providers, including on-premise KVM/BareMetal/VMware, private clouds such as OpenStack, and public clouds such as AWS, Azure, Alibaba Cloud, etc. Cloudpods provides a set of APIs and a web UI dashboard. Cloudpods hides the differences of underlying IaaS resources, and makes it easy to build PaaS services and applications across multi-cloud/hybrid-cloud IaaS platforms.

Proposal

The goal of this proposal is to bring the existing Cloudpods codebase, developers and community into the Apache Software Foundation (ASF) in order to build a global and diverse opensource community for a unified multi-cloud/hybrid-cloud IaaS platform.

This project started in 2017 and was internally developed in Yunion Co. Ltd. It was open-sourced at GitHub in March 2019. Around the project, an opensource community was formed. Regular monthly online meetups were hold since October 2019. Meanwhile, Yunion was building its commercial solutions based on the opensource code and serving hundreds of enterprise customers.

We are submitting this proposal to donate the sources code and associated artifacts (documentation, web site content, wiki, etc.) to the Apache Software Foundation Incubator under the Apache License, Version 2.0. These artifacts are currently available on GitHub at https://github.com/yunionio and include the following five major parts:

  • yunioncloud: the backend service components and utilities of Cloudpods.
  • dashboard: the frontend Web UI of Cloudpods
  • kubecomps: a Kubernetes management system, built upon Cloudpods as a proof-of-concept example which uses unified IaaS APIs of Cloudpods to build a multi-cloud/hybrid-cloud container platform.
  • yunioncloud-operator: a Kubernetes operator that automates the deployment and operations of Cloudpods service components in Kubernetes cluster.
  • docs: the documentation


Background

The enterprise IT infrastructure has been evolving into the multi-cloud/hybrid-cloud era. The scope of enterprise IT was not only limited to on-premise servers, but also extended to the cloud, including both private cloud and public cloud. Hybrid cloud has been the de-facto standard of IT in enterprise. The complexity of multi-cloud/hybrid-cloud IT architecture calls for a multi-cloud/hybrid-cloud management platform.

The project started in 2017 by QIU Jian, the core developer of a homegrown cloud platform of a large Chinese Internet company. He cofounded Yunion with the faith of building a next generation cloud for enterprises. The project was started with several initial goals:

First, the trends of multi-cloud gradually appeared around 2017. The new cloud should not only be `yet another cloud', but also a cloud that is capable of managing resources from other clouds, especially the public clouds.

Second, cloud native began to be the new paradigm of development and Kubernetes appeared to be the de-facto standard of cloud native run-time. The new cloud should abstract and hide the differences of underlying clouds and expose a set of unified IaaS APIs to above, especially for the Kubernetes. This ensures Kubernetes run anywhere regardless of underlying IaaS providers.

Third, the cloud itself should be cloud native, i.e., it can be running in Kubernetes and managed by Kubernetes so as to take advantages of cutting-edge technologies.

As a result, we chose golang as the backend programming language, and first developed a cloud that was capable of managing local KVM instances, and later expanded the cloud to accomendate VMs from Alibaba Cloud, and so on.

Developed for over three years, with approximately 1 million lines of codes, Cloudpods has been evolving into a multi-cloud/hybrid-cloud platform with following highlight features.

  • A self-contained light-weight private cloud that manages KVM hypervisor in scale
  • A self-contained BareMetal cloud that automates the full life-cycle management of baremetal physical machines
  • A self-contained VMware vSphere management platform that enables self-service and automation
  • A multi-cloud solution that is able to manage a wide range of major cloud providers, including private cloud, such as OpenStack, and public clouds, such as AWS, Azure, Google Cloud, Alibaba Cloud, Tencent Cloud, Huawei Cloud, etc.
  • A full set of feature-rich APIs to access a wide range of the IaaS resources from platforms above with consistent resource models and APIs
  • A multi-tenancy RBAC-enabled identity and access management system
  • A multi-cloud image management system that automates image conversion between different cloud platforms
  • The platform adopted a cloud native architecture and run in Kubernetes cluster and support high availability deployment

Rationale

We decided to opensource Cloudpods March 2019 with the faith that Cloudpods could be better developed with the help of opensource community.

On one hand, Cloudpods needs an opensource community. As an IaaS platform, Cloudpods needs large user base to ensure its maturity and robustness. Meanwhile, as a multi-cloud/hybrid-cloud platform, Cloudpods needs more developers, not only Yunion self, to broaden the coverage of cloud providers. We consistently received requests of connecting some clouds, such as Digital Ocean, Linode, etc. or some legacy cloud platforms, such as Xen, Microsoft SCM, etc. But due to the limited bandwidth, we cannot satisfy all of these needs. Once Cloudpods open sourced, the outside contributors could help extends it to cover more cloud platforms.

On the other hand, the community needs an opensource multi-cloud/hybrid-cloud solution like Cloudpods. As we observed that many Cloud-native companies suffering from the management complexity of multi-cloud/hybrid-cloud. They have to develop their own solutions, which were actually duplicated work that could have been done by Cloudpods. It would be far more efficient if all of these efforts were contributed into one open-source solution and benefits the whole community.

Since its initial open source in March 2019, we have built two WeChat groups of over 600 users and held 18 online meetups.

Now, we decided to donate Cloudpods to ASF to make it be better known by a broader range of audiences and bring the community of Cloudpods into another level of maturity.

Initial Goals

Our initial goals are to bring Cloudpods into the ASF. Following ASF's governance module, integrating with Apache development process, expanding the community and bringing more diversified contributors/users to the community.

Current Status

Meritocracy:

We believe strongly in meritocracy in electing committers and PMC members. We believe that contributions can come in forms other than just code. We encourage contributions and participation of all types, and ensure that contributors are appropriately recognized.

Community:

Cloudpods was internally developed in Yunion since 2017 and open sourced on GitHub in March 2019. Cloudpods has been building an active community since its open source, accumulated two WeChat user groups of more than 500 people, and hosted 16 online meetups. The contributors of Cloudpods includes Yunion and other companies like Meituan, Xiaomi and BONC. We hope to grow the user base as well as contributors during incubation.

Core Developers:

• Jian Qiu (swordqiu@gmail.com)

• Zexi Li (zexi.li@qq.com)

• Yousong Zhou (yszhou4tech@gmail.com)

• Yaoqi Wan (d3lx.yq@gmail.com)

• Xuan Qu (admin@quxuan.org)

• Rain Zheng (mjoycarry@gmail.com)

• Bin Tang (tangbin365@qq.com)

• Jiazhong Hou (houjiazong@gmail.com)

• Peng Wang (lengband@163.com)

• Mingjiu Gao (1170733165@qq.com)

Alignment:

Currently, there is no other active Apache project exactly focusing on providing unified multi-cloud/hybrid-cloud management as Cloudpods did. However, the goals of Cloudpods have some overlaps with several other Apache projects:

  • CloudStack: this is a mature on-premise cloud platform. Cloudpods also contains a fully functional on-premise cloud. However, Cloudpods is more than an on-premise cloud, it is a unified multi-cloud/hybrid-cloud management platform that is able to management resources from a wide range of cloud providers, especially those of public clouds, which is specially designed for multi-cloud/hybrid-cloud architecture.
  • LibCloud and jClouds: both are unified in-process libraries for accessing APIs across many cloud providers. LibCloud is written in python and jClouds is written in Java. Cloudpods is similar in the sense that it also includes a set of golang libraries to access multi-cloud APIs. However, Cloudpods is more than that, it is a cloud platform that provides unified multi-cloud/hybride-coud web-service APIs.
  • DeltaCloud: this is a ruby-based web-service that provides unified API gateway to accessing multi-clouds. However, this project was retired in 2015. Similarly, Cloud also provides unified APIs to access multiple cloud providers. However, Cloudpods is not only a thin layer of multi-cloud API gateway but also a standalone multi-cloud platform that stores unified cloud resource models and orchestrate resources across multiple clouds.

We consider Cloudpods a next-generation cloud platform that is dedicated to the multi-cloud/hybrid-cloud era. At first, Cloudpods is a full-featured cloud platform like CloudStack. Beyond that, it provides the capability of managing resources across multi-cloud providers. Meanwhile, it exposes unified APIs and data models to facilitate accessing underlying heterogeneous IaaS resources. Further, we could leverage the unified APIs to deliver multi-cloud Kubernetes clusters. Finally, it is a cloud-native cloud platform that is developed in golang and adopts a cloud native architecture.

Known Risks

Project Name

The project used to be named 'OneCloud'. Due to trademark conflict, we rename the project to Cloudpods. At the time of this writing, the name Cloudpods was checked against trademark registries of US, EU, China and India, and found no conflict with any existing entity. We have submitted the trademark application of Cloudpods in China.

Orphaned products

The risk of Cloudpods becoming orphaned products is low. As multi-cloud/hybrid-cloud is a common needs of IT industry. Cloudpods have been persistently developed for over three years. Several companies have built their internal systems or products based on Cloudpods APIs. Yunion have been serving hundreds of clients based on commercial products, whose core were Cloudpods.

Inexperience with Open Source:

Most of the core developers in project are enthusiastic to open source. And the development style is already very likely the Apache way

  • We have open community meetups to discuss product designs, problems and roadmaps
  • We publish all patches and issue related discussions on GitHub
  • We enforce the code review and log all comments in GitHub issues

Length of Incubation:

Cloudpods has been developed for three more years. The architecture is well optimized and the features are rich. We don't expect that there will be huge gaps before Cloudpods can graduate with regarding to ASF's policies on software and releases. The goal is to grow the community quickly and increase the user base while making releases that adhere to the ASF standards. When it reaches a reasonable size of adoption, and a strong community with a good number of committers/PMC members, we can prompt the graduation.

Homogenous Developers:

The initial proposed list of committers and contributors includes developers mainly from Yunion Co. We hope Cloudpods could attract more developers from more diversified groups during incubation.

Reliance on Salaried Developers:

Yunion Co. Ltd. has contributed most of the development through salaried developers. However, some developers left Yunion but continued to contribute to Cloudpods. Meanwhile, Cloudpods has received contributions from several contributors from outside, including contributors from Meituan, Xiaomi, BONC. We expect our reliance on salaried developers to decrease during the incubation process.

Relationships with Other Apache Products:

Cloudpods does not closely rely on other Apache Products. The functionalities of Cloudpods has some kind of overlap with several other Apache Projects, such as CloudStack, LibCloud, jClouds and DeltaCloud. We clarify the difference and make our positions in the Alignment section.

An Excessive Fascination with the Apache Brand:

We understand the value of the Apache way, and how to operate the project development on a day-to-day basis. The reason for proposing Cloudpods as an ASF incubator project is to build a healthier community, increasing adoption & the size of the community and end users, because we believe the only way to build a highly valuable infrastructure layer software is to have wide adoption and cater to common use cases.

Documentation

Project summary:

https://github.com/yunionio/yunioncloud/blob/master/README.md

User guides

https://docs.yunion.io/en/docs/

Developer guides

https://docs.yunion.io/en/docs/contribute/

API docs in Swagger format:

https://docs.yunion.io/en/docs/swagger/

Change logs:

https://docs.yunion.io/en/docs/changelog/

Roadmap:

https://docs.yunion.io/en/docs/roadmap/

Initial Source

Cloudpods is developed in golang and Vue.js. The majority of source code is hosted in the following GitHub repositories.

Source and Intellectual Property Submission Plan

External Dependencies:

All the backend source codes are in golang. Majority of the external dependencies are Apache v2.0, BSD or MIT. Frontend source codes depends on Vue.js. Most of the external dependences are either Apache v2.0. BSD or MIT.


Backend dependencies:


Frontend dependencies:

  • Ant Design Vue, MIT
  • Babel, MIT
  • D3, BSD
  • Echarts, Apache v2.0
  • Lodash, MIT
  • Vue.js, MIT
  • Webpack, MIT
  • Vxe-table, MIT
  • Axios, MIT
  • CodeMirror, MIT
  • moment, MIT
  • numerify, MIT
  • xterm, MIT
  • vuex, MIT
  • sheetjs, Apache v2.0


Cryptography:

Except using AES and RSA to secure sensitive information in database, and employing https/SSL to encapsulate API traffic, Cloudpods does not currently include any other cryptography-related code.

Required Resources

Mailing lists:

No mailing list yet.

Git Repositories:

Issue Tracking:

We use github issues for issue tracking.

Continuous Integration tool:

• Jenkins

Initial Committers

Champion:

Willem Jiang ningjiang@apache.org

Nominated Mentors:

Willem Jiang ningjiang@apache.org

Jean-Baptiste Onofre jbonofre@apache.org

Juan Pan panjuan@apache.org



  • No labels