Promote a provider to the main repo

This guide describes the steps required to promote an API or provider to the main jclouds repository. The goals of these steps are to:

  • Keep the commit history from the old repository
  • Move the code to the correct folder in the new repository
  • Be able to work with the main repository "normally": being able to merge, rebase, etc, without issues.  

Example: cloudfiles

To illustrate the process, we'll see how an API and provider can be promoted from the jclouds-labs-openstack repository into the main jclouds repository. In this example, we will merge the rackspace-cloudfiles API and two associated providers, rackspace-cloudfiles-us and rackspace-cloudfiles-uk.

We want to merge those three to the correct places in the main repository's structure, so we have to move:  

  • jclouds-labs-openstack/rackspace-cloudfiles -> jclouds/apis/rackspace-cloudfiles

  • jclouds-labs-openstack/rackspace-cloudfiles-us -> jclouds/providers/rackspace-cloudfiles-us

  • jclouds-labs-openstack/rackspace-cloudfiles-uk -> jclouds/providers/rackspace-cloudfiles-uk

In general, we will have a set of directories in the source repository that need to be moved to a different location in the jclouds main repository. We want to move them preserving history, but rewriting it using git so as to appear as if the new locations were always part of that history. Yes, we are rewriting some of that history.

To do this, several steps are required. The steps below can seem quite involved, especially if you're running through them for the first time. The steps include a pull request review which should catch any issues, but if you feel in any way unsure about the steps involved, practise the steps on different repositories first, or get in touch!

1. Clone the repositories

 cd /tmp
 git clone https://git-wip-us.apache.org/repos/asf/jclouds.git
 git clone https://git-wip-us.apache.org/repos/asf/jclouds-labs-openstack.git

2. Make a new branch to contain your work

cd jclouds-labs-openstack
git checkout -b move-promote

3. Move folders to their correct location in the current branch

# Make sure we don't accidentally commit the filtered branches - optional but highly recommended
git remote rm origin
git remote rm upstream
 
# This will make a new directory "jclouds/apis" and move "rackspace-cloudfiles" to "jclouds/apis/rackspace-cloudfiles", changing history as needed.
git filter-branch -f --tree-filter 'if [ -d rackspace-cloudfiles ]; then mkdir -p jclouds/apis && mv rackspace-cloudfiles jclouds/apis; fi' HEAD
# This will make a new directory "jclouds/providers" and move "rackspace-cloudfiles-us" to "jclouds/providers/rackspace-cloudfiles-us", changing history as needed.
git filter-branch -f --tree-filter 'if [ -d rackspace-cloudfiles-us ]; then mkdir -p jclouds/providers && mv rackspace-cloudfiles-us jclouds/providers; fi' HEAD
# This will make a new directory "jclouds/providers" and move "rackspace-cloudfiles-uk" to "jclouds/providers/rackspace-cloudfiles-uk", changing history as needed.
git filter-branch -f --tree-filter 'if [ -d rackspace-cloudfiles-uk ]; then mkdir -p jclouds/providers && mv rackspace-cloudfiles-uk jclouds/providers; fi' HEAD
 
# Repeat the above steps until all directories are in their proper locations in the new "jclouds" directory in your "move-remote" branch in your "jclouds-labs-openstack" repo
# The generic form of the command would be
# git filter-branch -f --tree-filter 'if [ -d {directory to move} ]; then mkdir -p jclouds/{path in the jclouds repo} && mv {directory to move} jclouds/{path in the jclouds repo}; fi' HEAD
 
# Now remove everyting else in your branch and leave just the new "jclouds" directory as the root, creating a new branch (promote-rackspace-cloudfiles-moved) for it:
git subtree split -P jclouds -b promote-rackspace-cloudfiles-moved
 
# Now you have a branch called promote-rackspace-cloudfiles-moved. It contains all the directories with amended history and in the right paths - they are ready to be merged from that branch.
 
# This should work on both bash and Windows with GitHub shell environments.

4. Merge the branch into the main repo

cd /tmp/jclouds
 
# Make a new branch to contain the promoted code
git checkout -b promoted-cloudfiles
 
# Add the repository with the filtered branches as a remote
git remote add jclouds-labs-openstack-local ../jclouds-labs-openstack/
# Always fetch the remote
git fetch jclouds-labs-openstack-local
 
# Merge the branches
git merge jclouds-labs-openstack-local/promote-rackspace-cloudfiles-moved --allow-unrelated-histories # See instruction above

5. Amend the API and provider POM files

The POM files of API(s) and provider(s) from a labs repository need to be amended to match the main jclouds conventions:

  • Change the groupId of the promoted API(s) and provider(s): replace "labs" with "api" or "provider", as appropriate. E.g. org.apache.jclouds.labs -> org.apache.jclouds.api
  • Change the reference to the parent POM of the promoted API(s) and providers to inherit from the correct one, org.apache.jclouds:jclouds-project. See a different API (respectively, provider) in the main repository for an example
  • Add the API(s) and provider(s) to the appropriate aggregator POM, e.g. the apis POM
  • If desired, add the API(s) and provider(s) to the appropriate "convenience" POM, e.g. the allcompute POM

Once the POMs have been fixed, add a commit with the changes.

6. Review the changes before pushing

Now everything should be in place. The status before pushing should be the following:

  1. The last commit is the "Updating POMs" commit
  2. The next commit is a merge commit that joins:
    1. The commits from the promoted provider
    2. The commits pointed from the latest master version

Note that, in general in jclouds, we usually don't want merge commits, but in this case we do. We use a merge commit here to properly express the fact that an entire commit set from another repository has been merged. We do not want to rebase or squash these the commits on top of the latest commit in master, as that would move ALL the commits from the source repository on top of the latest commit in master - definitely not an accurate representation of the history of the source API or providers development.

We want to preserve the time when those commits were made, so please remember: do not rebase or squash!

7. Push the resulting branch as a pull request

At this point, the resulting branch can be pushed as a pull request, so it can be reviewed.

8. Push the reviewed code to the ASF repository

Once the pull request has been reviewed and approved, push the branch to the ASF jclouds repository. The branch should already be up to date with the latest version of master. If this is not the case, then delete the branch, update master, and merge it back again. It is important that you don't rebase the branch after promoting the provider, or you will be changing all the commits we want to preserve.

Once everything is fine and your branch is properly prtomoted and up to date with master, it can be pushed directly. We do this to avoid generating an additional merge commit when merging the branch back to master. Since the branch is up to date, it should be OK to push it directly to master:

git push origin promoted-cloudfiles:master
  • No labels