Differences between revisions 87 and 88
Revision 87 as of 2013-12-03 01:28:59
Size: 9916
Editor: HerveBoutemy
Revision 88 as of 2013-12-14 13:02:24
Size: 9940
Editor: HerveBoutemy
Comment: link to build services page in general services pages
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
Part of [[http://ci.apache.org/|ASF Build Services]], the ASF runs a [[http://jenkins-ci.org/|Jenkins continuous-integration server]] at https://builds.apache.org, with builds@apache.org mailing-list (see [[http://mail-archives.apache.org/mod_mbox/www-builds/|archive]]). Part of [[http://www.apache.org/dev/services.html#build|ASF Build Services]], the ASF runs a [[http://jenkins-ci.org/|Jenkins continuous-integration server]] at https://builds.apache.org, with builds@apache.org mailing-list (see [[http://mail-archives.apache.org/mod_mbox/www-builds/|archive]]).

Part of ASF Build Services, the ASF runs a Jenkins continuous-integration server at https://builds.apache.org, with builds@apache.org mailing-list (see archive).

The Jenkins setup consists of a master server running on an Ubuntu machine (Aegis) and several build slave servers, including six Ubuntu slaves (minerva, vesta, hemera, janus, juno, quirinus), FreeBSD and Windows slaves.

Any ASF project can run build and test tasks for their projects from this server. PMC chairs can grant access to Jenkins to any committer (see below for instructions) to get an administrative account on Jenkins, in order to create the Jenkins jobs representing your projects.

Those of you in the Incubator, there are many PMC Chairs belonging to the Incubator PMC, including but not limited to the Incubator PMC Chair. So if a mentor can't do it, ask around. If no answer within a reasonable time frame, then by all means open an INFRA ticket, mentioning your earlier attempts to get a PMC Chair to do it.

Your users will then be able to view results of those builds on the web.


How do I get an account

Jenkins uses the Apache LDAP servers for authentication. To give a committer access to Jenkins, the committer must be made a member of the hudson-jobadmin group. This is done using the modify_appgroups.pl script on people.apache.org. Ask your PMC chair to execute this command. For example:

modify_appgroups.pl hudson-jobadmin --add=<Apache username>

You must subscribe to infrastructure@ and builds@ to receive notifications of Jenkins upgrades, outages, etc.

To list current members of that group use

list_appgroups.pl hudson-jobadmin

If you need shell access to any of the slaves, you need to open an issue as described below. This should normally only be needed if your builds require custom tooling that you need to install.

How do I report a bug / contact the maintainers?

Use this JIRA form to create an issue under the "Jenkins" component.

Getting started

When your account request has been processed, you will receive an e-mail containing your login details.

What should I do with the build and test tools that my project needs?

First discuss with the Jenkins admins (on builds@apache.org, see archive ) to see if this a tool of general interest, in which case the Jenkins admins can take care of the installation.

Various version of the following tools are supplied in /home/hudson/tools:

  • ant
  • clover
  • java
  • maven
  • checkstyle
  • jdiff
  • forrest ($HOME/tools/forrest/apache-forrest-0.8 on hudson-solaris, minerva and vesta)
  • erlang ($HOME/tools/erlang/latest on hudson-solaris, minerva and vesta)
  • litmus ($HOME/tools/litmus/latest on hudson-solaris, minerva and vesta)

When configuring your job, these settings are a good idea:

Locks (Mandatory)

  • Use one of these locks for long running builds (> 1 hour):

    • Long-running jobs on minerva.apache.org
    • Long-running jobs on vesta.apache.org
    • Long-running jobs on Ubuntu (ie. both vesta and minerva)

Timeout (Mandatory)

Note that this is forced by the Admins by running a script on an irregularly basis which set it to the double of the estimated duration with a minimum of 10 minutes.

Repository browser (Recommended)

Tie this project to a node (Recommended)

  • For Ubuntu 12.04, tie your job to one of:
    • Ubuntu

    • For legacy Ubuntu 10.04 use ubuntu2

  • For Solaris 10, tie your job to:
    • Solaris

  • For Windows, tie your job to:
    • Windows

  • For OS X, tie your job to:
    • osx

If your build is not system-dependent (most simple Java builds, etc.) you can leave your build untied to maximize the number of executors available for the build. Otherwise please use the generic system labels Ubuntu, Solaris and Windows unless your build needs special tooling that has only been installed on one of the build servers.

How do I allow Jenkins to mail to my project's "dev" list?

Send an email to FOO-allow-subscribe-jenkins=builds.apache.org@TLP.apache.org. Replace "FOO" with the name of your dev list, e.g. "dev", and "TLP.apache.org" with the name of your TLP. Note that you should send this request from a moderator email address.

On the job's Config page, tick "E-mail Notification", and type FOO@TLP.apache.org in the Recipients field. Tick 'Send e-mail for every unstable build'.

Mails will now be sent every time a build starts failing, or when a failing series of builds starts working again.

How do I setup pre-commit Builds (similar to Hadoop's)

See PreCommitBuilds for more info.

Where do I find more information on Jenkins?

Who maintains the Jenkins servers?

Currently, the administrators are:

  • Justin Mason <jm /at/ jmason.org>

  • Jukka Zitting <jukka /at/ apache.org>

  • Tim Ellison <tellison /at/ apache.org>

  • Giridharan Kesavan <gkesavan /at/ apache.org>

  • Niklas Gustavsson <ngn /at/ apache.org>

  • Olivier Lamy <olamy /at/ apache.org>

FAQ For Administrators

How do I restart Jenkins?

Log into builds.apache.org via SSH. The following command can be used to clean up all pending Jenkins processes and start from a fresh state:

  sudo /etc/init.d/hudson restart

Note that given the number of historical builds that Jenkins must load into memory, it may take up to 10 minutes for the Jenkins webapp to become functional after a restart.

How do I restart a Jenkins Slave?

Visit https://builds.apache.org/computer/ (logging in if necessary), click the slave's name, and hit Disconnect. Give a reason and confirm you want to disconnect the slave. Then ssh to the slave box, kill all processes owned by the "hudson" or "jenkins" uid:

  sudo pkill -15 -u hudson ; sleep 5
  sudo pkill -9 -u hudson

and, back on the web UI, hit Connect to restart the slave.

How can I see what tasks are open in JIRA?

This JIRA search will display the current list of tasks for the 'Jenkins' component of the Infrastructure project (that's us). You may want to subscribe to that Filter; hit 'Save' to save it as a filter, then 'Manage' and 'Subscribe'.

How do I install a new version of Jenkins?

SSH to builds.apache.org and run /home/hudson/upgrade-to-latest.sh to update to the latest Jenkins version. The script takes care of downloading Jenkins and performing the necessary installation procedures.

How do I install a new Jenkins plugin?

Jenkins plugins extend the functionality of Jenkins but also might risk some stability on the build cluster, so be careful. Plugins are used to collect and present data for Findbugs and Checkstyle, to name a few. These plugins are packaged as .hpi files and are available from http://jenkins-ci-org/. The plugin file goes in ~hudson/hudson/plugins/. To install or upgrade a plugin, follow these steps:

  1. Download the new plugin file
  2. Move the old plugin file to a backup location incase you need to rollback
  3. Copy in the new plugin file into the plugins directory

  4. Shutdown and startup Jenkins

Note that the plugin directory must be owned by the hudsonadmin user, not the hudson user. The start script should take care of this for you.

Setting up a Ubuntu slave

  1. Create user
    • adduser jenkins
  2. Set up SSH access by softlinking ssh pub key and add jenkins to sshusers
  3. rsync over tools from some other Ubuntu slave
  4. Install packages
    • sudo apt-get install erlang-dev erlang-eunit g++-multilib gcc-multilib ia32-libs rubygems libmono-system-web2.0-cil libboost-test1.40-dev libevent-dev libbit-vector-perl php5-cli erlang-base mono-gmcs libmono-dev librspec-ruby libboost-dev flex bison pkg-config ruby-dev python-dev php5-dev libglib2.0-dev cabal-install ghc6 rake python-twisted libghc6-binary-dev libghc6-network-dev libghc6-http-dev
  5. Softlink /home/hudson to /home/jenkins for tools locations to work
  6. Add ulimit -n 40000 to /etc/init.d/ssh and restart ssh. If sshd is running with the -D flag, you will first need to stop it using sudo service ssh stop.
  7. Copy over ~/.m2/settings.xml from some other Ubuntu slave

Where do I find more information on administering our Jenkins installation?

What do Administrators do?

  • Jenkins upgrades and restarts
  • Add plugins
  • Kill stuck builds
  • Set policy for Jenkins usage
  • Watch changes to this wiki page


The Jenkins zone is backed up nightly to /zonestorage/backup , which in turn is backed up to another ASF machine periodically. /export/home/hudson, user home directories, and /opt are backed up. Note however that historical build data and artifacts are not backed up.

Jenkins (last edited 2016-12-03 08:10:08 by GavinMcDonald)