Part of ASF Build Services, the ASF runs a Jenkins continuous-integration server at https://builds.apache.org, with builds@apache.org mailing-list (see archive) and JIRA component for issue tracking.

The Jenkins setup consists of a master server running on an Ubuntu machine (crius) and several build slave servers, including six Ubuntu slaves (ubuntu1 to ubuntu6= minerva, vesta, hemera, janus, juno, quirinus, being transitioned on mid-july 2014 to ubuntu-1 to ubuntu-6= ubuntu-1, ubuntu-2, ubuntu-3, ubuntu-4, ubuntu-5, ubuntu-6), FreeBSD, Solaris 10 x86 and Windows slaves. There are also some slaves dedicated to specific projects (Hadoop, Lucene, Tapestry)

Any ASF project can run build and test tasks for their projects from this server. PMC chairs can grant access to Jenkins to any committer (see below for instructions) to get an administrative account on Jenkins, in order to create the Jenkins jobs representing your projects.

Those of you in the Incubator, there are many PMC Chairs belonging to the Incubator PMC, including but not limited to the Incubator PMC Chair. So if a mentor can't do it, ask around. If no answer within a reasonable time frame, then by all means open an INFRA ticket, mentioning your earlier attempts to get a PMC Chair to do it.

Your users will then be able to view results of those builds on the web.

FAQ For PMCs

How do I get an account

Jenkins uses the Apache LDAP servers for authentication. To give a committer access to Jenkins, the committer must be made a member of the hudson-jobadmin group. This is done using the modify_appgroups.pl script on people.apache.org. Ask your PMC chair to execute this command. For example:

modify_appgroups.pl hudson-jobadmin --add=<Apache username>

You must subscribe to infrastructure@ and builds@ to receive notifications of Jenkins upgrades, outages, etc.

To list current members of that group use

list_appgroups.pl hudson-jobadmin

If you need shell access to any of the slaves, you need to open an issue as described below. This should normally only be needed if your builds require custom tooling that you need to install.

How do I report a bug / contact the maintainers?

Use this JIRA form to create an issue under the "Jenkins" component. Before creating a new issue, you can have a look at already opened issues with this JIRA search.

Getting started

When your account request has been processed, you will receive an e-mail containing your login details.

What should I do with the build and test tools that my project needs?

First discuss with the Jenkins admins (on builds@apache.org, see archive ) to see if this is a tool of general interest, in which case the Jenkins admins can take care of the installation.

Various version of the following tools are supplied in /home/hudson/tools:

When configuring your job, these settings are a good idea:

Locks (Mandatory)

Timeout (Mandatory)

Note that this is forced by the Admins by running a script on an irregularly basis which set it to the double of the estimated duration with a minimum of 10 minutes.

Repository browser (Recommended)

Tie this project to a node (Recommended)

If your build is not system-dependent (most simple Java builds, etc.) you can leave your build untied to maximize the number of executors available for the build. Otherwise please use the generic system labels Ubuntu, Solaris and Windows unless your build needs special tooling that has only been installed on one of the build servers.

How do I allow Jenkins to mail to my project's "dev" list?

Send an email to FOO-allow-subscribe-jenkins=builds.apache.org@TLP.apache.org. Replace "FOO" with the name of your dev list, e.g. "dev", and "TLP.apache.org" with the name of your TLP. Note that you should send this request from a moderator email address.

On the job's Config page, tick "E-mail Notification", and type FOO@TLP.apache.org in the Recipients field. Tick 'Send e-mail for every unstable build'.

Mails will now be sent every time a build starts failing, or when a failing series of builds starts working again.

How do I setup pre-commit Builds (similar to Hadoop's)

See PreCommitBuilds for more info.

Where do I find more information on Jenkins?

Who maintains the Jenkins servers?

Currently, the administrators are:

FAQ For Administrators

How do I restart Jenkins?

Log into builds.apache.org via SSH. The following command can be used to clean up all pending Jenkins processes and start from a fresh state:

  sudo /etc/init.d/hudson restart

Note that given the number of historical builds that Jenkins must load into memory, it may take up to 10 minutes for the Jenkins webapp to become functional after a restart.

Restarting the Jenkins master will automatically restart every Unix slaves, but not Windows ones: restarting the Windows slaves has to be done manually.

How do I restart a Jenkins Unix Slave?

Visit https://builds.apache.org/computer/ (logging in if necessary), click the slave's name, and hit Disconnect. Give a reason and confirm you want to disconnect the slave. Then ssh to the slave box, kill all processes owned by the "hudson" or "jenkins" uid:

  sudo pkill -15 -u hudson ; sleep 5
  sudo pkill -9 -u hudson

and, back on the web UI, hit Connect to restart the slave.

How do I restart a Jenkins Windows Slave?

Visit https://builds.apache.org/computer/ (logging in if necessary), click the slave's name, and hit Disconnect. Give a reason and confirm you want to disconnect the slave. Then... TBD

How can I see what tasks are open in JIRA?

This JIRA search will display the current list of tasks for the 'Jenkins' component of the Infrastructure project (that's us). You may want to subscribe to that Filter; hit 'Save' to save it as a filter, then 'Manage' and 'Subscribe'.

How do I install a new version of Jenkins?

SSH to builds.apache.org and run /home/hudson/upgrade-to-latest.sh to update to the latest Jenkins version. The script takes care of downloading Jenkins and performing the necessary installation procedures. Then send a notice on build@ to let people know of the upgrade.

How do I install a new Jenkins plugin?

Jenkins plugins extend the functionality of Jenkins but also might risk some stability on the build cluster, so be careful. Plugins are used to collect and present data for Findbugs and Checkstyle, to name a few. These plugins are packaged as .hpi files and are available from http://jenkins-ci-org/. The plugin file goes in ~hudson/hudson/plugins/. To install or upgrade a plugin, follow these steps:

  1. Download the new plugin file
  2. Move the old plugin file to a backup location incase you need to rollback
  3. Copy in the new plugin file into the plugins directory

  4. Shutdown and startup Jenkins

Note that the plugin directory must be owned by the hudsonadmin user, not the hudson user. The start script should take care of this for you.

Setting up a Ubuntu slave

  1. Create user
    • adduser jenkins
  2. Set up SSH access by softlinking ssh pub key and add jenkins to sshusers
  3. rsync over tools from some other Ubuntu slave
  4. Install packages
    • sudo apt-get install erlang-dev erlang-eunit g++-multilib gcc-multilib ia32-libs rubygems libmono-system-web2.0-cil libboost-test1.40-dev libevent-dev libbit-vector-perl php5-cli erlang-base mono-gmcs libmono-dev librspec-ruby libboost-dev flex bison pkg-config ruby-dev python-dev php5-dev libglib2.0-dev cabal-install ghc6 rake python-twisted libghc6-binary-dev libghc6-network-dev libghc6-http-dev
  5. Softlink /home/hudson to /home/jenkins for tools locations to work
  6. Add ulimit -n 40000 to /etc/init.d/ssh and restart ssh. If sshd is running with the -D flag, you will first need to stop it using sudo service ssh stop.
  7. Copy over ~/.m2/settings.xml from some other Ubuntu slave

Where do I find more information on administering our Jenkins installation?

What do Administrators do?

Backups

The Jenkins zone is backed up nightly to /zonestorage/backup , which in turn is backed up to another ASF machine periodically. /export/home/hudson, user home directories, and /opt are backed up. Note however that historical build data and artifacts are not backed up.

Jenkins (last edited 2014-07-12 15:23:32 by HerveBoutemy)