This page covers the different build processes for several forks of CouchDB while we look at approaches for merging them, & improving the usability & flexibility of the current toolchain.
Voilà, hopefully this thread can be a good start for merging rcouch, bigcouch & apache couchdb and will ease merge of the other features imo.
Apache CouchDB build
- based on autotools
- install couchdb on the system
- use and require shared libraries installed on the system:
- spidermonkey, openssl, curl, icu
- require Erlang installed on the target system
- Embedding couchdb is possible but difficult.
- no hot upgrade possible
- build on linux*, unix*, bsd*, macosx, windows on i386, amd64
- platforms. Building on other platorms is possible modulo hacks on the
libraries and erlang. Windows requires a specific environment
- makefile based
build as an OTP release : the runtime is distributed as part of
- the release and custom Erlang application can be added to the build
easily, rebar is used for it.
- all libraries are fetched and built as static module that will be
- statically linked: no need to install the libraries on the target
system once the release is built. The libraries are automatically patched and built for each supported platforms.
- build on linux*, unix*, bsd*, solaris, macosx, windows possible but need to be
- can be embedded in other Erlang applications easily: propose a
couch_core that can be used by any other platforms
- an rcouch build can be distributed and installed on the same platform
- it has be been built for without requiring any other installation.
- can build a source release with all dependencies integrated.
propose release templates to build custom release
- hot upgrade possible
- makefile based
- use rebar
- build an OTP release, rebar is used for it
- build couchjs using scons to find spidermonkey on the system
- libraries are not statically linked and path to find libs are set in
- build on linux*, unix*, bsd*, solaris
- can be embedded in other erlang apps, but requires libs installed in
- fixed paths. Also the couch application is integrated on the release
so specific build require to fork this release.
- hot upgrade possible
New build system proposal
This proposal is based on the following requirements:
- Possibility to embed couchdb in other erlang apps
- Create a full Erlang OTP release
- Support rebar
- Possibility to ship a static build of apache couchdb for each
- supported platform
- Possibility to reuse installed libraries on targeted systems (good for
- distributions packagers)
- Erlang dependencies should be always available and shipped with the
- source release.
The first 2 points requires in my opinion that we can build a full couchdb core that can be included easily in other erlangs apps. It should also be fetched without apache-couchdb build system. The perfect way to handle it in the Erlang world today is by using rebar and create a rebar.config eventually fetching needed dependencies etc. Also some will want to build their release statically and not care about dependencies to install when they distribute or build their own code.
A rebar build need to know where to find shared or static libs. Some paths can be set using env. It can be also possible to call external scripts that will be used to build external C code and eventually set some makefiles. rebar proposes a system of pluging to extend its possibilities.
To use shared libraries the 2 majors and working systems today are autotools and cmake. While cmake is interresting I think that some of us are pefectly fluent with autotools so it may be better to use them.
I propose to mix autotools and rebar in the build process. Also I propose to fetch erlang dependencies on build or on release instead of having them in our sources like now. Erlang dependencies are for now: mochiweb, oauth, ibrowse, snappy & ejson.
Imo the new build process could be the following:
- autootools used to boostrap the build: create make file and set rebar
- config files using templates.
- 2 rebar config files will be generated : - 1 allowing static build against libraries installed using the
- apache-couchdb-sdk, this file is generated on bootstrap. It will be
used by other erlangs code if needed.
- The others will be created by running ./configure with needed options
- The release will be built using reltools and rebar. Using either
- dynamically generated rebar files or the static one using the
- A source release will be generated using the bootstrapped makefiled. It
- will fetch dependencies and then a signed tar.gz willl be created from
The apache-couchdb-sdk is something like the android sdk or other sdk around. It proposes a build environment for each platforms supported by apache couchdb with all the dependencies already statically builded (erlang, openssl, spidermonkey, icu, curl) . Something we already do for windows and which is done somehow by rcouch too. (Note that couchbase did the same for ios & android platforms.)
It may require some change in our current code structure. Generally an Erlang application is organized like this:
deps/ ebin/ src/ include/ rebar.config
deps/ is the folder where rebar put all dependencies fetches. It's configurable now. ebin/ is the folder where all binaries (.beam & .app) files are put during compilation.
When multiple apps are handled by the same project they can be integrated eithers as dependencies or as included apps and generally the project became:
apps/ deps/ rebar.config
See for example the couch_core repository or th ebigcouch one https://github.com/cloudant/bigcouch. The apps folder is also configurable.
For releases there are 2 ways to handled it. One is to handle it in the same sources. So above become
apps/ deps/ rel/ rebar.config
Where the rel folder contains all the files needed to generate a release. (generally a reltool.config file and a files/ folder) . This is the way chosen by bigcouch. But generally it's better to put the release generation in its own repository so people can integrate the core applications they need (here couch_core) in their own releases templates. See the rcouch repository for example https://github.com/refuge/rcouch. Lot of Erlang developers are doing this way now. My preference go for this one too but I'm not sure how it can be done in apache. If we follow this way it will imply to have 2 or more subprojects
1 subproject : the apache couchdb release (doc, OTP release generation) 1 subproject for the couch_core
Eventually we could split the couch_core in different apps:
- couch_core (the K/V api)
But that's can be discussed in another thread.