Differences between revisions 4 and 5
Revision 4 as of 2008-06-12 14:17:46
Size: 7823
Editor: chello089076118047
Comment:
Revision 5 as of 2008-06-12 16:17:36
Size: 8373
Editor: eqf220
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
["Cl-CouchDb"] is a very young library, and it doesn't have its own webpage yet. You can download it using darcs: = Overview =

["Cl-CouchDb"] is a Common Lisp library for interacting with CouchDB databases.

It defines three systems:

 * {{{cl-couchdb-client}}} -- a client server for making requests to a CouchDB database from Common Lisp
 * {{{cl-couchdb-view-server}}} -- a view server for programming CouchDB ["Views"] with Common Lisp
 * {{{cl-couchdb-object-layer}}} -- a simple object layer for making {{{cl-couchdb-client}}} easier to work with


= Getting =

There are no official releases of ["Cl-CouchDB"] yet.

Check out the latest version from the [http://darcs.net/ darcs] repo:
Line 7: Line 22:
= Installation =

Put symlinks to the .asd files in some place where ASDF can see them
and configure CouchDB (see below). Then load the systems you need using ASDF.

= Tutorial =


Cl-CouchDB consists of three main components:

 * {{{cl-couchdb-client}}}
 * {{{cl-couchdb-view-server}}}
 * {{{cl-couchdb-object-layer}}}---a simple object layer (in fact a thin wrapper over json
  objects/association lists)
or [http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=submarine-cl-couchdb;a=tree browse] the source code online.
Line 24: Line 26:
Allows to make requests to the running CouchDB server. The main entry
points are {{{r}}} and {{{r*}}}. {{{r}}} is a macro for doing comfortably requests
from the REPL, {{{r*}}} is the functional interface to {{{r}}}.

Some examples:
{{{
COUCHDB-CLIENT> (r :get (blog "150fedd5d14f0771eb5e44d071a1df5d")) ;a GET request to http://localhost:5984/blog/150fedd5d14f0771eb5e44d071a1df5d
{{{Cl-CouchDB-client}}} allows the user to make requests to the running CouchDB server from lisp. It's main entry point is the macro {{{R}}}.

=== R ===

{{{R}}} takes specifiers for a request to a CouchDB database and returns a Lisp object. HTTP PUT requests with {{{R}}} take Lisp objects as well, thus enabling the programmer to black-box the JSON layer completely.

==== Examples ====

Using {{{R}}} to make a GET request to {{{http://localhost:5984/blog/150fedd5d14f0771eb5e44d071a1df5d}}}:

{{{
COUCHDB-CLIENT> (r :get (blog "150fedd5d14f0771eb5e44d071a1df5d"))
Line 34: Line 41:

COUCHDB-CLIENT> (r :get (blog _all_docs :count 2)) ; GET http://localhost:5984/blog/_all_docs?count=2
}}}

Using {{{R}}} to make a GET request to {{{http://localhost:5984/blog/_all_docs?count=2}}}:

{{{
COUCHDB-CLIENT> (r :get (blog _all_docs :count 2))
Line 43: Line 54:
As you can see, {{{r}}} (and {{{r*}}}) return lisp objects (alists) and take lisp
objects, which are translated to JSON without bothering the programmer
(of course, there's a lower level interface if you prefer to do the
json things yourself).

An important utility function is {{{@}}}. This allows you to access data
stored in alists (this is what json objects get translated into) in
the JavaScript dot style. For example,

=== R* ===

{{{R*}}} is the functional equivalent to the macro {{{R}}}.

=== @ ===

{{{@}}} is a utility function that enables you to access Lisp data in the JavaScript dot style. For example,
Line 53: Line 64:
When looking into lists, {{{@}}} does an alist lookup by default. Since it's a generic function, however, it's behavior can be extended for any type of object (eg, those created from CouchDB documents by {{{cl-CouchDB-object-layer}}}).
Line 55: Line 68:
This is a view-server implementation. It supports mapreduce and should
be also able to deal with a rereduce.

As you may know, a lisp image is rather heavy, so you shouldn't be
starting every now and then. This means that I had to take a slightly
different approach to allow communicating CouchDB with lisp. A lisp
image that has started a view-server will be listening to port
5477. So, you need to put something like
{{{Cl-CouchDB-View-Server}}} enables you to write views with Common Lisp instead of having to use JavaScript. It supports {{{mapreduce}}} and should also be able to deal with a {{{rereduce}}}.

Writing views with Common Lisp has several advantages:

 * Common Lisp compiles to fast code.
 * Views are compiled (even ad hoc views) since CouchDB just sends symbols naming functions, rather than their source.
 * You can do all sorts of weird tricks with the running Lisp image, such as maintaining an in-memory database for faster indexed lookup, making requests to the CouchDB server itself, or well... any of the weird things you can do with Lisp...

=== Starting the Lisp View Server ===

==== From the CouchDB side ====

A Lisp image that with a running view server will be listening to port 5477. So, you need to add something like
Line 66: Line 85:
 to the {{{[Couch Query Servers]}}} section of your
{{{couch.ini}}} (you can substitute socat for any program that will allow a
socket open on port 5477 look like a program with standard input and
standard output).

This approach has some advantages, however. First of all, you can use
all the goodness a running lisp image provides, specially its loaded
libraries. This allows for example to have an SQLite db in memory and
use it in views to calculate stuff that otherwise would very difficult
to do in CouchDB. Second, views can make requests to the couchdb
server itself (though I am not sure this is always a good
idea). Finally, views are compiled (even ad hoc views) instead of
being interpreted, and CouchDB sends just the symbols naming the
functions to call rather than the source.
{{{
COUCHDB-SERVER> (open-server ) ; we need to be able to speak with couchdb through http

to the {{{[Couch Query Servers]}}} section of your {{{couch.ini}}} (you can substitute socat for any program that will allow a
socket open on port 5477 look like a program with standard input and standard output).

==== From the Lisp Side ====

The Lisp image just needs to open a client server (we need to be able to speak with couchdb through HTTP) and a view server and it's ready for action.

{{{
COUCHDB-SERVER> (open-server)
Line 86: Line 99:
}}}

=== Creating Design Documents ===

The {{{DEFDESIGN}}} macro creates a design document and saves it to the database.

This example creates a design document "test" with one view ({{{by-author-type}}}) and saves it to the database "blog":

{{{
Line 94: Line 115:
This creates a design document "test" with one view: by-author-type,
and saves it automatically to the database "blog" (it assumes that a
server is already running).

Now, you can easily query this view:


=== Querying the View Server ===

You can query views with the function{{{query-view}}}:
Line 116: Line 138:
It is also easy to query an ad-hoc view using `query':
You
can query ad-hoc views with the function {{{query}}}:
Line 126: Line 149:

Overview

["Cl-CouchDb"] is a Common Lisp library for interacting with CouchDB databases.

It defines three systems:

  • cl-couchdb-client -- a client server for making requests to a CouchDB database from Common Lisp

  • cl-couchdb-view-server -- a view server for programming CouchDB ["Views"] with Common Lisp

  • cl-couchdb-object-layer -- a simple object layer for making cl-couchdb-client easier to work with

Getting

There are no official releases of ["Cl-CouchDB"] yet.

Check out the latest version from the [http://darcs.net/ darcs] repo:

darcs get http://www.common-lisp.net/project/submarine/darcs/cl-couchdb

or [http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=submarine-cl-couchdb;a=tree browse] the source code online.

Cl-CouchDB-client

Cl-CouchDB-client allows the user to make requests to the running CouchDB server from lisp. It's main entry point is the macro R.

R

R takes specifiers for a request to a CouchDB database and returns a Lisp object. HTTP PUT requests with R take Lisp objects as well, thus enabling the programmer to black-box the JSON layer completely.

Examples

Using R to make a GET request to http://localhost:5984/blog/150fedd5d14f0771eb5e44d071a1df5d:

COUCHDB-CLIENT> (r :get (blog "150fedd5d14f0771eb5e44d071a1df5d"))
((:_ID . "150fedd5d14f0771eb5e44d071a1df5d") (:_REV . "253381451")
 (:AUTHOR . "foo") (:BODY . "Zażółć") (:POST . "third") (:TYPE . "comment")
 (:N . 66))

Using R to make a GET request to http://localhost:5984/blog/_all_docs?count=2:

COUCHDB-CLIENT> (r :get (blog _all_docs :count 2))
((:TOTAL-ROWS . 48) (:OFFSET . 0)
 (:ROWS
  ((:ID . "06672346ffc093ce68a07692a5f12db5")
   (:KEY . "06672346ffc093ce68a07692a5f12db5") (:VALUE (:REV . "3441371051")))
  ((:ID . "14328cab564dfec5eac0ff0a44d2083d")
   (:KEY . "14328cab564dfec5eac0ff0a44d2083d") (:VALUE (:REV . "1258191009")))))

R*

R* is the functional equivalent to the macro R.

@

@ is a utility function that enables you to access Lisp data in the JavaScript dot style. For example, (@ doc :friend :id) is equivalent to js doc.friend.id.

When looking into lists, @ does an alist lookup by default. Since it's a generic function, however, it's behavior can be extended for any type of object (eg, those created from CouchDB documents by cl-CouchDB-object-layer).

Cl-CouchDB-View-Server

Cl-CouchDB-View-Server enables you to write views with Common Lisp instead of having to use JavaScript. It supports mapreduce and should also be able to deal with a rereduce.

Writing views with Common Lisp has several advantages:

  • Common Lisp compiles to fast code.
  • Views are compiled (even ad hoc views) since CouchDB just sends symbols naming functions, rather than their source.
  • You can do all sorts of weird tricks with the running Lisp image, such as maintaining an in-memory database for faster indexed lookup, making requests to the CouchDB server itself, or well... any of the weird things you can do with Lisp...

Starting the Lisp View Server

From the CouchDB side

A Lisp image that with a running view server will be listening to port 5477. So, you need to add something like

common-lisp=/usr/bin/socat -TCP4:localhost:5477

to the [Couch Query Servers] section of your couch.ini (you can substitute socat for any program that will allow a socket open on port 5477 look like a program with standard input and standard output).

From the Lisp Side

The Lisp image just needs to open a client server (we need to be able to speak with couchdb through HTTP) and a view server and it's ready for action.

COUCHDB-SERVER> (open-server)
*COUCHDB-SERVER*

COUCHDB-SERVER> (start-view-server)
#<view-server :host "127.0.0.1" :port 5477>

Creating Design Documents

The DEFDESIGN macro creates a design document and saves it to the database.

This example creates a design document "test" with one view (by-author-type) and saves it to the database "blog":

COUCHDB-SERVER> (defdesign test
                    ((by-author-type :map (doc)
                                     (emit (list (@ doc :author) (@ doc :type)) doc)))
                  (:documentation "A test view.")
                  (:sync blog))
#<design-document :name TEST :revision NIL :views (#<view BY-AUTHOR-TYPE :map "#'CL-COUCHDB-VIEW-SERVER::BY-AUTHOR-TYPE-MAP" :reduce NIL>)>

Querying the View Server

You can query views with the functionquery-view:

COUCHDB-SERVER> (query-view 'by-author-type :startkey '("foobar") :endkey '("foobar" #()))
(((:ID . "first") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "first") (:_REV . "2718626630") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 5)))
 ((:ID . "fourth") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "fourth") (:_REV . "2695588251") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 8)))
 ((:ID . "second") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "second") (:_REV . "230136489") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 6)))
 ((:ID . "third") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "third") (:_REV . "2453743212") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 7))))
47
39

You can query ad-hoc views with the function query:

COUCHDB-SERVER> (query 'blog '(lambda (doc) (emit (@ doc :author) (@ doc :body))) :count 2)
(((:ID . "06672346ffc093ce68a07692a5f12db5") (:KEY . "foo")
  (:VALUE . "Zażółć"))
 ((:ID . "146238e235f0cc36661ce82c909044be") (:KEY . "foo")
  (:VALUE . "Zażółć")))
47
0

Cl-CouchDB-Object-Layer

The objects (which are called docs) are in fact a thin layer over alists. Specifically, you can call @ on docs to get the value of an attribute, exactly as you would do with an alist. CouchDB doesn't check in any way how the documents we put in it look like, so it seems a good idea to have some way of checking a document is valid before sending it to the database. This is why I introduced the concept of validators.

To define a doc class, use defdoc (which is similar to defclass). For example:

COUCHDB-OBJECTS> (defdoc blog-post 
                     ((:author :validator #'stringp)
                      (:title :validator #'stringp)
                      (:_id :initform (lambda (doc) (url-encode (@ doc :title))))
                      (:body :validator #'stringp))
                   (:default-db 'blog))
#<STANDARD-METHOD MAKE ((EQL BLOG-POST)) {BEB6679}>

The validators are one argument functions taking the value of an attribute (something like a slot, but identified by a keyword) and returns true if it is valid. The initform may be either a normal lisp value or a one argument function, which is called on the object itself after setting other attributes.

COUCHDB-OBJECTS> (make 'blog-post :author "Kuba" :title "O czym dziś napisać" :body "Foo")
#<doc(NIL) :_ID "o_czym_dzis_napisac" :BODY "Foo" :TITLE "O czym dziś napisać" :AUTHOR "Kuba" :TYPE "BLOG-POST">

COUCHDB-OBJECTS> (let ((doc (make 'blog-post :author "Kuba" :title "O czym dziś napisać" :body "Foo")))
                   (@ doc :title))
"O czym dziś napisać"

We can call make-and-save to create a document and save it in the database:

COUCHDB-OBJECTS> (make-and-save 'blog-post :author "Kuba" :title "Zażółć gęślą jaźń" :body "foobar") ;we'll get the rev in return
"2591270477"

Notice that if the object is invalid, it won't be saved:

COUCHDB-OBJECTS> (make-and-save 'blog-post :author "Kuba" :title "Zażółć gęślą jaźń")
The document #<doc(NIL) :_ID "zazolc_gesla_jazn" :TITLE "Zażółć gęślą jaźń" :AUTHOR "Kuba" :TYPE "BLOG-POST"> is invalid. Reason: attribute
                      :BODY with value NIL didn't validate using #<FUNCTION STRINGP>
   [Condition of type VALIDATOR-FAILED]
...

Licensing

BSD sans advertising clause.

Author

Ryszard Szopa <ryszard.szopa@gmail.com>, with one component (logv) being authored by Nick Allen <nallen05@gmail.com>.