Differences between revisions 2 and 3
Revision 2 as of 2008-06-12 14:10:31
Size: 7815
Editor: chello089076118047
Comment:
Revision 3 as of 2008-06-12 14:11:20
Size: 7816
Editor: chello089076118047
Comment:
Deletions are marked like this. Additions are marked like this.
Line 19: Line 19:
 * {{cl-couchdb-object-layer}}}---a simple object layer (in fact a thin wrapper over json  * {{{cl-couchdb-object-layer}}}---a simple object layer (in fact a thin wrapper over json

["Cl-CouchDB"] is a very young library, and it doesn't have its own webpage yet. You can download it using darcs:

darcs get http://www.common-lisp.net/project/submarine/darcs/cl-couchdb

Installation

Put symlinks to the .asd files in some place where ASDF can see them and configure CouchDB (see below). Then load the systems you need using ASDF.

Tutorial

Cl-CouchDB consists of three main components:

  • cl-couchdb-client

  • cl-couchdb-view-server

  • cl-couchdb-object-layer---a simple object layer (in fact a thin wrapper over json

    • objects/association lists)

Cl-CouchDB-client

Allows to make requests to the running CouchDB server. The main entry points are r and r*. r is a macro for doing comfortably requests from the REPL, r* is the functional interface to r.

Some examples:

COUCHDB-CLIENT> (r :get (blog "150fedd5d14f0771eb5e44d071a1df5d")) ;a GET request to http://localhost:5984/blog/150fedd5d14f0771eb5e44d071a1df5d
((:_ID . "150fedd5d14f0771eb5e44d071a1df5d") (:_REV . "253381451")
 (:AUTHOR . "foo") (:BODY . "Zażółć") (:POST . "third") (:TYPE . "comment")
 (:N . 66))

COUCHDB-CLIENT> (r :get (blog _all_docs :count 2)) ; GET http://localhost:5984/blog/_all_docs?count=2
((:TOTAL-ROWS . 48) (:OFFSET . 0)
 (:ROWS
  ((:ID . "06672346ffc093ce68a07692a5f12db5")
   (:KEY . "06672346ffc093ce68a07692a5f12db5") (:VALUE (:REV . "3441371051")))
  ((:ID . "14328cab564dfec5eac0ff0a44d2083d")
   (:KEY . "14328cab564dfec5eac0ff0a44d2083d") (:VALUE (:REV . "1258191009")))))

As you can see, r (and r*) return lisp objects (alists) and take lisp objects, which are translated to JSON without bothering the programmer (of course, there's a lower level interface if you prefer to do the json things yourself).

An important utility function is @. This allows you to access data stored in alists (this is what json objects get translated into) in the JavaScript dot style. For example, (@ doc :friend :id) is equivalent to js doc.friend.id.

Cl-CouchDB-View-Server

This is a view-server implementation. It supports mapreduce and should be also able to deal with a rereduce.

As you may know, a lisp image is rather heavy, so you shouldn't be starting every now and then. This means that I had to take a slightly different approach to allow communicating CouchDB with lisp. A lisp image that has started a view-server will be listening to port 5477. So, you need to put something like

common-lisp=/usr/bin/socat -TCP4:localhost:5477
  • to the [Couch Query Servers] section of your

couch.ini (you can substitute socat for any program that will allow a socket open on port 5477 look like a program with standard input and standard output).

This approach has some advantages, however. First of all, you can use all the goodness a running lisp image provides, specially its loaded libraries. This allows for example to have an SQLite db in memory and use it in views to calculate stuff that otherwise would very difficult to do in CouchDB. Second, views can make requests to the couchdb server itself (though I am not sure this is always a good idea). Finally, views are compiled (even ad hoc views) instead of being interpreted, and CouchDB sends just the symbols naming the functions to call rather than the source.

COUCHDB-SERVER> (open-server ) ; we need to be able to speak with couchdb through http
*COUCHDB-SERVER*

COUCHDB-SERVER> (start-view-server)
#<view-server :host "127.0.0.1" :port 5477>

COUCHDB-SERVER> (defdesign test
                    ((by-author-type :map (doc)
                                     (emit (list (@ doc :author) (@ doc :type)) doc)))
                  (:documentation "A test view.")
                  (:sync blog))
#<design-document :name TEST :revision NIL :views (#<view BY-AUTHOR-TYPE :map "#'CL-COUCHDB-VIEW-SERVER::BY-AUTHOR-TYPE-MAP" :reduce NIL>)>

This creates a design document "test" with one view: by-author-type, and saves it automatically to the database "blog" (it assumes that a server is already running).

Now, you can easily query this view:

COUCHDB-SERVER> (query-view 'by-author-type :startkey '("foobar") :endkey '("foobar" #()))
(((:ID . "first") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "first") (:_REV . "2718626630") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 5)))
 ((:ID . "fourth") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "fourth") (:_REV . "2695588251") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 8)))
 ((:ID . "second") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "second") (:_REV . "230136489") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 6)))
 ((:ID . "third") (:KEY "foobar" "blogPost")
  (:VALUE (:_ID . "third") (:_REV . "2453743212") (:AUTHOR . "foobar")
   (:BODY . "Zażółć gęślą jaźń") (:TYPE . "blogPost") (:N . 7))))
47
39

It is also easy to query an ad-hoc view using `query':

COUCHDB-SERVER> (query 'blog '(lambda (doc) (emit (@ doc :author) (@ doc :body))) :count 2)
(((:ID . "06672346ffc093ce68a07692a5f12db5") (:KEY . "foo")
  (:VALUE . "Zażółć"))
 ((:ID . "146238e235f0cc36661ce82c909044be") (:KEY . "foo")
  (:VALUE . "Zażółć")))
47
0

Cl-CouchDB-Object-Layer

The objects (which are called `docs') are in fact a thin layer over alists. Specifically, you can call `@' on docs to get the value of an attribute, exactly as you would do with an alist. CouchDB doesn't check in any way how the documents we put in it look like, so it seemed a good idea to have some way of checking a document is valid before sending it to the database. This is why I introduced the concept of validators.

To define a doc class, use defdoc (which is similar to defclass). For example:

COUCHDB-OBJECTS> (defdoc blog-post 
                     ((:author :validator #'stringp)
                      (:title :validator #'stringp)
                      (:_id :initform (lambda (doc) (url-encode (@ doc :title))))
                      (:body :validator #'stringp))
                   (:default-db 'blog))
#<STANDARD-METHOD MAKE ((EQL BLOG-POST)) {BEB6679}>

The validators are one argument functions taking the value of an attribute (something like a slot, but identified by a keyword) and returns true if it is valid. The initform may be either a normal lisp value or a one argument function, which is called on the object itself after setting other attributes.

COUCHDB-OBJECTS> (make 'blog-post :author "Kuba" :title "O czym dziś napisać" :body "Foo")
#<doc(NIL) :_ID "o_czym_dzis_napisac" :BODY "Foo" :TITLE "O czym dziś napisać" :AUTHOR "Kuba" :TYPE "BLOG-POST">

COUCHDB-OBJECTS> (let ((doc (make 'blog-post :author "Kuba" :title "O czym dziś napisać" :body "Foo")))
                   (@ doc :title))
"O czym dziś napisać"

We can call make-and-save to create a document and save it in the database:

COUCHDB-OBJECTS> (make-and-save 'blog-post :author "Kuba" :title "Zażółć gęślą jaźń" :body "foobar") ;we'll get the rev in return
"2591270477"

Notice that if the object is invalid, it won't be saved:

COUCHDB-OBJECTS> (make-and-save 'blog-post :author "Kuba" :title "Zażółć gęślą jaźń")
The document #<doc(NIL) :_ID "zazolc_gesla_jazn" :TITLE "Zażółć gęślą jaźń" :AUTHOR "Kuba" :TYPE "BLOG-POST"> is invalid. Reason: attribute
                      :BODY with value NIL didn't validate using #<FUNCTION STRINGP>
   [Condition of type VALIDATOR-FAILED]
...

Licensing

BSD sans advertising clause.

Author

Ryszard Szopa <ryszard.szopa@gmail.com>, with one component (logv) being authored by Nick Allen <nallen05@gmail.com>.