Getting Started with Clojure

I'm starting to enjoy using Clojure - the Lisp built on top of Java if you like. Great set of libraries, available and relevant documentation, a good base of users and plenty of examples.

I admit I'm already familiar with the functional programming style and have been for many years since discovering ML, Hope and Gofer many, many years ago, so I guess moving back to the one true language - Lisp - isn't so difficult, and anyway I don't want to talk about functional programming.

As everything seems to be about web based services, it tends to always be an quick experiment for me to build a simple web based service to get to grips with a language and its support. I must admit I loved Opa, but was let down by the support; Ruby on Rails I never really liked and let's not discuss Java and all its frameworks. Anyway, Clojure has been getting a lot of attention and a couple of colleagues of mine have had some very favourable things to say about it (one is a Lisp-fanatic however...) So here goes.

First go get Leiningen, the rather cool Clojure project automator. Dump that in a place where everyone can see it. I'm using Debian 7, so it goes in /usr/local/bin for me. I assume you're already familiar with things like this. Create a new Clojure project, which creates a whole bunch of directories under ./test

ian@deb1:~$ lein new test
Generating a project called test based on the 'default' template.
To see other templates (app, lein plugin, etc), try `lein help new`.
ian@ian@deb1:~$ ls -l test
total 36
drwxr-xr-x 2 ian ian  4096 Jan 15 10:52 doc
-rw-r--r-- 1 ian ian 11220 Jan 15 10:52 LICENSE
-rw-r--r-- 1 ian ian   264 Jan 15 10:52 project.clj
-rw-r--r-- 1 ian ian   230 Jan 15 10:52 README.md
drwxr-xr-x 2 ian ian  4096 Jan 15 10:52 resources
drwxr-xr-x 3 ian ian  4096 Jan 15 10:52 src
drwxr-xr-x 3 ian ian  4096 Jan 15 10:52 test

So what we're going to do is write a small service that queries a database from a call over HTTP or via a browser.

First however, we need to make sure we've a database somewhere. I'm using MariaDB, you could use MySQL or any other RBMS system, but you'll need to check some specifics on how to connect. I've created a database called testdb, a single table and populated it with some data. You'll need to use whatever tools your database provides to do this, but for MariaDB (and MySQL I guess) it is the following:

ian@ian@deb1:~/dbtest$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 28
Server version: 5.5.34-MariaDB-1~wheezy-log mariadb.org binary distribution
Copyright (c) 2000, 2013, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> create database fred;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> use "fred";
Database changed
MariaDB [fred]> create table ids (id integer, name varchar(50) );
Query OK, 0 rows affected (0.17 sec)
MariaDB [fred]> insert into ids values (1,"Alice");
Query OK, 1 row affected (0.02 sec)
MariaDB [fred]> insert into ids values (2,"Bob");
Query OK, 1 row affected (0.01 sec)
MariaDB [fred]> insert into ids values (3,"Eve");
Query OK, 1 row affected (0.01 sec)
MariaDB [fred]> commit;
Query OK, 0 rows affected (0.00 sec)

Aside: yes, I should be shot for using the database root user and not bothering about security...agreed! Don't do it and go read up on such things.

Now the two important files are project.clj and core.clj ... use your favourite text editor to open these:

ian@ian@deb1:~$ cd dbtest
ian@ian@deb1:~/dbtest$ gedit project.clj src/dbtest/core.clj &

Edit the file project.clj so it looks like this:

(defproject dbtest "v0.1 DBTest"
  :description "Simple Test Application"
  :url "http://http://ijosblog.blogspot.fi/"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.clojure/java.jdbc "0.2.3"]
                 [compojure "1.1.6"]
                 [mysql/mysql-connector-java "5.1.18"]]
  :plugins [[lein-ring "0.8.10"]]
  :ring {:handler dbtest.core/app }
  :profiles {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
                        [ring-mock "0.1.5"]]}}
)

Don't worry about the description, url and license properties or the name of the project. The important things here are the dependencies, plugins, profiles and ring.

These describe the packages that we'll be including in our project, think of them as #includes in C/C++. What we're stating here is that we require Clojure version 1.5.1, Java JDBC , the MySQL connector (works with MariaDB) and a package called Compojure which will handle requests. Additionally we have a plugin ca lled lein-ring which handles the low-level details of HTTP calls plus provides a server to receive the calls. The line stating with :ring states the function that will be called on the start up of the application. The specifics aren't too important at the moment, we can learn about that later.

The actual code that we will run is held in the core.clj file by default. There are clues to Clojure's structuring and packaging mechanisms in the above code and in the directory structure. Firstly the we define the namespace and the packages we wish to include:

(ns dbtest.core  (:use [compojure.core]        [clojure.test] )      (:require [compojure.handler :as handler]            [compojure.route :as route] ))(use 'clojure.java.jdbc)

Then we define the function helloworld:

(defn helloworld []
  "Hello World")

and the functions that handle requests from the client

(defroutes app-routes
  (GET "/" [] ( helloworld ))
  (route/resources "/")
  (route/not-found "Not Found"))

(def app
  (handler/site app-routes))

And that is effectively everything we need to do to get something running. But before you do run, it is necessary to make sure we have all the libraries installed and lein does this for us by consulting the project.clj file, so run:

ian@ian@deb1:~/dbtest$ lein deps

and let it download everything you need. You generally only have to do ths when you add libraries to the system. Once that has completed (there'll be lots of messages about downloads, or nothing if you've already got everything) start the server:

ian@ian@deb1:~/dbtest$ lein ring server
2014-01-15 11:12:17.221:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-01-15 11:12:17.345:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3000
Started server on port 3000

A browser window should pop up, but if not start your favourity browser, eg: firefox and point it at http://127.0.0.1:3000 and you should get a page starting Hello World. NB: 127.0.0.1 always resolves to the local machine, so if you have the browser running on a seperate machine (virtual or not) then make sure you use the correct IP address. For the brave you can always use curl to get the body and headers by running the following in a seperate terminal window:

ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000
Hello World
ian@ian@deb1:~/dbtest$ curl -I  http://127.0.0.1:3000
HTTP/1.1 200 OK
Date: Wed, 15 Jan 2014 09:19:09 GMT
Content-Type: text/html;charset=UTF-8
Content-Length: 0
Server: Jetty(7.6.8.v20121106)

After you're happy, kill the server with ctrl-c.

Let's now connect to the database by adding the following code between the (ns...) and (use...) and our hello world function:

(let [db-host "localhost"
      db-port 3306
      db-name "testdb"]

  (def db {:classname "com.mysql.jdbc.Driver" ; must be in classpath
           :subprotocol "mysql"
           :subname (str "//" db-host ":" db-port "/" db-name)
           ; Any additional keys are passed to the driver
           ; as driver-specific properties.
           :user "root"
           :password "badpassword"}))

Firstly localhost is the local machine again. I'm running everything on a single machine, but if the database were elsewhere then obviously I'd need to the use the IP address of that machine. And if I were being clever then I'd pull all this in from some configuration file...again, later...

The user and password are used to log in to that specific database (testdb), which if you remember I created using the MariaDB root user (bad!) and also note the horrificly insecure password I was using. Oh and it is an even worse idea to hard code it in plain text in your source file. Don't say you weren't warned. DO NOT DO THIS IN ANYTHNG OTHER THAN A TRIVIAL EXAMPLE, and even then it is probably a bad idea.

After the above code which connects to our database, we add a simple function:

(defn list-all []  (with-connection db    (with-query-results rs ["select * from ids"]      (doall rs))))

This states to use the connection details stored in the db variable and make a query on the database and store the results in as a list called rs and then process it. In this case we're forcing the list to be evaluated using the doall function over the list rs. Clojure is a lazy functional programming language like Haskell, or at least has lazy features.

Now modify the app-routes function so:

(defroutes app-routes
  (GET "/" [] ( helloworld ))
  (GET "/listall" [] ( list-all ))
  (route/resources "/")
  (route/not-found "Not Found"))

Don't forget to restart the server:

ian@ian@deb1:~/dbtest$ lein ring server
2014-01-15 11:12:17.221:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-01-15 11:12:17.345:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3000
Started server on port 3000

and in a seperate terminal (or put the URL below into your browser)

ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000/listall
{:name "Alice", :id 1}{:name "Bob", :id 2}{:name "Eve", :id 3}

And there you have Clojure returning the contents of the database. After this, kill the server again (note: there is a way to get lein to automatically rebuild and restart the server...exercise left to reader ;-)

Now add the function:

(defn get-by-id [the_id]
  (with-connection db
   (with-query-results rs ["select * from ids where id=?" the_id ]
     (doall rs))))

Aside: the above probably could do with some validation to protect against SQL injection and other bad data (cf: obxkcd)

and modify app-routes so it contains an additional line:

  (GET "/id/:the_id" [the_id] ( get-by-id the_id )) 

and restart the server as before and test with curl (or your browser):

ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000/id/1
{:name "Alice", :id 1}
ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000/id/2
{:name "Bob", :id 2}
ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000/id/3
{:name "Eve", :id 3}
ian@ian@deb1:~/dbtest$ curl http://127.0.0.1:3000/id/4
Not Found

And that's it, a "fully" functioning web service of sorts. 0/10 probably for style and -100000s for security but quite cool for so few lines of code.

--

Here's the final code for the core.clj file:

(ns dbtest.core  (:use [compojure.core]        [clojure.test])      (:require [compojure.handler :as handler]            [compojure.route :as route]          ))
(use 'clojure.java.jdbc) (let [db-host "localhost"    ;; change this as necessary      db-port 3306           ;; change this as necessary      db-name "testdb"]      ;; change this as necessary   (def db {:classname "com.mysql.jdbc.Driver" ; must be in classpath           :subprotocol "mysql"           :subname (str "//" db-host ":" db-port "/" db-name)           ; Any additional keys are passed to the driver           ; as driver-specific properties.           :user "root"           :password "badpassword"}))     ;; change this as necessary!

(defn list-all []  (with-connection db    (with-query-results rs ["select * from ids"]      (doall rs))))

(defn get-by-id [the_id]  (with-connection db    (with-query-results rs ["select * from ids where id=?" the_id ]      (doall rs))))
(defn helloworld []  "Hello World")
(defroutes app-routes  (GET "/" [] ( helloworld ))  (GET "/listall" [] ( list-all ))  (GET "/id/:the_id" [the_id] ( get-by-id the_id ))   (route/resources "/")  (route/not-found "Not Found"))
(def app  (handler/site app-routes))