dj supports lazy classpath and dependency resolution

Imagine you've been developing code and have all these data structures in memory. You want to visualize them but didn't add incanter as part of a dependency. Now its not a problem since we can set both classpath and obtain dependencies lazily.

We can create a repl anytime and anywhere using:

$ dj repl
Clojure 1.3.0
;user=>

Let's add a simple hello.clj file:

$ echo '(println "hello world")' > hello.clj

Let's add the classpath to the current directory and see if we can require it:

;user=> (dj.classloader/add-to-classpath! "/home/user/")
#
;user=> (require '[hello])
hello world
nil

Let's load some dependencies, how about incanter?:

;user=> (dj.classloader/add-dependencies! '[[incanter/incanter "1.3.0-SNAPSHOT"]])
resolving #dj.deps.maven.maven-dependency{:name incanter, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name incanter-core, :version 1.3.0-SNAPSHOT, :group incanter}
excluding #dj.deps.maven.maven-dependency{:name clojure, :version 1.3.0, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name math.combinatorics, :version 0.0.1, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name parallelcolt, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name netlib-java, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name arpack-combo, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name csparsej, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name jplasma, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name jtransforms, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name optimization, :version 0.9.4, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name incanter-io, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name opencsv, :version 2.0.1, :group net.sf.opencsv}
resolving #dj.deps.maven.maven-dependency{:name clojure-json, :version 1.1-SNAPSHOT, :group org.danlarkin}
excluding #dj.deps.maven.maven-dependency{:name clojure, :version 1.2.0-master-SNAPSHOT, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name incanter-charts, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name jfreechart, :version 1.0.13-no-gnujaxp, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name jcommon, :version 1.0.16, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name incanter-processing, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name processing-core, :version 1.1, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name incanter-mongodb, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name congomongo, :version 0.1.7-SNAPSHOT, :group congomongo}
excluding #dj.deps.maven.maven-dependency{:name clojure, :version 1.2.1, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name core.incubator, :version 0.1.0, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name data.json, :version 0.1.1, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name mongo-java-driver, :version 2.6.5, :group org.mongodb}
resolving #dj.deps.maven.maven-dependency{:name incanter-pdf, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name itext, :version 1.4, :group com.lowagie}
resolving #dj.deps.maven.maven-dependency{:name incanter-latex, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name jlatexmath, :version 0.9.1-SNAPSHOT, :group net.sf.alxa}
resolving #dj.deps.maven.maven-dependency{:name incanter-excel, :version 1.3.0-SNAPSHOT, :group incanter}
resolving #dj.deps.maven.maven-dependency{:name poi, :version 3.6, :group org.apache.poi}
resolving #dj.deps.maven.maven-dependency{:name poi-ooxml, :version 3.6, :group org.apache.poi}
resolving #dj.deps.maven.maven-dependency{:name poi-ooxml-schemas, :version 3.6, :group org.apache.poi}
resolving #dj.deps.maven.maven-dependency{:name xmlbeans, :version 2.3.0, :group org.apache.xmlbeans}
resolving #dj.deps.maven.maven-dependency{:name stax-api, :version 1.0.1, :group stax}
resolving #dj.deps.maven.maven-dependency{:name geronimo-stax-api_1.0_spec, :version 1.0, :group org.apache.geronimo.specs}
resolving #dj.deps.maven.maven-dependency{:name dom4j, :version 1.6.1, :group dom4j}
resolving #dj.deps.maven.maven-dependency{:name xml-apis, :version 1.0.b2, :group xml-apis}
resolving #dj.deps.maven.maven-dependency{:name swingrepl, :version 1.3.0, :group swingrepl}
excluding #dj.deps.maven.maven-dependency{:name clojure, :version 1.3.0-alpha2, :group org.clojure}
resolving #dj.deps.maven.maven-dependency{:name jline, :version 0.9.94, :group jline}
nil
;user=> (require '[incanter.core])
nil

Don't forget that the input is a vector of dependencies, not just a single dependency.

In conclusion, this enables us to develop anywhere (directory) and anytime (during development after the repl started). This is very powerful.

dj update

I added native dependency support. The implementation sets the java.library.path during runtime and not at commandline. This means no forking at startup and a net decrease in start up time. More importantly, the work I did should help make it easier to dynamically load native libraries during runtime. Just using System/load doesn't seem to work with penumbra, which is why I opted for the implementation specific solution at the moment. So the one caveat is this probably only works under sun's (oracle's) jvm.

Anyways, short tutorial:

cd ~/dj/usr/src/
git clone (penumbra git url)
dj repl penumbra

NOTE: I had to edit the sources of penumbra for the current master because there were some namespace conflicts. I'll leave it up to the viewers to figure it out at the moment.

dj published on github

Check it out here!

http://github.com/bmillare/dj

Manual Installation (not using ELPA) of swank-clojure

There appears to be a lot of discussion on getting slime and emacs integrated with Clojure similar to how it is with Common Lisp. For most setups, Technomancy has made it easy to quickly setup the IDE by installing Swank Clojure via ELPA in emacs. Those who don't like using ELPA or need to hack on the sources, need to install the software manually. I present my manual installation experiences and I hope it may be useful to others.

Obtaining Software

I relied on my distribution's package manager to install emacs and slime (as well as git, java, and everything else not listed) but the following packages,

clojure
clojure-contrib
leiningen
swank-clojure
clojure-mode

are rapidly developed and at the moment are not best handled by my package manager so I installed them from source. To keep things organized, I downloaded the sources in their own directory.

mkdir ~/clj
cd ~/clj
git clone git://github.com/richhickey/clojure.git
git clone git://github.com/richhickey/clojure-contrib.git
git clone git://github.com/technomancy/leiningen.git
git clone git://github.com/technomancy/swank-clojure.git
git clone git://github.com/jochu/clojure-mode.git

I was now free to hack, but I needed to build them before use.

Building Software

Swank Clojure depends on Leiningen and Leiningen depends on Clojure so I needed to build that first.

I built clojure as specified in the readme.

cd ~/clj/clojure
ant

I setup an installation directory for Clojure with a minimal folder structure. I don't know what recommended layout is but I copied my distribution's version of clojure.

cd ~/clj
mkdir -p runtime runtime/bin runtime/classes runtime/lib

I then populated the folders. In runtime/bin, I had a simple repl.sh script

#!/bin/bash
java -cp ~/clj/runtime/lib/clojure.jar clojure.main

and in runtime/lib I symlinked all the jars. At the time, the only clojure.jar was built so I linked it.

cd ~/clj/runtime/lib
ln -s ~/clj/clojure/clojure.jar

I tested clojure by by running the repl.sh script.

Next I built clojure-contrib as specified in the README.

cd ~/clj/clojure-contrib
ant -Dclojure.jar=~/clj/runtime/lib/clojure.jar
ln -s ~/clj/clojure-contrib/clojure-contrib.jar ~/clj/runtime/lib

Next was Leiningen, which depends on itself so I needed to bootstrap it. The stable build script installs a version in the ~/.m2 folder. I put the script in my PATH and ran it.

cd ~/bin # this folder is in my path
wget http://github.com/technomancy/leiningen/raw/stable/bin/lein
chmod +x lein
mv lein lein-stable
lein-stable self-install

Next I built my checkouted Leiningen, added the checkouted lein into my path, and added the jar to ~/clj/runtime/lib

cd ~/clj/leiningen
lein-stable deps
ln -s ~/clj/leiningen/bin/lein ~/bin/lein
lein jar
ln -s ~/clj/leiningen/leiningen.jar ~/clj/runtime/lib

Next I built swank-clojure and linked the jar

cd ~/clj/swank-clojure
lein jar
ln -s ~/clj/swank-clojure/swank-clojure.jar ~/clj/runtime/lib

Next I needed to configure my .emacs file to swank-clojure and clojure-mode with some slight modifications in order to work with mismatched versions of clojure and clojure-contrib with swank-clojure

Reading ~/clj/swank-clojure/swank-clojure.el allowed me to understand all the configuration parameters for swank-clojure. I highly recommend reading that file if you want to tweak anything in swank. It's well written and has good documentation.

;; in the .emacs file, clojure configuration
;; to run slime, M-- M-x slime then type clojure

;;tells emacs where to find clojure-mode.el
(add-to-list 'load-path "~/clj/clojure-mode")
(require 'clojure-mode)

(setq swank-clojure-jar-home "~/clj/runtime/lib")
;; need to override default deps so self-install is not evoked
;; the URLs are honestly arbitrary, I essentially just removed
;; the version info so that the filenames match the filenames
;; of the jars I built
(setq swank-clojure-deps
      (list (concat "http://repo.technomancy.us/"
                    "swank-clojure.jar")
            (concat "http://build.clojure.org/snapshots/org/"
                    "clojure/clojure/1.1.0-master-SNAPSHOT/"
                    "clojure.jar")
            (concat "http://build.clojure.org/snapshots/org/"
                    "clojure/clojure-contrib/1.1.0-master-SNAPSHOT/"
                    "clojure-contrib.jar")))
(add-to-list 'load-path "~/clj/swank-clojure")
(require 'swank-clojure)
;; needed for overriding default method for invoking slime
(ad-activate 'slime-read-interactive-args)

That was it. To run slime, I just typed in emacs, "M-- M-x slime" then "clojure". I use "M--" so I can still run SBCL if I need to.

Hopefully this information may be useful to the community.