Clojure

tools.build Guide

tools.build is a library of functions for building Clojure projects. It is intended to be used in a build program to create user-invokable target functions. Also see the API docs.

Builds are programs

The philosophy behind tools.build is that your project build is inherently a program - a series of instructions to create one or more project artifacts from your project source files. We want to write this program with our favorite programming language, Clojure, and tools.build is a library of functions commonly needed for builds that can be connected together in flexible ways. Writing a build program does take a bit more code than other declarative approaches, but can be easily extended or customized far into the future, creating a build that grows with your project.

Setup

There is no installation step - tools.build is simply a library that your build program uses. You will create an alias in your deps.edn that includes tools.build as a dependency and a source path to the build program. Builds are designed to be easily executed as a project "tool" in the Clojure CLI (with -T). In the Clojure CLI, "tools" are programs that provide functionality and do not use your project deps or classpath. Tools executed with -T:an-alias remove all project deps and paths, add "." as a path, and include any other deps or paths as defined in :an-alias.

As such, you will need an alias in your deps.edn that defines the build classpath and includes the path to your build source, for example:

{:paths ["src"] ;; project paths
 :deps {}       ;; project deps

 :aliases
 {;; Run with clj -T:build function-in-build
  :build {:deps {io.github.clojure/tools.build {:git/tag "TAG" :git/sha "SHA"}}
          :ns-default build}}}

Find the most recent TAG and SHA to use at https://github.com/clojure/tools.build#release-information.

The git dep and Clojure CLI examples in this guide assume the use of Clojure CLI 1.10.3.933 or higher.

As mentioned above, running a tool with -T will create a classpath that does not include the project :paths and :deps. Using -T:build will use only the :paths and :deps from the :build alias. The root deps.edn is still included, which will pull in Clojure as well (but it would also come in as a dependency of tools.build). The :paths are not specified here, so no additional paths are added, however, -T includes the project root "." as a path by default.

So executing clj -T:build jar will use an effective classpath here of:

  • "." (added by -T)

  • org.clojure/clojure (from the root deps.edn :deps) and transitive deps

  • org.clojure/tools.build (from the :build alias :deps) and transitive deps

The :ns-default specifies the default Clojure namespace to find the function specified on the classpath. Because the only local path is the default ".", we should expect to find the build program at build.clj in the root of our project. Note that the path roots (via the :build alias :paths) and the namespace of the build program itself relative to those paths roots are fully under your control. You may wish to put them in a subdirectory of your project too.

And then finally, on the command line we specify the function to run in the build, here jar. That function will be executed in the build namespace, and passed a map built using the same arg passing style as -X - args are provided as alternating keys and values.

The remainder of this guide demonstrates individual common use cases and how to satisfy them with tools.build programs.

Source library jar build

The most common Clojure build creates a jar file containing Clojure source code. To do this with tools.build we’ll use the following tasks:

  • create-basis - to create a project basis (note: this will download deps as a side effect)

  • copy-dir - to copy Clojure source and resources into a working dir

  • write-pom - to write a pom file in the working dir

  • jar - to jar up the working dir into a jar file

The build.clj will look like this:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def lib 'my/lib1)
(def version (format "1.2.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def jar-file (format "target/%s-%s.jar" (name lib) version))

;; delay to defer side effects (artifact downloads)
(def basis (delay (b/create-basis {:project "deps.edn"})))

(defn clean [_]
  (b/delete {:path "target"}))

(defn jar [_]
  (b/write-pom {:class-dir class-dir
                :lib lib
                :version version
                :basis @basis
                :src-dirs ["src"]})
  (b/copy-dir {:src-dirs ["src" "resources"]
               :target-dir class-dir})
  (b/jar {:class-dir class-dir
          :jar-file jar-file}))

Some things to notice:

  • This is just normal Clojure code - you can load this namespace in your editor and develop it interactively at the REPL.

  • As a single-purpose program, it’s fine to build shared data in the set of vars at the top.

  • We are choosing to build in the "target" directory and assemble the jar contents in "target/classes" but there is nothing special about these paths - it is fully in your control. Also, we’ve repeated those paths and others in multiple places here but you can remove that duplication to the extent that feels right.

  • We’ve used the tools.build task functions to assemble larger functions like build/jar for the user to invoke. These functions take a parameter map and we’ve chosen not to provide any configurable parameters here, but you could!

The deps.edn file will look like this:

{:paths ["src"]
 :aliases
 {:build {:deps {io.github.clojure/tools.build {:git/tag "TAG" :git/sha "SHA"}}
          :ns-default build}}}

And then you can run this build with:

clj -T:build clean
clj -T:build jar

We expect to be able to do these both together on the command line but that is a work in progress.

Compiled uberjar application build

When preparing an application, it is common to compile the full app + libs and assemble the entire thing as a single uberjar.

It is important that your main Clojure namespace should have (:gen-class), for example:

(ns my.lib.main
  ;; any :require and/or :import clauses
  (:gen-class))

And that namespace should have a function like:

(defn -main [& args]
  (do-stuff))

An example build for a compiled uberjar will look like this:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def lib 'my/lib1)
(def version (format "1.2.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def uber-file (format "target/%s-%s-standalone.jar" (name lib) version))

;; delay to defer side effects (artifact downloads)
(def basis (delay (b/create-basis {:project "deps.edn"})))

(defn clean [_]
  (b/delete {:path "target"}))

(defn uber [_]
  (clean nil)
  (b/copy-dir {:src-dirs ["src" "resources"]
               :target-dir class-dir})
  (b/compile-clj {:basis @basis
                  :ns-compile '[my.lib.main]
                  :class-dir class-dir})
  (b/uber {:class-dir class-dir
           :uber-file uber-file
           :basis @basis
           :main 'my.lib.main}))

This example directs compile-clj to compile the main namespace (by default source will be loaded from the basis :paths). Compilation is transitive and all namespaces loaded by the compiled namespace will also be compiled. You may need to add additional namespaces if code is dynamically or optionally loaded.

The deps.edn and build execution will look the same as the prior example.

You can create the uber jar build with:

clj -T:build uber

The output of this build will be an uberjar at target/lib1-1.2.100-standalone.jar. That jar contains both a compiled version of this project and all of its dependencies. The uberjar will have a manifest referring to the my.lib.main namespace (which should have a -main method) and can be invoked like this:

java -jar target/lib1-1.2.100-standalone.jar

Parameterized builds

In the builds above we did not parameterize any aspect of the build, just chose which functions to call. You may find that it’s useful to parameterize your builds to differentiate dev/qa/prod, or version, or some other factor. To account for function chaining at the command line, it is advisable to establish the common set of parameters to use across your build functions and have each function pass the parameters along.

For example, consider a parameterization that includes an extra set of dev resources to set a local developer environment. We’ll use a simple :env :dev kv pair to indicate this:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def lib 'my/lib1)
(def version (format "1.2.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def jar-file (format "target/%s-%s.jar" (name lib) version))
(def copy-srcs ["src" "resources"])

;; delay to defer side effects (artifact downloads)
(def basis (delay (b/create-basis {:project "deps.edn"})))

(defn clean [params]
  (b/delete {:path "target"})
  params)

(defn jar [{:keys [env] :as params}]
  (let [srcs (if (= env :dev) (cons "dev-resources" copy-srcs) copy-srcs)]
    (b/write-pom {:class-dir class-dir
                  :lib lib
                  :version version
                  :basis @basis
                  :src-dirs ["src"]})
    (b/copy-dir {:src-dirs srcs
                 :target-dir class-dir})
    (b/jar {:class-dir class-dir
            :jar-file jar-file})
    params))

The other aspects of deps.edn and invocation remain the same.

Invocation that activates :dev environment will look like this:

clj -T:build jar :env :dev

The kv params are passed to the jar function.

Mixed Java / Clojure build

A common case that occurs is needing to introduce a Java implementation class or two into a mostly Clojure project. In this case, you need to compile the Java classes and include them with your Clojure source. In this setup, we’ll assume that your Clojure source is in src/ and Java source is in java/ (where you actually put these is of course up to you).

This build creates a jar with classes compiled from Java sources and your Clojure sources.

(ns build
  (:require [clojure.tools.build.api :as b]))

(def lib 'my/lib1)
(def version (format "1.2.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def jar-file (format "target/%s-%s.jar" (name lib) version))

;; delay to defer side effects (artifact downloads)
(def basis (delay (b/create-basis {:project "deps.edn"})))

(defn clean [_]
  (b/delete {:path "target"}))

(defn compile [_]
  (b/javac {:src-dirs ["java"]
            :class-dir class-dir
            :basis @basis
            :javac-opts ["--release" "11"]}))

(defn jar [_]
  (compile nil)
  (b/write-pom {:class-dir class-dir
                :lib lib
                :version version
                :basis @basis
                :src-dirs ["src"]})
  (b/copy-dir {:src-dirs ["src" "resources"]
               :target-dir class-dir})
  (b/jar {:class-dir class-dir
          :jar-file jar-file}))

The compile task here can also be used as the prep task for this lib.

Task documentation

See the API docs for detailed task documentation.

Original author: Alex Miller