Improving Development Startup Time

Did you know:

  • Most libraries are distributed as source and you compile them over and over

  • You can compile namespaces explicitly using compile

  • Namespace compilation is transitive

  • compile writes these files to disk and require will use them

  • You can use compile on the namespace you load when you start development, or on your user.clj, or on the main namespace you run as a server to improve your startup time

The compile function takes a namespace symbol and compiles that namespace and all the namespaces it requires into *compile-path* (which defaults to classes). That directory must exist and be on your classpath:

(compile 'my.namespace)    ;; writes .class files to *compile-path*

Subsequently, when any of those compiled namespaces are required, the class file will be loaded, rather than the original .clj file. If a source file is updated (and thus newer), it will be loaded instead. Periodically, you will need to re-compile to account for new dependencies or changing code.

Note that compilation is a side effect of loading, so has no effect on already loaded namespaces. In particular, one special case is the user.clj file, which is loaded automatically by the Clojure runtime. If you are using a user.clj in dev (or need to compile namespaces that are already compiled), you can do so by forcing a reload while compiling:

(binding [*compile-files* true] ;; compile during load
  (require 'user :reload-all))  ;; reload this and all transitively loaded namespaces

That’s it! This technique can substantially reduce your startup time during development, particularly as the number of dependencies you load increases.

Using an alias for development

In a deps.edn project, you should create a development alias (like :dev) that includes the classes directory:

{:deps { ... }
 {:dev {:extra-paths ["classes"]}}}

You will also need to ensure the classes directory exists. It may be useful to include the empty classes directory as part of your version controlled project structure (make sure to ignore and NOT include the compiled .class files).

You can then start your REPL with the alias and compile to populate your classes directory and start seeing the benefits:

$ clj -A:dev
Clojure 1.10.1
user=> (compile 'my.namespace)

Incorporating user.clj into your development alias

If you want to use an automatically loaded user.clj, you should incorporate that into your dev alias by adding a source path dev:

{:deps { ... }
 {:dev {:extra-paths ["dev" "classes"]}}}

And then create dev/user.clj:

(ns user
  (:require ... ))

;; dev-only functions, etc

Remember to use the modified compilation process for user.clj:

$ clj -A:dev
Clojure 1.10.1
user=> (binding [*compile-files* true] (require 'user :reload-all))

Original author: Alex Miller