ClojureScript to Go
Way to Go! -- Siobhan Sharpe
[2015-08-01] Was based on ClojureScript 0.0-2411 in 2014, and passes (most of) the tests, though the compiler is JVM based. Should be redone on top of self hosted ClojureScript 1.7 at some point.
ClojureScript to Go. Implemented as an overlay onto ClojureScript, instead of a fork. Go is emitted from
cljs.go.compiler, which is a patched version of
cljs.compiler. At run time, a thin JS compatibility is provided to avoid having to touch
googpackages aren't intended for end-user usage.
Once the compiler starts working, the plan is to provide a second monkey patch in spirit of the first, but this time for the Java dependencies, like
java.io.Fileand their Clojure counterpart,
clojure.java.io. This patch is to be able to compile the compiler itself to Go. This won't change the run-time characteristics of the compiled programs, nor will it introduce
eval, it's simply done to leverage the Go build pipeline. ClojureScript has a reader in
cljs.readerthat potentially can replace
tools.reader. Macros used via
:require-macrosneeds to be compiled to Go first, and an ad-hoc wrapper for the compiler depending on them will be created to compile the actual source (the basic compiler has the
cljs.coremacros built in).
While not introducing
eval, it does open up for adding it as a third layer, the problem lies on the Go side, as you need a way to build and link code into the running process. This is certainly doable in various ways, but not yet sure what the right way to do so is. One potential way is to use Go's own
cgoto both call and expose Go packages as C combined with
dlopen, potentially using go-ffi. Another is via
net/rpc. A third interesting option is to model the namespaces using go9p. Regardless the approach, it will need a way to keep or migrate the state of vars. There's also Go Execution Modes which hints at a future
-buildmode=plugin, potentially in Go 1.5.
Mainly for fun and to learn Go. But it is also trying to address issues around fast compilation and start-up time (a current concern in the Clojure world, see below). On the other side, see Fernando's motivation for his corvus LLVM Lisp: "The era of dynamic languages is over. There is currently a 'race to the bottom': Languages like Rust and Nimrod are merging the world of low-level programming with high-level compiler features." The reason to start this exploration with Go itself is most pragmatically explained in Rob's motivating talk. And, Go has momentum.
The goal isn't to compete with the JVM or V8 on performance. Another obvious reason would be to leverage the Go ecosystem using ClojureScript. It's also meant to simplify creating new emitters (at least for myself) for ClojureScript. I'm aware of
tools.analyzeretc., but they don't help you implement
clojure.coreand the actual language run-time. That said, building on a fork of
cljs.analyzeris a likely future direction. Currently, ClojureScript's analyzer is not modified, and a lot of extra mess happens in the Go compiler. Some of this could / should be moved to analyzer passes.
The easiest way to start understanding how the compiler works is to look at the emitted Go code.
core.cljscompiled to Go.
overrides.goare Go specific overrides compiled from
rt.gois a handwritten Go file providing the implementation needed, mainly
AFnand wrappers around Go
reflectcapabilities and some coercing functions (like
Truth_) used by the emitted code. ClojureScript functions are represented by this
AFnstruct, which bundles up potentially more than one Go function into a ClojureScript one.
deftypecompiles to Go structs,
defprotocolto Go interfaces. Protocol methods are real Go methods, not
seqare recognized and compiled to
float64for simplicity, but this is likely to change.
As can be seen on
ISeqabove, types and protocols are ns-prefixed to not clash with functions (like
symbol). Public functions in Go must start with an uppercase character. Functions starting with a
-) have an
Xin front of them. The arity is appended, so a full compiled name will look like this:
ArityVariadicis a special case which regardless of how many fixed parameters take a single varargs parameter which is then unpacked inside the generated body. This is to simplify dispatch and avoid having 20+ different varargs signatures (this might change). Functions with primitives (up to 3 arguments) are compiled into something like
float64and returns a
float64). These functions live beneath the normal protocol
IFndispatch. A normal (without primitives) bridge
IFnfunction is also generated (using MakeFunc) for the matching arity. Like in ClojureScript, there's a special protocol
Objectthat allow creating of methods that look like real Go methods (no
_ArityN). These methods, like any host methods or functions, are invoked by the dot notation, like
(.toString x). There's currently no way to create a plain Go
func, but this will likely become a macro.
When compiling, the unaltered
cljs.analyzerfrom ClojureScript is used to build the AST (see Nicola's AST Quickref). The analyzer depends on the
core.clj), which (like all ClojureScript macros) are written in Clojure. This file is replaced by
cljs.go.core, and heavily uses the
js*macro to emit literal Go code. Once the AST has been generated, it's fed into the
cljs.go.compiler, which emits the Go source code. This is in turn (usually) fed into
goimports(a version of
gofmtthat also fixes the imports) and then finally
Not ready yet.
Ensure you have Go 1.6.x installed and
GOPATHsetup properly, see How to Write Go Code. Then clone this repo like this:
# Clone and build the Go packages: $ go get github.com/hraberg/cljs2go # The repo itself lives under src by convention: $ cd $GOPATH/src/github.com/hraberg/cljs2go
go test, for Go tests checked into git, both generated and handwritten ones:
$ go test -v ./...
To re-generate the tests from ClojureScript (this might dirty the repo):
$ lein test
(This also generates and runs some tests under target/generated which aren't checked in.)
To re-generate the Go for ClojureScript itself:
$ go generate
While the compiler more or less works, and passes most of ClojureScript's test suite, it's not packaged for actual use, as I first intend to compile it to Go. If you want to play with it, it's easiest to fire up a REPL and look at the
The Hello World sample can be built like this:
;; Generate the Go source from a Clojure REPL: cljs.go> (compile-file "samples" "samples/hello.cljs")
# go run, < 1s $ cd samples/hello $ go run main.go Hello World
$ go build -o hello main.go
Native binary, < 50ms, 7.5M
$ ./hello Hello World
We're currently in phase 1.
funcmacro, coercions functions, ability to write moderately useful programs.
compiler.cljto compiling ClojureScript.
cljs.reader(which might need extensions) or
require-macrosto work. Assume the macros are valid ClojureScript.
core.asyncusing real Go blocks and channels.
Copyright © 2014 Håkan Råberg
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.
Copyright (c) Rich Hickey. All rights reserved. The use and distribution terms for this software are covered by the Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php) which can be found in the file epl-v10.html at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.