Cascalog: start uberjar and main on hadoop

Question

I have compiled an uberjar from a file like:

(defmain HadoopTest (:use 'cascalog.api) (defn bla ("alot of code"))

I run that uberjar on hadoop like:

$ hadoop jar myStandalone.jar clojure.main

and i get a REPL, but nothing from that file is executed. I still have to type (:use 'cascalog.api) and (defn bla) by hand. Why is that the case and how do i fix it?

thanks a lot!

Answer 1

If you supply a class name to hadoop jar <jar file> [<main class>] ... it will call the main method that is contained in that class. Since you are using clojure.main here, a REPL will be spun up (because that's what clojure.main.main() is supposed to do).

So, either use the right class (your AOT-compiled Clojure namespace, I guess), or store that information in your Uberjar (e.g. via Leiningen's :main key in the project file) and leave out the classname, calling only hadoop jar myStandalone.jar.