Clojure error- GC overhead limit exceeded using reducer with lazy sequence

241 views Asked by At
; Test 1 - Using Map Reduce (Successful)
(ns example
    (:gen-class))
(require '[clojure.core.reducers :as r])

(def n 100000000000)
(time (println "map: " (reduce + 0N (map inc (range n)))))

I get: map: 5000000000050000000000N "Elapsed time: 8540888.550507 msecs"

; Test 2 - Using Map Reducer (Creates GC Error)
(ns example
     (:gen-class))
(require '[clojure.core.reducers :as r])

(def n 100000000000)
(time (println  "rmap: " (reduce + 0N (r/map inc (range n)))))

I get: Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded, compiling:... Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

; Test 3 - Using Reducer with Fold (Creates GC Error)
(ns example
  (:gen-class))
(require '[clojure.core.reducers :as r])

(def n 100000000000)
(time (println "fold: " (r/fold + (r/map inc (range n)))))

I get: Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded, compiling:... Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

Would have expected all three to produce the same result. Instead only #1 works but the other two have GC issues.

Note: You can get all three to work using smaller values of n.

0

There are 0 answers