`clojure.core/eduction`¶

Type: function Added: Clojure 1.7 Examples: 6 Runnable: 0

([xform* coll])

Returns a reducible/iterable application of the transducers to the items in coll. Transducers are applied in order as if combined with comp. Note that these applications will be performed every time reduce/iterator is called.

Examples¶

by ghiden on 2016-01-14 . May have evaluation errors.

;; `eduction` calls the stack of transformers on each element, each time.
;; `->>` calls the transformer named 'left' on a collection, then the transformer
;; named 'right' on the result, etc.

(def cnt (atom nil))

;; inc with debugging output
(defn inc-with-print [fn-id coll-idx x]
  (printf ";; fn-id: %s; coll-idx: %s; cnt: %s; x: %s\n" fn-id coll-idx @cnt x)
  (swap! cnt inc)
  (inc x))

(reset! cnt 0)
(eduction
 (map-indexed (fn [coll-idx x] (inc-with-print " left" coll-idx x)))
 (map-indexed (fn [coll-idx x] (inc-with-print "right" coll-idx x)))
 (range 3))

;; fn-id:  left; coll-idx: 0; cnt: 0; x: 0
;; fn-id: right; coll-idx: 0; cnt: 1; x: 1
;; fn-id:  left; coll-idx: 1; cnt: 2; x: 1
;; fn-id: right; coll-idx: 1; cnt: 3; x: 2
;; fn-id:  left; coll-idx: 2; cnt: 4; x: 2
;; fn-id: right; coll-idx: 2; cnt: 5; x: 3
;;=> (2 3 4)

(reset! cnt 0)
(->> (range 3)
     (map-indexed (fn [coll-idx x] (inc-with-print " left" coll-idx x)))
     (map-indexed (fn [coll-idx x] (inc-with-print "right" coll-idx x))))

;; fn-id:  left; coll-idx: 0; cnt: 0; x: 0
;; fn-id:  left; coll-idx: 1; cnt: 1; x: 1
;; fn-id:  left; coll-idx: 2; cnt: 2; x: 2
;; fn-id: right; coll-idx: 0; cnt: 3; x: 1
;; fn-id: right; coll-idx: 1; cnt: 4; x: 2
;; fn-id: right; coll-idx: 2; cnt: 5; x: 3
;;=> (2 3 4)

by freckletonj on 2016-11-29 . May have evaluation errors.

by jvanderhyde on 2016-12-29 . May have evaluation errors.

by vspinu on 2017-06-05 . May have evaluation errors.

;; Result of eduction is of the `clojure.core.Eduction` type which acts as a
;; lazy collection that re-executes all the steps again and again. This could be
;; useful when you don't want to store the collection separately.
;;
;; Eductions can be efficiently used with `reduce` and `transduce`.

(defn inc-with-print [x]
  (println ";;" x)
  (inc x))

(def ed (eduction (map inc-with-print) (map inc-with-print) (range 3)))

(defn identity-with-print [x]
  (println "identity:" x)
  x)

(map identity-with-print ed)
;; 0
;; 1
;; 1
;; 2
;; 2
;; 3
;; identity: 2
;; identity: 3
;; identity: 4
;; => (2 3 4)

(defn sum-with-print [x y]
  (println "sum:" x "+" y)
  (+ x y))

(reduce sum-with-print ed)
;; 0
;; 1
;; 1
;; 2
;; sum: 2 + 3
;; 2
;; 3
;; sum: 5 + 4
;; => 9

;; The above line does not work in Clojure 1.10.1
;; Execution error (ClassCastException) at user/eval193 (REPL:1).
;; clojure.core.Eduction cannot be cast to clojure.lang.IReduce

(transduce (map identity-with-print) + ed)
;; 0
;; 1
;; identity: 2
;; 1
;; 2
;; identity: 3
;; 2
;; 3
;; identity: 4
;; => 9

by onionpancakes on 2024-02-03 . May have evaluation errors.

;; Converting from seqs to transducers is trivial with eduction.

(->> (range 100)   ; <- coll source
     (map inc)
     (remove odd?)
     (reduce + 0)) ; <- reduction target

;; Refactor by placing an eduction in the middle.

(->> (range 100)
     (eduction (map inc)
               (remove odd?))
     (reduce + 0)) ; => 2550

;; Because Eduction type implements clojure.lang.IReduceInit,
;; and its reduce implementation calls transduce,
;; it is equivalent to this.

(transduce (comp (map inc) (remove odd?)) + 0 (range 100)) ; => 2550

;; So long as the target is a reduce, or is implemented by reduce,
;; transduce will be called. It is a good idea to use clojure.repl/source
;; to check.

(source mapv)

;; Examples functions implemented by reduce include:
;; - into
;; - mapv
;; - filterv
;; - frequencies
;; - group-by
;; - run!

;; Since Eduction calls transduce on the given coll,
;; for maximum performance, the coll passed to eduction should
;; ideally implement clojure.lang.IReduceInit.
;; Having both a clojure.lang.IReduceInit coll and a reduce target
;; cause the reduce to be executed on the internal fast path.
;; It is a good idea to verify with the instance? function.

(instance? clojure.lang.IReduceInit (range 100)) ; => true

;; Example clojure.lang.IReduceInit types include:
;; - range
;; - vector
;; - iterate
;; - eduction

by onionpancakes on 2024-02-20 . May have evaluation errors.

;; Use eduction within mapcat transducers
;; to avoid intermediate allocations.

(->> [1 2 2 2 3 4 4 4 4 5 5]
     (eduction (partition-by identity)
               (mapcat (fn [nums]
                         ;; This makes a intermediate vector!
                         (into [] (map #(* (count nums) %)) nums))))
     (vec))
;; => [1 6 6 6 3 16 16 16 16 10 10]

;; Refactor using eduction inside mapcat instead!

(->> [1 2 2 2 3 4 4 4 4 5 5]
     (eduction (partition-by identity)
               (mapcat (fn [nums]
                         ;; Allocates a cheaper intermediate eduction
                         ;; in place of a collection
                         (eduction (map #(* (count nums) %)) nums))))
     (vec))
;; => [1 6 6 6 3 16 16 16 16 10 10]

;; This works because cat (and by extension mapcat) is implemented with reduce.
;; See the source

(source cat)

;; In general, pass IReduceInit types into cat transducers for maximum performance.
;; Use eduction in particular to create a non-coll-allocating IReduceInit.

Note by glts¶

eduction is particularly useful as an adapter for collection-processing functions that don’t have a transducers arity. For example, you might want to transform a collection before passing it to frequencies. eduction makes that possible and efficient:

(->> coll
     (eduction (map #(quot % 5))
               (filter odd?))
     frequencies)

So eduction turns out to be quite powerful in that it brings the benefits of transducers to all collection-processing functions, even those that predate transducers: first, last, group-by, run!, str/join, …

Note by glts¶

Interesting background about sequence versus eduction: https://groups.google.com/d/msg/clojure/9I6MtgOTD0w/NiG5PimBCP8J

Note by Hindol¶

eduction does not cache the result while sequence does. eduction is intended to be used when you are eventually going to reduce over the sequence.

https://groups.google.com/d/msg/clojure/9I6MtgOTD0w/NiG5PimBCP8J

clojure.core/eduction¶

Examples¶

Note by glts¶

Note by glts¶

Note by Hindol¶

See also¶

`clojure.core/eduction`¶