`clojure.core/transduce`¶

Type: function Added: Clojure 1.7 Examples: 7 Runnable: 5

([xform f coll] [xform f init coll])

reduce with a transformation of f (xf). If init is not supplied, (f) will be called to produce it. f should be a reducing step function that accepts both 1 and 2 arguments, if it accepts only 2 you can add the arity-1 with 'completing'. Returns the result of applying (the transformed) xf to init and the first item in coll, then applying xf to that result and the 2^nd item, etc. If coll contains no items, returns init and f is not called. Note that certain transforms may inject or skip items.

Examples¶

by bsvingen on 2015-07-02

by upgradingdave on 2016-09-14

;; When studying Korean, I had notes with mixture of Korean and
;; English and I wanted to filter out any English.

(def example (str "I will write an autobiography(자서전) later\n"
                  "(저는) 나중에 자서전을 쓸 거에요"))

;; Here's a transducer to filter out english characters

(defn filter-out-english 
  "filter out english characters in a string"
  []
  (filter (fn [c] 
            (let [i (int c)] 
              (not (or (and (>= i 65) (<= i 90)) 
                       (and (>= i 97) (<= i 122))))))))

;; Here's a transducer to help deal with extra spaces and newlines.
;; Notice the mapcat ensures that the output will always be the same
;; shape as the input

(defn trim-chars [c n]
  "Ensure exactly n characters c in a row. For example, squash
  multiple spaces into single space or expand newlines into 2
  newlines"
  (comp (partition-by #{c})
        (mapcat #(if (= c (first %)) (repeat n c) %))))

;; put it all together, we filter out english characters, replace
;; multiple spaces with single space, and ensure each line is double
;; spaced (two line breaks between each line)
(def xf (comp (filter-out-english) 
              (trim-chars \space 1)
              (trim-chars \newline 2)))

(apply str (transduce xf conj example))
;; => " (자서전) \n\n(저는) 나중에 자서전을 쓸 거에요"

by jvanderhyde on 2016-12-29 . May have evaluation errors.

by vspinu on 2017-06-27 . May have evaluation errors.

;;; BUILD A STATEFULL TRANSDUCER

;; Make a transducer that accumulates a sequence when pred is truthy and
;; returns individual values when pred is falsy.
;;
;; For example when pred is odd?, partition
;;
;;    [1 1 1 2 2 3 3 3]
;;    
;; into
;; 
;;    [[1 1 1] [2] [2] [3 3 3]]
;;

(defn accumulate-when [pred]
  ;; A transducer takes a reducer function and returns a reducer function.
  (fn [rf]
    ;; State (an accumulator) which is closed over by the reducer function.
    (let [acc (java.util.ArrayList.)]
      (fn
        ;; Arity 0 (state initializer). In this step we can initialize `acc`
        ;; based on the returned valued of (rf), but here, as it is usually the
        ;; case, this is not needed.
        ([] (rf))

;; Arity 1 (completer). Called after the reducing process has ended (if
        ;; ever). In this step local state must be cleaned and residual reducing
        ;; step may be performed. `result` is an unreduced value (see reduced
        ;; and unreduced).
        ([result]
         (let [result (if (.isEmpty acc)
                        ;; No residual state. Simply return the result.
                        result
                        ;; Need to clear the residual state and perform one last
                        ;; reducing step on the so far accumulated values.
                        (let [v (vec (.toArray acc))]
                          (.clear acc)
                          ;; This step might return a completed value (i.g. on
                          ;; which reduced? gives true). We need to deref it
                          ;; with `unreduced` in order to supply it to rf.
                          (unreduced (rf result v))))]
           ;; Nested rf call. Must happen once!
           (rf result)))

;; Arity 2 (reducer). This is where the main work happens.
        ([result input]
         (if (pred input)
           ;; When pred is truthy, accumulate and don't call the nested reducer.
           (do
             (.add acc input)
             result)
           ;; When pred is falsy, call nested reducer (possibly twice).
           (if (.isEmpty acc)
             ;; When accumulator is empty, reduce with a singleton.
             (rf result [input])
             (let [v (vec (.toArray acc))]
               (.clear acc)
               ;; First reduce on the accumulated sequence.
               (let [ret (rf result v)]
                 (if (reduced? ret)
                   ;; If sequence is completed, no more reductions
                   ret
                   ;; else, reduce once more with the current (falsy) input.
                   (rf ret [input])))))))))))

(def x [1 1 1 2 2 3 3 3])

;; Step through with the debugger in order to gain a better understanding of the
;; involved steps.

(transduce (accumulate-when odd?) conj x)
;; user> [[1 1 1] [2] [2] [3 3 3]]

(transduce (comp (take 4) (accumulate-when odd?)) conj x)
;; user> [[1 1 1] [2]]

(transduce (comp (accumulate-when odd?) (take 3)) conj x)
;; user> [[1 1 1] [2] [2]]

(transduce (comp (accumulate-when odd?) (take 4)) conj x)
;; user> [[1 1 1] [2] [2] [3 3 3]]

;; Clojure core statefull transducers are partition-by, partition-all, take,
;; drop, drop-while, take-nth, distinct, interpose, map-indexed and
;; keep-indexed.

by didibus on 2020-04-05

by tomdl89 on 2023-03-05

by cstml on 2025-09-05

;; transduce and reduce are not equivalent in their reduction strategy and have
;; slightly different behaviours that you should be aware of. For example, the
;; next function is a tranduce where we have a no-op xf.
(transduce identity
           (fn
             ([] (println :no-args))
             ([ret] (println [:one-arg ret]) ret)
             ([next ret] (println [:two-args ret next]) ret))
           (range 10))
;; this will print:

;; :no-args          <- observe that the f is called first with no arguments
;; [:two-args 0 nil] <- then it is called with the return from the no argument function
;; [:two-args 1 0]
;; [:two-args 2 1]
;; [:two-args 3 2]
;; [:two-args 4 3]
;; [:two-args 5 4]
;; [:two-args 6 5]
;; [:two-args 7 6]
;; [:two-args 8 7]
;; [:two-args 9 8]
;; [:one-arg 9]      <- lastly it is called with the return of the last call, in the 1 arrity

;; the "equivalent" reduce function
(reduce
 (fn
   ([] (println :no-args))
   ([ret] (println [:one-arg ret]) ret)
   ([next ret] (println [:two-args ret next]) ret))
 (range 10))
;; this will print:

;; [:two-args 1 0]
;; [:two-args 2 1]
;; [:two-args 3 2]
;; [:two-args 4 3]
;; [:two-args 5 4]
;; [:two-args 6 5]
;; [:two-args 7 6]
;; [:two-args 8 7]
;; [:two-args 9 8]

;; only the 2 arrity function version is called in this case

Note by jvanderhyde¶

Usually you use existing functions to create the transformation, using map, filter, paritition-all, etc. But you can also define your own transformations. A transformation (or transducer) is a function that takes a reducing function and returns a reducing function. See the source for take and filter for examples.

clojure.core/transduce¶

Examples¶

Note by jvanderhyde¶

See also¶

`clojure.core/transduce`¶