Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR PYTHON

Reduces the elements of this RDD using the specified commutative and associative binary operator

from operator import add
sc.parallelize([1, 2, 3, 4, 5]).reduce(add)
# 15
sc.parallelize((2 for _ in range(10))).map(lambda x: 1).cache().reduce(add)
# 10
sc.parallelize([]).reduce(add)
"""
Traceback (most recent call last):
    ...
ValueError: Can not reduce() empty RDD
"""
Source by spark.apache.org #
 
PREVIOUS NEXT
Tagged: #Reduces #elements #RDD #commutative #associative #binary #operator
ADD COMMENT
Topic
Name
9+2 =