Python

Search

wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) 
             .map(lambda word: (word, 1)) 
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Comment

PREVIOUS	NEXT

Code Example
Python :: pyad create user
Python :: find in python
Python :: np.array_equal
Python :: python venv
Python :: python sound
Python :: convert to string in python
Python :: avoid bad request django
Python :: aiohttp specify app IP
Python :: get raster corners python
Python :: python add new key to dictionary
Python :: python one line key increment or add
Python :: make array consecutive 2 python
Python :: python create pem file
Python :: how to inheritance in python
Python :: union type python
Python :: looping over dictionary python
Python :: float inf in python
Python :: decode a qrcode inpython
Python :: :: python
Python :: odoo manifest
Python :: python dictionary map function
Python :: boder color in tkinter
Python :: if in one line python
Python :: decorators in python
Python :: pandas pull value from column
Python :: Dictionary Cache
Python :: django request.data
Python :: df.pivot_table
Python :: join list of string into a single string with comma
Python :: how to create an app under a folder in django

Search

PYTHON

wordcount pyspark

ADD CONTENT