Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR PYTHON

pyspark pivot max aggregation

from pyspark.sql.functions import *
from pyspark.sql import Window
var win = Window.partitionBy("date") 
data.withColumn("max_vol",max("volume").over(win)).groupBy("date","max_vol") .pivot("recipe") .agg(avg("percent")).show()
Source by stackoverflow.com #
 
PREVIOUS NEXT
Tagged: #pyspark #pivot #max #aggregation
ADD COMMENT
Topic
Name
3+3 =