Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR SCALA

repartition in spark scala


  val spark:SparkSession = SparkSession.builder()
    .master("local[5]")
    .appName("SparkByExamples.com")
    .getOrCreate()

 val df = spark.range(0,20)
 println(df.rdd.partitions.length)

 df.write.mode(SaveMode.Overwrite)csv("partition.csv")
Source by sparkbyexamples.com #
 
PREVIOUS NEXT
Tagged: #repartition #spark #scala
ADD COMMENT
Topic
Name
2+2 =