Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR PYTHON

Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder

# if there is no existing SparkSession, creates a new one

s1 = SparkSession.builder.config("k1", "v1").getOrCreate()
s1.conf.get("k1") == s1.sparkContext.getConf().get("k1") == "v1"
# True

# In case an existing SparkSession is returned

s2 = SaprkSession.builder.config("k2", "v2").getOrCreate()
s1.conf.get("k1") == s2.conf.get("k1")
# True
s1.conf.get("k2") == s2.conf.get("k2")
# True
Source by spark.apache.org #
 
PREVIOUS NEXT
Tagged: #Gets #existing #SparkSession #existing #creates #based #options #set #builder
ADD COMMENT
Topic
Name
5+9 =