Python

Search

drop columns delta table

schema='<schema>'
table='<table>'

columnstoDrop = ["<column or columns to drop>"]
display(spark.sql(f"describe table {schema}.{table}"))
df = spark.sql(f"Select * from {schema}.{table}")

df1 = df.drop(*columnstoDrop)

spark.sql(f"DROP TABLE if exists {schema}.{table}")

df1.write 
  .format(write_format) 
  .mode('overwrite') 
  .option("overwriteSchema", "true") 
  .save(f"{save_path}{table}")

spark.sql(f"CREATE  TABLE  {schema}.{table}  USING DELTA LOCATION '{save_path}{table}'")

display(spark.sql(f"describe table {schema}.{table}"))

Comment

PREVIOUS	NEXT

Code Example
Python :: get complete path from reletive path python
Python :: How to use glob.escape() function in python
Python :: install python 3.10 pip
Python :: python typing optional argument
Python :: python split respect quotes
Python :: grab element based on text from html page in python
Python :: Python NumPy squeeze function Syntax
Python :: Python NumPy atleast_1d Function Syntax
Python :: Python NumPy Shape function example verifying the value of last dimension
Python :: display colors in python console
Python :: use count() function to find if a row is there in sqlite database or not.
Python :: Python NumPy asfarray Function Syntax
Python :: Python NumPy vstack Function Example with 1d array
Python :: objects list
Python :: fpdf latin-1
Python :: pandas dt.weekday to string
Python :: modles en django
Python :: else clause in for loop python
Python :: Snippet for inverse a matrix using numpy
Python :: Printing a long code line to span over multiple lines
Python :: Python matplotlib multiple bars
Python :: Fatal Python error: Cannot recover from stack overflow.
Python :: python to dart converter
Python :: pandas dataframe select columns multiple cell value
Python :: how to make a half pyramid in python
Python :: find sum of all elements in a matrix by python
Python :: Wtforms: How to generate blank value using select fields with dynamic choice values
Python :: python compare number with a precision
Python :: how to close python in terminal
Python :: delet categories from coco dataset

Search

PYTHON

drop columns delta table

ADD CONTENT