Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR PYTHON

read parquet from s3 and convert to dataframe

import pyarrow.parquet as pq
import s3fs

dataset = pq.ParquetDataset('s3://<s3_path_to_folder_or_file>', 
filesystem=s3fs.S3FileSystem(), filters=[('colA', '=', 'some_value'), ('colB', '>=', some_number)])
table = dataset.read()
df = table.to_pandas()
Source by stackoverflow.com #
 
PREVIOUS NEXT
Tagged: #read #parquet #convert #dataframe
ADD COMMENT
Topic
Name
8+3 =