Python

Search

pandas dataframe to parquet s3

import awswrangler as wr
wr.pandas.to_parquet(
    dataframe=df,
    path="s3://my-bucket/key/my-file.parquet"
)

Comment

read parquet from s3 and convert to dataframe

import pyarrow.parquet as pq
import s3fs

dataset = pq.ParquetDataset('s3://<s3_path_to_folder_or_file>', 
filesystem=s3fs.S3FileSystem(), filters=[('colA', '=', 'some_value'), ('colB', '>=', some_number)])
table = dataset.read()
df = table.to_pandas()

Comment

PREVIOUS	NEXT

Code Example
Python :: taille du liste python
Python :: hugingface ner
Python :: split pdf python
Python :: python runserver port
Python :: hover show all Y values in Bokeh
Python :: python community
Python :: django admin create project
Python :: python ord()
Python :: selenium get h1 text python
Python :: python cast to float
Python :: np ln
Python :: argmax implementation
Python :: how to change values of dictionary in python
Python :: How to install packages offline? Ask Question
Python :: how to delete previous message using discord.py
Python :: tk is not defined python 3
Python :: flask docs
Python :: return max value in list python
Python :: sqlalchemy convert row to dict
Python :: how to return number in binary python
Python :: py string in list
Python :: list reverse method in python
Python :: django x-frame-options allowall
Python :: how to install httplib in python
Python :: discord py fetch message
Python :: swapping in python
Python :: count no of nan in a 2d array python
Python :: soup.find_all attr
Python :: python json write utf 8
Python :: how to plot box plot python

Search

PYTHON

pandas dataframe to parquet s3

read parquet from s3 and convert to dataframe

ADD CONTENT