Spark dataframe Examples: Reading and Writing Dataframes

Last updated:
Table of Contents

WIP Alert This is a work in progress. Current information is correct but more content may be added in the future.

Read parquet from S3

val df = spark.read.parquet("s3://path/to/df.parquet")

Write parquet to S3

val df = ....

df.write.mode("overwrite").parquet("s3://path/to/df.parquet")

Dialogue & Discussion