Breaking News: Grepper is joining You.com. Read the official announcement!

how to read avro file in pyspark

MazelTov27 answered on September 21, 2022 Popularity 4/10 Helpfulness 6/10

answer how to read avro file in pyspark

how to read avro file in pyspark

Comment

# Reading an AVRO file into a DataFrame
df = (spark.read.format("avro")
    .load("your_avro_file_location"))
df.show(truncate=False)

# Reading an AVRO file into a Spark SQL
spark.sql("""CREATE OR REPLCE TEMPORARY VIEW your_view_name
        USING avro
        OPTIONS(
            path "your_avro_path_location"
        )""")

# Writing DataFrame to Avro Files
(df.write
.format("avro")
.mode("overwrite")
.save("save_location_path"))

Popularity 4/10 Helpfulness 6/10 Language python

Source: Grepper

Tags: avro file pyspark python

Link to this answer
Share Copy Link

Contributed on Sep 21 2022

MazelTov27

0 Answers Avg Quality 2/10

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.

how to read avro file in pyspark

Contents

More Related Answers

how to read avro file in pyspark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.