Breaking News: Grepper is joining You.com. Read the official announcement!

how to drop column in spark dataframe

MazelTov27 answered on September 22, 2022 Popularity 9/10 Helpfulness 3/10

answer how to drop column in spark dataframe

related How to Drop a DataFrame/Dataset column in pyspark

how to drop column in spark dataframe

Comment

# Use drop() method
foo3 = foo2.drop("delay") 
foo3.show()

Popularity 9/10 Helpfulness 3/10 Language python

Source: Grepper

Tags: drop python spark-dataframe

Link to this answer
Share Copy Link

Contributed on Sep 22 2022

MazelTov27

0 Answers Avg Quality 2/10

Closely Related Answers

How to Drop a DataFrame/Dataset column in pyspark

Comment

To drop a column in a PySpark DataFrame, you can use the drop method.
This method takes two arguments:

col: The name of the column you want to drop.
axis: The axis along which you want to drop the column. 
In this case, you should set axis=1 to indicate that you want to drop a column.

Here's an example of how you can use the drop method to drop a column in a 
PySpark DataFrame:


from pyspark.sql import SparkSession

# Create a SparkSession
spark = SparkSession.builder.appName("Drop Column").getOrCreate()

# Load a DataFrame
df = spark.read.csv("path/to/data.csv", header=True)

# Drop a column
df = df.drop("col_name", axis=1)
This will drop the column with the name "col_name" from the DataFrame. 
If you want to drop multiple columns, you can pass a list of column names 
to the col argument.

df = df.drop(["col_name_1", "col_name_2"], axis=1)

Keep in mind that the drop method returns a new DataFrame with the specified
column(s) removed. It does not modify the original DataFrame.

Popularity 9/10 Helpfulness 3/10 Language python

Source: Grepper

Tags: drop pyspark pys

Link to this answer
Share Copy Link

Contributed on Dec 19 2022

Soumyaranjan Rout

0 Answers Avg Quality 2/10

how to drop column in spark dataframe

Contents

More Related Answers

how to drop column in spark dataframe

Closely Related Answers

How to Drop a DataFrame/Dataset column in pyspark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.