Breaking News: Grepper is joining You.com. Read the official announcement!

drop columns pyspark

Add Answer

Super Shrew answered on September 22, 2021 Popularity 10/10 Helpfulness 6/10

answer drop columns pyspark

related Pyspark Drop columns

related How to Drop a DataFrame/Dataset column in pyspark

drop columns pyspark

Comment

df.drop("firstname") \
  .printSchema()
""" import col is required """  
df.drop(col("firstname")) \
  .printSchema()  
  
df.drop(df.firstname) \
  .printSchema()

Popularity 10/10 Helpfulness 6/10 Language python

Source: sparkbyexamples.com

Tags: drop pyspark python

Link to this answer
Share Copy Link

Contributed on Sep 22 2021

Super Shrew

0 Answers Avg Quality 2/10

Closely Related Answers

Pyspark Drop columns

Comment

list = ['Employee ID','Employee NAME','Company Name']
  
# delete two columns
dataframe = dataframe.drop(*list)
dataframe.show()

Popularity 10/10 Helpfulness 4/10 Language whatever

Source: www.geeksforgeeks.org

Tags: drop

Link to this answer
Share Copy Link

Contributed on Jul 27 2022

Auckland Lauwo

0 Answers Avg Quality 2/10

How to Drop a DataFrame/Dataset column in pyspark

Comment

To drop a column in a PySpark DataFrame, you can use the drop method.
This method takes two arguments:

col: The name of the column you want to drop.
axis: The axis along which you want to drop the column. 
In this case, you should set axis=1 to indicate that you want to drop a column.

Here's an example of how you can use the drop method to drop a column in a 
PySpark DataFrame:


from pyspark.sql import SparkSession

# Create a SparkSession
spark = SparkSession.builder.appName("Drop Column").getOrCreate()

# Load a DataFrame
df = spark.read.csv("path/to/data.csv", header=True)

# Drop a column
df = df.drop("col_name", axis=1)
This will drop the column with the name "col_name" from the DataFrame. 
If you want to drop multiple columns, you can pass a list of column names 
to the col argument.

df = df.drop(["col_name_1", "col_name_2"], axis=1)

Keep in mind that the drop method returns a new DataFrame with the specified
column(s) removed. It does not modify the original DataFrame.

Popularity 9/10 Helpfulness 3/10 Language python

Source: Grepper

Tags: drop pysp

Link to this answer
Share Copy Link

Contributed on Dec 19 2022

Soumyaranjan Rout

0 Answers Avg Quality 2/10

drop columns pyspark

Contents

More Related Answers

drop columns pyspark

Closely Related Answers

Pyspark Drop columns

How to Drop a DataFrame/Dataset column in pyspark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.