Breaking News: Grepper is joining You.com. Read the official announcement!

pyspark scaling

Innocent Iguana answered on February 27, 2024 Popularity 3/10 Helpfulness 7/10

answer pyspark scaling

pyspark scaling

Comment

max_val = df.agg({'col_name': 'max'}).collect()[0][0]
min_val = df.agg({'col_name': 'min'}).collect()[0][0]
mean_val = df.agg({'col_name': 'mean'}).collect()[0][0]
std_val = df.agg({'col_name': 'stddev'}).collect()[0][0]
df = df.withColumn("min_max_scaled_col", (df['col_name'] - min_val) / (max_val - min_val))
df = df.withColumn("standard_scaled_col", (df['col_name'] - mean_val) / std_val)
df = df.withColumn("feature_scaled_col", df['col_name']  / max_val)
# Log transformation
from pyspark.sql.functions import log
df = df.withColumn('log_col', log(df['col_name']))

Popularity 3/10 Helpfulness 7/10 Language python

Source: Grepper

Tags: pyspark python scaling

Link to this answer
Share Copy Link

Contributed on Feb 27 2024

Innocent Iguana

0 Answers Avg Quality 2/10

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.

pyspark scaling

Contents

More Related Answers

pyspark scaling

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.