Breaking News: Grepper is joining You.com. Read the official announcement!

pyspark get hour from timestamp

macarrony00 answered on April 13, 2023 Popularity 5/10 Helpfulness 6/10

answer pyspark get hour from timestamp

pyspark get hour from timestamp

Comment

from pyspark.sql.functions import *
# Step 1: transform to the correct col format
df = df.withColumn("timestamp", to_timestamp("timestamp", 'yyyy-MM-dd HH:mm:ss'))

# Step 2 & 3: Extract the needed information
df = df.withColumn('Date', date(df.timestamp))
df = df.withColumn('Hour', hour(df.timestamp))

# Display the result
df.show(1, False)
#+----------+--------------------+-------------------+-------------------+
#|identifier|           timestamp|               Date|               Hour|
#+----------+--------------------+-------------------+-------------------+
#|         1|2020-03-19 03:00:...|                 19|                 03|
#+----------+--------------------+-------------------+-------------------+

Popularity 5/10 Helpfulness 6/10 Language whatever

Source: stackoverflow.com

Tags: get hour pyspark timestamp whatever

Link to this answer
Share Copy Link