Breaking News: Grepper is joining You.com. Read the official announcement!

spark session creation examples

Add Answer

Anush Sinha answered on November 27, 2020 Popularity 9/10 Helpfulness 3/10

answer spark session creation examples

related making spark session

related spark session

related Spark SEssion object

spark session creation examples

Comment

# Need to import to use date time
from datetime import datetime, date
 
# need to import for working with pandas
import pandas as pd
 
# need to import to use pyspark
from pyspark.sql import Row
 
# need to import for session creation
from pyspark.sql import SparkSession
 
# creating the session
spark = SparkSession.builder.getOrCreate()
 
# schema creation by passing list
df = spark.createDataFrame([
    Row(a=1, b=4., c='GFG1', d=date(2000, 8, 1),
        e=datetime(2000, 8, 1, 12, 0)),
   
    Row(a=2, b=8., c='GFG2', d=date(2000, 6, 2),
        e=datetime(2000, 6, 2, 12, 0)),
   
    Row(a=4, b=5., c='GFG3', d=date(2000, 5, 3),
        e=datetime(2000, 5, 3, 12, 0))
])
 
# show table
df.show()
 
# show schema
df.printSchema()

Popularity 9/10 Helpfulness 3/10 Language whatever

Source: www.geeksforgeeks.org

Tags: creation session whatever

Link to this answer
Share Copy Link

Contributed on Apr 19 2022

Anush Sinha

0 Answers Avg Quality 2/10

Closely Related Answers

making spark session

Comment

from pyspark.sql import SparkSession

spark = SparkSession \
    .builder \
    .appName("Python Spark SQL basic example") \
    .config("spark.some.config.option", "some-value") \
    .getOrCreate()

Popularity 10/10 Helpfulness 9/10 Language python

Source: Grepper

Tags: python py

Link to this answer
Share Copy Link

Contributed on Nov 27 2020

Okabe

0 Answers Avg Quality 2/10

spark session

Comment

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

Popularity 8/10 Helpfulness 4/10 Language whatever

Source: spark.apache.org

Tags: session whatever w

Link to this answer
Share Copy Link

Contributed on Oct 12 2022

codeWithSaboor

0 Answers Avg Quality 2/10

spark session

Comment

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

Popularity 8/10 Helpfulness 2/10 Language whatever

Source: spark.apache.org

Tags: session

Link to this answer
Share Copy Link

Contributed on Oct 12 2022

codeWithSaboor

0 Answers Avg Quality 2/10

Spark SEssion object

Comment

spark = (
...     SparkSession.builder
...         .master("local")
...         .appName("Word Count")
...         .config("spark.some.config.option", "some-value")
...         .getOrCreate()
... )

Popularity 5/10 Helpfulness 1/10 Language python

Source: spark.apache.org

Tags: object python session

Link to this answer
Share Copy Link

Contributed on Apr 12 2024

Sumit Rawal

0 Answers Avg Quality 2/10

spark session creation examples

Contents

More Related Answers

spark session creation examples

Closely Related Answers

making spark session

spark session

spark session

Spark SEssion object

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.