Breaking News: Grepper is joining You.com. Read the official announcement!

how to select specific column with Dimensionality Reduction pyspark

Akshay R answered on March 1, 2022 Popularity 1/10 Helpfulness 1/10

answer how to select specific column with Dimensionality Reduction pyspark

related how to select specific column with Dimensionality Reduction pyspark

how to select specific column with Dimensionality Reduction pyspark

Comment

import org.apache.spark.mllib.linalg.Matrix
import org.apache.spark.mllib.linalg.SingularValueDecomposition
import org.apache.spark.mllib.linalg.Vector
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.linalg.distributed.RowMatrix

val data = Array(
  Vectors.sparse(5, Seq((1, 1.0), (3, 7.0))),
  Vectors.dense(2.0, 0.0, 3.0, 4.0, 5.0),
  Vectors.dense(4.0, 0.0, 0.0, 6.0, 7.0))

val rows = sc.parallelize(data)

val mat: RowMatrix = new RowMatrix(rows)

// Compute the top 5 singular values and corresponding singular vectors.
val svd: SingularValueDecomposition[RowMatrix, Matrix] = mat.computeSVD(5, computeU = true)
val U: RowMatrix = svd.U  // The U factor is a RowMatrix.
val s: Vector = svd.s     // The singular values are stored in a local dense vector.
val V: Matrix = svd.V     // The V factor is a local dense matrix.

Popularity 1/10 Helpfulness 1/10 Language whatever

Source: spark.apache.org

Tags: dimensionality-reduction pyspark select whatever

Link to this answer
Share Copy Link

Contributed on Mar 01 2022

Akshay R

0 Answers Avg Quality 2/10

Closely Related Answers

how to select specific column with Dimensionality Reduction pyspark

Comment

from pyspark.mllib.linalg import Vectors
from pyspark.mllib.linalg.distributed import RowMatrix

rows = sc.parallelize([
    Vectors.sparse(5, {1: 1.0, 3: 7.0}),
    Vectors.dense(2.0, 0.0, 3.0, 4.0, 5.0),
    Vectors.dense(4.0, 0.0, 0.0, 6.0, 7.0)
])

mat = RowMatrix(rows)

# Compute the top 5 singular values and corresponding singular vectors.
svd = mat.computeSVD(5, computeU=True)
U = svd.U       # The U factor is a RowMatrix.
s = svd.s       # The singular values are stored in a local dense vector.
V = svd.V       # The V factor is a local dense matrix.

Popularity 1/10 Helpfulness 1/10 Language python

Source: spark.apache.org

Tags: dimensionality-reduction select pyspark python

Link to this answer
Share Copy Link

Contributed on Mar 01 2022

Akshay R

0 Answers Avg Quality 2/10

how to select specific column with Dimensionality Reduction pyspark

Contents

More Related Answers

how to select specific column with Dimensionality Reduction pyspark

Closely Related Answers

how to select specific column with Dimensionality Reduction pyspark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.