RowMatrix

Instance Constructors

new RowMatrix(rows: RDD[Vector])

Alternative constructor leaving matrix dimensions to be determined automatically.
new RowMatrix(rows: RDD[Vector], nRows: Long, nCols: Int)

rows
rows stored as an RDD[Vector]
nRows
number of rows. A non-positive value means unknown, and then the number of rows will be determined by the number of records in the RDD rows.
nCols
number of columns. A non-positive value means unknown, and then the number of columns will be determined by the size of the first row.

Value Members

final def !=(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def !=(arg0: Any): Boolean

Definition Classes
Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def ==(arg0: Any): Boolean

Definition Classes
Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def computeColumnSummaryStatistics(): MultivariateStatisticalSummary

Computes column-wise summary statistics.
def computeCovariance(): Matrix

Computes the covariance matrix, treating each row as an observation.
Computes the covariance matrix, treating each row as an observation.
returns
a local dense matrix of size n x n
def computeGramianMatrix(): Matrix

Computes the Gramian matrix A^T A.
def computePrincipalComponents(k: Int): Matrix

Computes the top k principal components.
Computes the top k principal components. Rows correspond to observations and columns correspond to variables. The principal components are stored a local matrix of size n-by-k. Each column corresponds for one principal component, and the columns are in descending order of component variance.
k
number of top principal components.
returns
a matrix of size n-by-k, whose columns are principal components
def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[RowMatrix, Matrix]

Computes the singular value decomposition of this matrix.
Computes the singular value decomposition of this matrix. Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
There is no restriction on m, but we require n^2 doubles to fit in memory. Further, n should be less than m.
The decomposition is computed by first computing A'A = V S^{2 V',
computing svd locally on that (since n x n is small), from which we recover S and V.
Then we compute U via easy matrix multiplication as U = A * (V * S}-1). Note that this approach requires O(n^3) time on the master node.
At most k largest non-zero singular values and associated vectors are returned. If there are k such values, then the dimensions of the return will be:
U is a RowMatrix of size m x k that satisfies U'U = eye(k), s is a Vector of size k, holding the singular values in descending order, and V is a Matrix of size n x k that satisfies V'V = eye(k).
k
number of singular values to keep. We might return less than k if there are numerically zero singular values. See rCond.
computeU
whether to compute U
rCond
the reciprocal condition number. All singular values smaller than rCond * sigma(0) are treated as zero, where sigma(0) is the largest singular value.
returns
SingularValueDecomposition(U, s, V)
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def multiply(B: Matrix): RowMatrix

Multiply this matrix by a local matrix on the right.
Multiply this matrix by a local matrix on the right.
B
a local matrix whose number of rows must match the number of columns of this matrix
returns
a org.apache.spark.mllib.linalg.distributed.RowMatrix representing the product, which preserves partitioning
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def numCols(): Long

Gets or computes the number of columns.
Gets or computes the number of columns.

Definition Classes
RowMatrix → DistributedMatrix
def numRows(): Long

Gets or computes the number of rows.
Gets or computes the number of rows.

Definition Classes
RowMatrix → DistributedMatrix
val rows: RDD[Vector]

rows stored as an RDD[Vector]
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

class RowMatrix extends DistributedMatrix with Logging

Instance Constructors

new RowMatrix(rows: RDD[Vector])

new RowMatrix(rows: RDD[Vector], nRows: Long, nCols: Int)

Value Members

final def !=(arg0: AnyRef): Boolean

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: AnyRef): Boolean

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

def computeColumnSummaryStatistics(): MultivariateStatisticalSummary

def computeCovariance(): Matrix

def computeGramianMatrix(): Matrix

def computePrincipalComponents(k: Int): Matrix

def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[RowMatrix, Matrix]

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

def isTraceEnabled(): Boolean

def log: Logger

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

def logDebug(msg: ⇒ String): Unit

def logError(msg: ⇒ String, throwable: Throwable): Unit

def logError(msg: ⇒ String): Unit

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

def logInfo(msg: ⇒ String): Unit

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

def logTrace(msg: ⇒ String): Unit

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

def logWarning(msg: ⇒ String): Unit

def multiply(B: Matrix): RowMatrix

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def numCols(): Long

def numRows(): Long

val rows: RDD[Vector]

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Logging

Inherited from DistributedMatrix

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped