org.apache.spark.sql.execution

Limit

case class Limit(limit: Int, child: SparkPlan)(sqlContext: SQLContext) extends SparkPlan with UnaryNode with Product with Serializable

:: DeveloperApi :: Take the first limit elements. Note that the implementation is different depending on whether this is a terminal operator or not. If it is terminal and is invoked using executeCollect, this operator uses something similar to Spark's take method on the Spark driver. If it is not terminal or is invoked using execute, we first take the limit on each partition, and then repartition all the data to a single partition to compute the global limit.

Annotations
@DeveloperApi()
Linear Supertypes
Serializable, Serializable, Product, Equals, UnaryNode, catalyst.trees.UnaryNode[SparkPlan], SparkPlan, com.typesafe.scalalogging.slf4j.Logging, QueryPlan[SparkPlan], TreeNode[SparkPlan], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Limit
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. UnaryNode
  7. UnaryNode
  8. SparkPlan
  9. Logging
  10. QueryPlan
  11. TreeNode
  12. AnyRef
  13. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Limit(limit: Int, child: SparkPlan)(sqlContext: SQLContext)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. def apply(number: Int): SparkPlan

    Definition Classes
    TreeNode

  7. def argString: String

    Definition Classes
    TreeNode

  8. def asCode: String

    Definition Classes
    TreeNode

  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  10. def buildRow(values: Seq[Any]): Row

    Attributes
    protected
    Definition Classes
    SparkPlan

  11. val child: SparkPlan

    Definition Classes
    Limit → UnaryNode

  12. def children: List[SparkPlan]

    Definition Classes
    UnaryNode

  13. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  14. def collect[B](pf: PartialFunction[SparkPlan, B]): Seq[B]

    Definition Classes
    TreeNode

  15. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  16. def execute(): RDD[catalyst.expressions.Row]

    Runs this query returning the result as an RDD.

    Runs this query returning the result as an RDD.

    Definition Classes
    Limit → SparkPlan

  17. def executeCollect(): Array[catalyst.expressions.Row]

    A custom implementation modeled after the take function on RDDs but which never runs any job locally.

    A custom implementation modeled after the take function on RDDs but which never runs any job locally. This is to avoid shipping an entire partition of data in order to retrieve only a few rows.

    Definition Classes
    Limit → SparkPlan

  18. def expressions: Seq[Expression]

    Definition Classes
    QueryPlan

  19. def fastEquals(other: TreeNode[_]): Boolean

    Definition Classes
    TreeNode

  20. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  21. def flatMap[A](f: (SparkPlan) ⇒ TraversableOnce[A]): Seq[A]

    Definition Classes
    TreeNode

  22. def foreach(f: (SparkPlan) ⇒ Unit): Unit

    Definition Classes
    TreeNode

  23. def generateSchemaString(schema: StructType, prefix: String, builder: StringBuilder): StringBuilder

    Attributes
    protected
    Definition Classes
    QueryPlan

  24. def generateSchemaString(schema: Seq[Attribute]): String

    Attributes
    protected
    Definition Classes
    QueryPlan

  25. def generateTreeString(depth: Int, builder: StringBuilder): StringBuilder

    Attributes
    protected
    Definition Classes
    TreeNode

  26. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  27. def getNodeNumbered(number: MutableInt): SparkPlan

    Attributes
    protected
    Definition Classes
    TreeNode

  28. val id: Long

    Definition Classes
    TreeNode

  29. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  30. val limit: Int

  31. lazy val logger: Logger

    Attributes
    protected
    Definition Classes
    Logging

  32. def makeCopy(newArgs: Array[AnyRef]): Limit.this.type

    Definition Classes
    TreeNode

  33. def map[A](f: (SparkPlan) ⇒ A): Seq[A]

    Definition Classes
    TreeNode

  34. def mapChildren(f: (SparkPlan) ⇒ SparkPlan): Limit.this.type

    Definition Classes
    TreeNode

  35. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  36. def nodeName: String

    Definition Classes
    TreeNode

  37. final def notify(): Unit

    Definition Classes
    AnyRef

  38. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  39. def numberedTreeString: String

    Definition Classes
    TreeNode

  40. def otherCopyArgs: List[SQLContext]

    Definition Classes
    Limit → TreeNode

  41. def output: Seq[Attribute]

    Definition Classes
    Limit → QueryPlan

  42. def outputPartitioning: Partitioning

    Specifies how data is partitioned across different nodes in the cluster.

    Specifies how data is partitioned across different nodes in the cluster.

    Definition Classes
    UnaryNode → SparkPlan

  43. def outputSet: Set[Attribute]

    Definition Classes
    QueryPlan

  44. def printSchema(): Unit

    Definition Classes
    QueryPlan

  45. def requiredChildDistribution: Seq[Distribution]

    Specifies any partition requirements on the input data for this operator.

    Specifies any partition requirements on the input data for this operator.

    Definition Classes
    SparkPlan

  46. def sameInstance(other: TreeNode[_]): Boolean

    Definition Classes
    TreeNode

  47. def schemaString: String

    Definition Classes
    QueryPlan

  48. def simpleString: String

    Definition Classes
    TreeNode

  49. def stringArgs: Iterator[Any]

    Attributes
    protected
    Definition Classes
    TreeNode

  50. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  51. def toString(): String

    Definition Classes
    TreeNode → AnyRef → Any

  52. def transform(rule: PartialFunction[SparkPlan, SparkPlan]): SparkPlan

    Definition Classes
    TreeNode

  53. def transformAllExpressions(rule: PartialFunction[Expression, Expression]): Limit.this.type

    Definition Classes
    QueryPlan

  54. def transformChildrenDown(rule: PartialFunction[SparkPlan, SparkPlan]): Limit.this.type

    Definition Classes
    TreeNode

  55. def transformChildrenUp(rule: PartialFunction[SparkPlan, SparkPlan]): Limit.this.type

    Definition Classes
    TreeNode

  56. def transformDown(rule: PartialFunction[SparkPlan, SparkPlan]): SparkPlan

    Definition Classes
    TreeNode

  57. def transformExpressions(rule: PartialFunction[Expression, Expression]): Limit.this.type

    Definition Classes
    QueryPlan

  58. def transformExpressionsDown(rule: PartialFunction[Expression, Expression]): Limit.this.type

    Definition Classes
    QueryPlan

  59. def transformExpressionsUp(rule: PartialFunction[Expression, Expression]): Limit.this.type

    Definition Classes
    QueryPlan

  60. def transformUp(rule: PartialFunction[SparkPlan, SparkPlan]): SparkPlan

    Definition Classes
    TreeNode

  61. def treeString: String

    Definition Classes
    TreeNode

  62. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  63. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  64. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  65. def withNewChildren(newChildren: Seq[SparkPlan]): Limit.this.type

    Definition Classes
    TreeNode

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from UnaryNode

Inherited from catalyst.trees.UnaryNode[SparkPlan]

Inherited from SparkPlan

Inherited from com.typesafe.scalalogging.slf4j.Logging

Inherited from QueryPlan[SparkPlan]

Inherited from TreeNode[SparkPlan]

Inherited from AnyRef

Inherited from Any

Ungrouped