pyspark.resource.ResourceProfileBuilder¶

class pyspark.resource.ResourceProfileBuilder[source]¶

Resource profile Builder to build a resource profile to associate with an RDD. A ResourceProfile allows the user to specify executor and task requirements for an RDD that will get applied during a stage. This allows the user to change the resource requirements between stages.

New in version 3.1.0.

Notes

This API is evolving.

Methods

`clearExecutorResourceRequests`()
`clearTaskResourceRequests`()
`require`(resourceRequest)

Attributes

`build`
`executorResources`
`taskResources`

pyspark.resource.ResourceProfile pyspark.resource.ExecutorResourceRequest