pyspark.sql.DataFrame.unionAll

DataFrame.unionAll(other: pyspark.sql.dataframe.DataFrame) → pyspark.sql.dataframe.DataFrame

Return a new DataFrame containing union of rows in this and another DataFrame.

New in version 1.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

otherDataFrame

Another DataFrame that needs to be combined

Returns

DataFrame

Combined DataFrame

See also

DataFrame.union

Notes

This is equivalent to UNION ALL in SQL. To do a SQL-style set union (that does deduplication of elements), use this function followed by distinct().

Also as standard in SQL, this function resolves columns by position (not by name).

unionAll() is an alias to union()

pyspark.sql.DataFrame.union pyspark.sql.DataFrame.unionByName