Logical Execution Plan
-
RDD Lineage and Logical Execution PlanDistributedSystem/Spark 2019. 9. 25. 05:37
1. Overview RDD Lineage (aka RDD operator graph or RDD dependency graph) is a graph of all the parent RDDs of an RDD. It is built as a result of applying transformations to the RDD and creates a logical execution plan. Logical Execution Plan starts with the earliest RDDs (those with no dependencies on other RDDs or reference cached data) and ends with the RDD that produces the result of the acti..