pyspark.streaming.StreamingContext.getActiveOrCreate¶
-
classmethod
StreamingContext.getActiveOrCreate(checkpointPath, setupFunc)[source]¶ Either return the active StreamingContext (i.e. currently started but not stopped), or recreate a StreamingContext from checkpoint data or create a new StreamingContext using the provided setupFunc function. If the checkpointPath is None or does not contain valid checkpoint data, then setupFunc will be called to create a new context and setup DStreams.
- Parameters
- checkpointPathstr
Checkpoint directory used in an earlier streaming program. Can be None if the intention is to always create a new context when there is no active context.
- setupFuncfunction
Function to create a new JavaStreamingContext and setup DStreams