Spark Streaming + Kafka Integration Guide

Apache Kafka is publish-subscribe messaging rethought as a distributed, partitioned, replicated commit log service. Please read the Kafka documentation thoroughly before starting an integration using Spark.

At the moment, Spark requires Kafka 0.10 and higher. See Kafka 0.10 integration documentation for details.