This project was developed as part of UE20CS343 - Database Technologies to build a real-time data streaming pipeline using Apache Kafka and Spark Structured Streaming. It simulates ingesting San Francisco crime data into Kafka, processing it with Spark, and performing aggregations and stream-table joins.
A sandbox environment designed to simulate a pseudo-distributed Hadoop cluster with integrated Apache Spark and Kafka components. It allows developers to prototype and experiment with big data workflows, test distributed computing patterns, and explore cluster behavior in a contained virtual setup.
Add a description, image, and links to the spark-structured-streaming-kafka topic page so that developers can more easily learn about it.
To associate your repository with the spark-structured-streaming-kafka topic, visit your repo's landing page and select "manage topics."