This example application provides an API that let you send an Event to a (one or more) Spark Streaming Job(s) and show how you can manage a list of Jobs sending each type of event to the corresponding Spark Streaming Job.
Is out of discussion that is far better to do this with a kafka, but sometimes you want to test a solution before enter in a deeper development.
A very similar to this system, is in production and it's able to ingest thousands of events per second. Maybe is more than a simple Proof of Concept :)
This project is based in this other Project: https://github.com/NET-A-PORTER/spray-actor-per-request and try to be an example of a REST API for events that will be sent to Spark.
The second part (the Spark side of this project) will be uploaded in another example (working on it in my free time)
Actor Per Request pattern has many advantages over other patterns based on ASK If you want to know more about Actor Per Request:
- Net A Porter beautiful example: (link)
- Scala Exchange Presentation (video)
- Mathias describes the actor per request approach against others. (mailing list)