This repository is companion code for a blog post on our site: ADD LINK HERE.
This project is an all in one environment that sets up Vagrant machines with Couchbase and Apache NiFi installed. It has a NiFi data flow template that will retrieve GHCN weather data from NOAA and process it into Json documents for Couchbase.
-
Install Virtualbox: https://www.virtualbox.org/wiki/Downloads
-
Install Vagrant: http://www.vagrantup.com/downloads.html
-
Install necessary Vagrant plugins:
vagrant plugin install vagrant-hostmanager
vagrant plugin install vagrant-cachier- Install Ansible
brew install ansible-
Start by bringing up the Vagrant machines, it is configured to install everything you need to run the analysis, this may take a few minutes the first time.
vagrant up
-
Go to the NiFi UI at http://nifi.vagrant:8080/nifi. Import the template by clicking on the Templates icon in the top right. In the Templates menu, click Browse then select the NOAA_to_Couchbase.xml file from this project. Finally, click Import.
-
Drag a Template from the toolbar on the top left onto the canvas. Select the template you just imported. A whole template should be added to the canvas.
-
Click the Controller Settings icon in the top right, go to the Controller Services tab, then click the lightning bolt next to the CouchbaseClusterService. When prompted click Enable.
-
Finally, click the Start button at the top of the screen to start the dataflow.
-
At this point you should start to see data flowing through the system. When you see documents of the In line in the PutCouchbaseKey processor you can check Couchbase to see documents in the weather bucket.
You can access the Couchbase UI at http://couchbase.vagrant:8091 with credentials: couchbase//couchbase and the NiFi UI at http://nifi.vagrant:8080/nifi.