Advanced HAProxy configurations for Multi-Master, Active-Standby (Hadoop, HBase, Presto) and Peer-to-Peer technologies (Elasticsearch, SolrCloud etc).
Designed both for production-grade High Availability and also to make scripting and monitoring easier when connecting to APIs.
Forked as a submodule from the Advanced Nagios Plugins Collection, these configurations contain specialised Health Checks for each system based on experience and code from the Advanced Nagios Plugins Collection and DevOps Python Tools github repos.
-Recommended to run with High Availability HAProxy using VRRP to create full production-grade High Availability load balancer solutions.*
Hari Sekhon
Cloud & Big Data Contractor, United Kingdom
(ex-Cloudera, former Hortonworks Consultant)
(you're welcome to connect with me on LinkedIn)
Each config comes pre-tuned with:
- Specialised Health Checks per technology's API
- Load Balancing algorithm selected best suited per technology (depending on Active/Passive, Peer-to-Peer etc)
- Various buffer sizes, keepalives, connection limits and server retries already tuned (many via shared global config
10-global.cfg) - ACL protections - limits access to private IP addresses - safer by default but easy to disable if you need to serve the internet
- Additional logging
- Optional Stats Admin UI (excellent for admin, monitoring and debugging - password protected, change the default password from 'test' in
20-stats.cfgif you source that config)
You should use an expert consultant to fine tune to your needs but these should be extremely close to your finished production configurations. In most cases all you need to do is put in your addresses for the backend servers.
Configurations are split by service in the form of <service>.cfg for mix-and-match convenience and must be combined with 10-global.cfg settings like so:
haproxy -f 10-global.cfg -f elasticsearch.cfgFor multiple services just add those service configurations to the command line options:
haproxy -f 10-global.cfg -f elasticsearch.cfg -f solrcloud.cfgSome technologies default to the same port number (eg. Ambari and Presto both use port 8080), so if running both on the same HAProxy host then just modify one of the frontend listening port numbers.
If you want to add a Stats Admin UI then include the 20-stats.cfg configuration (remember to change the default password test):
haproxy -f 10-global.cfg -f 20-stats.cfg -f elasticsearch.cfg ...To shorten the command line you can simply use the run.sh script instead:
./run.sh elasticsearch.cfgand easily with multiple configs
./run.sh elasticsearch.cfg solrcloud.cfg ...In most cases all you'll need to do is add your server addresses in to the backend config and then start HAProxy.
Common backend server addresses have been pre-populated for convenience including:
<service>- generic service name matching the proxied technology - could be resolved by DNS to multiple IPs to be balanced across192.168.99.100- the common Docker Machine / MiniKube / MiniShift IP addressdocker- convenient to use DNS or/etc/hoststo point this to your Docker host
These addresses are used in Continuous Integration tests run on this repo from the Advanced Nagios Plugins Collection.
See the untested/ directory for a few more including SSL config versions I haven't got round to testing yet but should work.
Forked as a submodule from the Advanced Nagios Plugins Collection. These HAProxy configs are tested against Docker containers as part of CI tests in that repo.
The rest of my original source repos are here.
Pre-built Docker images are available on my DockerHub.