This tutorial is about creating a highly available HTTP load balancer using HAProxy. The setup can be slightly complicated but you will appreciate the result – you will get a load-balanced + highly available web service in your network. Basic linux skill is assumed.
Scenario:
Imagine we have 2 physical machines. In each machine, I have 2 virtual machines. All 4 virtual machines will be in the same subnet, ie 10.1.1.0/24 in this case.
Steps:
1. create 4 vm, centos1.dev (10.1.1.111), centos2.dev (10.1.1.112), centos3.dev (10.1.1.113) and centos4.dev (10.1.1.114). These 4 vm should have the bare min. packages installed.
unless using DNS, add this to /etc/hosts on all virtual machines
10.1.1.111 centos1.dev
10.1.1.112 centos2.dev
10.1.1.113 centos3.dev
10.1.1.114 centos4.dev
2. leave firewall and selinux on. allow port 80 for all 4 vm.
3. centos1.dev and centos2.dev will be the load balancer and centos3.dev and centos4.dev will be the 2 http servers. In centos3 and centos4,
yum groupinstall "web server"
chkconfig httpd on
4. Then in centos3.dev and centos4.dev again, edit /etc/httpd/conf/httpd.conf, in order to capture the real IP of the user, replace %h to %{X-Forwarded-For}i. We also add a virtual host.
#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
...
...
NameVirtualHost *:80
<VirtualHost *:80>
DocumentRoot /var/www/html
ServerName *
SetEnvIf Request_URI "^/haproxy\.txt$" dontlog
CustomLog /var/log/httpd/access.log combined env=!dontlog
</VirtualHost>
This virtual config is just for the sake of testing, you need to change it in the real environment.
5. In both centos3 and centos 4 again,
cd /var/www/html,
echo "centos3" > index.html
(in centos4, echo “centos4″ > index.html)
then create haproxy.txt in the same dir for both http servers. Without the check file, haproxy will fail.
touch haproxy.txt
Restart apache (httpd) in both servers
Installing HAProxy:
1. ssh into centos1.dev and centos2.dev and install Haproxy. Someone has compiled the rpm for us. Download it from the rpmbone website and install it.
http://rpm.pbone.net/index.php3/stat/4/idpl/13437166/com/haproxy-1.3.22-1.el5.x86_64.rpm.html
After installing it,
chkconfig haproxy on
2. edit /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
maxconn 4096
user haproxy
group haproxy
defaults
log global
mode http
option httplog
option dontlognull
option redispatch
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen ha-http 10.1.1.110:80
mode http
stats enable
stats auth user:password
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httpchk HEAD /haproxy.txt HTTP/1.0
server apache1 centos3.dev:80 cookie A check
server apache2 centos4.dev:80 cookie B check
3. To allow HAProxy to bind to the shared IP address, we add the following line to /etc/sysctl.conf:
net.ipv4.ip_nonlocal_bind=1
then reload sysctl config,
sysctl -p
Installing Heartbeat:
1. Heartbeat is necessary for any highly available systems. In both centos1 and centos2, to install heartbeat
yum install heartbeat
2. after that in centos1.dev, cd /etc/ha.d, edit /etc/ha.d/authkeys
auth 2
2 sha1 loadbalancing-ha
3. edit /etc/ha.d/ha.cf
keepalive 2
deadtime 10
udpport 694
bcast eth0
mcast eth0 225.0.0.1 694 1 0
ucast eth0 centos2.dev
udp eth0
logfacility local0
node centos1.dev
node centos2.dev
node needs to be the machine name, ie type “hostname” in command line to see. Now we want centos1 to be highly available, so edit /etc/ha.d/haresources:
centos1.dev 10.1.1.110
4. if firewall is turned on, remember to allow 694:udp (do it for both centos1.dev and centos2.dev)
5. after setting everything in centos1, copy the files over to centos2, ie
scp {authkeys,haresource,ha.cf} 10.1.1.112:/etc/ha.d
6. now in centos2, edit ha.cf
keepalive 2
deadtime 10
udpport 694
bcast eth0
mcast eth0 225.0.0.1 694 1 0
ucast eth0 centos1.dev
udp eth0
logfacility local0
node centos1.dev
node centos2.dev
Noticed the difference in ucast
7. Now we want to start heartbeat in both machines upon reboot
echo "service heartbeat start" >> /etc/rc.local
Testing
The ip 10.1.1.110:80 is now load balanced and highly available. To test it, shutdown 10.1.1.111 and the load balancer will still function. If 10.1.1.113 HTTP is down, 10.1.1.114 will take over and vice version.
Viewing Haproxy Stats
1. The options “stats enable” and “stats auth” in the HAProxy configuration allow the admin to view the stats, just go to http://10.1.1.110/haproxy?stats and type in username as user and password as password
Conclusion
I hope you follow me so far and appreciate what HAproxy can offer. I certainly enjoy blogging about it and I hope you it useful.
