Differences between revisions 51 and 80 (spanning 29 versions)
Revision 51 as of 2020-02-01 13:44:11
Size: 5919
Comment:
Revision 80 as of 2020-02-08 21:01:37
Size: 5196
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
We will use corosync to create a virtual IP-address which is shared by two systems. One of the systems has the virtual IP-address. If the system goes down, the other system will take over the virtual IP-address. HA-proxy will be running on each of the hosts relaying incoming HTTP-requests to two webservers. We will use corosync to create a virtual IP-address which is shared by two systems. One of the systems has the virtual IP-address. If the system goes down, the other system will take over the virtual IP-address. HA-proxy will be running on each of the hosts relaying incoming requests.
Line 5: Line 5:
 * 192.168.1.49 www (virtual IP-address)
 * 192.168.1.50 haproxy01
* 192.168.1.51 haproxy02
 * 192.168.1.49 haproxy01
 * 192.168.1.50 haproxy02
 * 192.168.1.51
www (virtual IP-address)
 * 192.168.1.54 mariadb (virtual IP)
Line 16: Line 17:

=== HA-proxy ===

==== HA-proxy resource script ====
Download the HA-proxy resource script, place it in `/usr/lib/ocf/resource.d/heartbeat/haproxy` and make it executeable.
{{{
wget https://raw.githubusercontent.com/thisismitch/cluster-agents/master/haproxy -O /usr/lib/ocf/resource.d/heartbeat/haproxy
chmod +x /usr/lib/ocf/resource.d/heartbeat/haproxy
}}}

As we will let corosync start haproxy, we will have to disable haproxy as a service.
{{{
service haproxy stop
systemctl disable haproxy
}}}

==== Webfarm ====
Add the following to `/etc/haproxy/haproxy.cfg`
{{{
listen www_farm
        bind www:80
        mode http
        balance roundrobin
        cookie LBN insert indirect nocache
        option httpclose
        option forwardfor
        server www01 www01:80 cookie node1 check
        server www02 www02:80 cookie node2 check

listen galera_cluster
        bind mariadb:3306
        mode tcp
        balance roundrobin
        option tcpka
        option mysql-check user haproxy
        server mariadb01 mariadb01:3306 check weight 1
        server mariadb02 mariadb02:3306 check weight 1
        server mariadb03 mariadb03:3306 check weight 1
}}}
Copy the configuration `/etc/haproxy/haproxy.cfg` to the other host.
Line 30: Line 71:
Generate the authkey. Note that this requires entropy, so use your system and the entropy will be generated. Speed this up by running `dd if=/dev/vda of=/dev/null` in another terminal. Generate the authkey. Note that this requires entropy, so use your system and the entropy will be generated. Speed this up by running `dd if=/dev/vda of=/dev/null` in another terminal or install `haveged`.
Line 50: Line 91:

---- /!\ '''Edit conflict - other version:''' ----

---- /!\ '''Edit conflict - your version:''' ----
Line 69: Line 106:
---- /!\ '''End of edit conflict''' ----
=== HA-proxy ===

==== HA-proxy resource script ====
Download the HA-proxy resource script, place it in `/usr/lib/ocf/resource.d/heartbeat/haproxy` and make it executeable.
{{{
wget https://raw.githubusercontent.com/thisismitch/cluster-agents/master/haproxy -O /usr/lib/ocf/resource.d/heartbeat/haproxy
chmod +x /usr/lib/ocf/resource.d/heartbeat/haproxy
}}}
This has to be done on both hosts.

==== Webfarm ====

Add the following to `/etc/haproxy/haproxy.cfg`
{{{
listen webfarm
        bind www:80
        mode http
        balance roundrobin
        cookie LBN insert indirect nocache
        option httpclose
        option forwardfor
        server haproxy01 www01:80 cookie node1 check
        server haproxy02 www02:80 cookie node2 check
}}}
Copy the configuration `/etc/haproxy/haproxy.cfg` to the other host.

Restart HA-proxy on both hosts to load the new configuration.
{{{
service haproxy restart
}}}

The two nodes should now see each other. Run `crm status` to check both nodes are online.
{{{
Stack: corosync
Current DC: haproxy01 (version 1.1.16-94ff4df) - partition with quorum
Last updated: Sat Feb 1 14:41:09 2020
Last change: Sat Feb 1 14:40:51 2020 by hacluster via crmd on haproxy01

2 nodes configured
0 resources configured

Online: [ haproxy01 haproxy02 ]

No resources
}}}


---- /!\ '''Edit conflict - other version:''' ----
The two nodes should now see each other. Run `crm status` to check both nodes are online.
{{{
Stack: corosync
Current DC: haproxy01 (version 1.1.16-94ff4df) - partition with quorum
Last updated: Sat Feb 1 14:41:09 2020
Last change: Sat Feb 1 14:40:51 2020 by hacluster via crmd on haproxy01

2 nodes configured
0 resources configured

Online: [ haproxy01 haproxy02 ]

No resources
}}}


---- /!\ '''Edit conflict - your version:''' ----

---- /!\ '''End of edit conflict''' ----
Line 138: Line 107:
Do not kill the other node. Do not kill the other node. This is needed as inactive nodes normally should be killed. It can be change later on, once fencing has been configured.
Line 146: Line 115:
Configure shared IP-address. Configure shared IP-address for apache. This is the virtual IP-address shared by apache servers.
Line 148: Line 117:
crm configure primitive haproxy_wwwSharedIP ocf:heartbeat:IPaddr2 params ip=192.168.1.49 cidr_netmask=24 op monitor interval=5s crm configure primitive wwwSharedIP ocf:heartbeat:IPaddr2 params ip=192.168.1.51 cidr_netmask=24 op monitor interval=5s
Line 152: Line 121:
crm configure primitive haproxy_wwwLoadBalance ocf:heartbeat:haproxy params conffile=/etc/haproxy/haproxy.cfg op monitor interval=10s crm configure primitive wwwLoadBalance ocf:heartbeat:haproxy params conffile=/etc/haproxy/haproxy.cfg op monitor interval=10s
Line 156: Line 125:
crm configure group haproxy_www haproxy_wwwSharedIP haproxy_wwwLoadBalance
}}}
Create the relation between IP-address and haproxy servers.

currently gives:
root@haproxy01:/etc/corosync# crm configure colocation haproxyWithIPs INFINITY: haproxy_wwwLoadBalance haproxy_wwwSharedIP

WARNING: haproxyWithIPs: resource haproxy_wwwLoadBalance is grouped, constraints should apply to the group

WARNING: haproxyWithIPs: resource haproxy_wwwSharedIP is grouped, constraints should apply to the group)

{{{
crm configure colocation haproxyWithIPs INFINITY: haproxy_wwwLoadBalance haproxy_wwwSharedIP
crm configure group www wwwSharedIP wwwLoadBalance
Line 172: Line 129:
crm configure order haproxyAfterIPs mandatory: haproxy_wwwSharedIP haproxy_wwwLoadBalance crm configure order haproxyAfterIP mandatory: wwwSharedIP wwwLoadBalance
Line 175: Line 132:
==== Start over ====
run this `rm /var/lib/heartbeat/crm/cib*`
Line 179: Line 133:
=== Apache ===
Line 181: Line 136:
}}}
=== Mariadb ===
{{{
while true; do mysql -u haproxy -p -h mariadb -e "select @@hostname"; sleep 1; done
Line 185: Line 144:
 * https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-cluster-options.html
Line 186: Line 146:
 * https://www.howtoforge.com/tutorial/how-to-setup-haproxy-as-load-balancer-for-mariadb-on-centos-7/#b-install-mariadb-galera-server
 * https://www.digitalocean.com/community/tutorials/how-to-use-haproxy-to-set-up-mysql-load-balancing--3

Corosync and HA-proxy

We will use corosync to create a virtual IP-address which is shared by two systems. One of the systems has the virtual IP-address. If the system goes down, the other system will take over the virtual IP-address. HA-proxy will be running on each of the hosts relaying incoming requests.

  • 192.168.1.49 haproxy01
  • 192.168.1.50 haproxy02
  • 192.168.1.51 www (virtual IP-address)
  • 192.168.1.54 mariadb (virtual IP)

Software

apt-get install corosync haproxy crmsh

Configuration

Before binding to the DNS name www make sure the DNS server knows that name. There is no physical or virtual network interfaces, so the DNS updates has to be done manually.

HA-proxy

HA-proxy resource script

Download the HA-proxy resource script, place it in /usr/lib/ocf/resource.d/heartbeat/haproxy and make it executeable.

wget https://raw.githubusercontent.com/thisismitch/cluster-agents/master/haproxy -O /usr/lib/ocf/resource.d/heartbeat/haproxy
chmod +x /usr/lib/ocf/resource.d/heartbeat/haproxy

As we will let corosync start haproxy, we will have to disable haproxy as a service.

service haproxy stop
systemctl disable haproxy

Webfarm

Add the following to /etc/haproxy/haproxy.cfg

listen www_farm 
        bind www:80
        mode http
        balance roundrobin
        cookie LBN insert indirect nocache
        option httpclose
        option forwardfor
        server www01 www01:80 cookie node1 check
        server www02 www02:80 cookie node2 check

listen galera_cluster
        bind mariadb:3306
        mode tcp
        balance roundrobin
        option tcpka
        option mysql-check user haproxy
        server mariadb01 mariadb01:3306 check weight 1 
        server mariadb02 mariadb02:3306 check weight 1
        server mariadb03 mariadb03:3306 check weight 1

Copy the configuration /etc/haproxy/haproxy.cfg to the other host.

Corosync

Encryption

Edit /etc/corosync/corosync.conf and make the following changes.

totem {
        cluster_name: www
        crypto_cipher: aes256
        crypto_hash: sha512
}

Authentication Key

Generate the authkey. Note that this requires entropy, so use your system and the entropy will be generated. Speed this up by running dd if=/dev/vda of=/dev/null in another terminal or install haveged.

corosync-keygen

Network

Add the local network and a multicast address to /etc/corosync/corosync.conf in the interface section.

interface {
        bindnetaddr: 192.168.1.0 
        mcastaddr: 239.192.1.1
}

Copy the configuration /etc/corosync/corosync.conf and the authkey /etc/corosync/authkey to the other host.

Restart corosync on both hosts to load the new configuration.

service corosync restart

The two nodes should now be configured to see each other. Run crm status to check.

Stack: corosync
Current DC: haproxy01 (version 1.1.16-94ff4df) - partition with quorum
Last updated: Sat Feb  1 14:41:09 2020
Last change: Sat Feb  1 14:40:51 2020 by hacluster via crmd on haproxy01

2 nodes configured
0 resources configured

Online: [ haproxy01 haproxy02 ]

No resources

Nodes

Do not kill the other node. This is needed as inactive nodes normally should be killed. It can be change later on, once fencing has been configured.

crm configure property stonith-enabled=false

Disable quorum due to two nodes only. This will make sure that the cluster keeps running if only one node is up.

crm configure property no-quorum-policy=ignore

Configure shared IP-address for apache. This is the virtual IP-address shared by apache servers.

crm configure primitive wwwSharedIP ocf:heartbeat:IPaddr2 params ip=192.168.1.51 cidr_netmask=24 op monitor interval=5s

Create heartbeat for haproxy.

crm configure primitive wwwLoadBalance ocf:heartbeat:haproxy params conffile=/etc/haproxy/haproxy.cfg op monitor interval=10s

Make sure the same server has both IP and service at the same time.

crm configure group www wwwSharedIP wwwLoadBalance

IP-address should be up before haproxy starts.

crm configure order haproxyAfterIP mandatory: wwwSharedIP wwwLoadBalance

Test it

Apache

while true; do wget --quiet http://www/ -O /dev/stdout; sleep 1; done

Mariadb

while true; do mysql -u haproxy -p -h mariadb -e "select @@hostname"; sleep 1; done

References

None: Corosync and HA-proxy (last edited 2021-08-12 09:31:08 by Kristian Kallenberg)