[Linux-HA] Active/Active configuration doesn't work (anymore?)

[Luca Corti]

Hello,

This evening I found myself doing package updates on an heartbeat
two-node cluster using an active/active configuration. It has alway
worked like this, with one host doing NFS and the other one doing MySQL.

This evening I upgraded heartbet. I'm on Debian Sarge, and the new
heartbeat package just fixes CAN-2005-2231.

When upgrading the package installation scripts restart the services.
Once finished the upgrade both NFS and MySQL are running on one of the
two nodes, and if I run hb_takeover failback I get:

Aug 15 21:58:34 iena heartbeat: info: Running /etc/ha.d/rc.d/hb_takeover
hb_takeover
Aug 15 21:58:34 iena heartbeat[24933]: info: coguaro wants to go standby
[all]


Here is my configuration.

ha.cf

logfacility daemon
node iena coguaro
keepalive 1
deadtime 10
mcast eth0 224.1.2.3 694 1 0
ping 10.130.33.253
serial /dev/ttyS1
auto_failback yes
respawn hacluster /usr/lib/heartbeat/ipfail

haresources

iena    MailTo::root IPaddr::10.130.33.33
Filesystem::/dev/sda1::/storage::ext3 statd nfsd
coguaro MailTo::root IPaddr::10.130.33.32
Filesystem::/dev/sdb1::/dbstorage::ext3 mysqld

and here is the log of a takeover

Aug 15 21:59:23 iena heartbeat[24933]: info: standby: acquire [all]
resources from coguaro
Aug 15 21:59:23 iena heartbeat[27788]: info: acquire all HA resources
(standby).
Aug 15 21:59:23 iena heartbeat: info: Acquiring resource group: iena
MailTo::root IPaddr::10.130.33.33 Filesystem::/dev/sda1::/storage::ext3
statd nfsd coguaro MailTo::root IPaddr::10.130.33.32
Filesystem::/dev/sdb1::/dbstorage::ext3 mysqld slapd
Aug 15 21:59:23 iena heartbeat: info:
Running /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:23 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:23 iena heartbeat: debug: /etc/ha.d/resource.d/MailTo root
start done. RC=0
Aug 15 21:59:23 iena heartbeat: info:
Running /etc/ha.d/resource.d/IPaddr 10.130.33.33 start
Aug 15 21:59:23 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/IPaddr 10.130.33.33 start
Aug 15 21:59:23 iena heartbeat: info: /sbin/ifconfig eth1:0 10.130.33.33
netmask 255.255.255.0^Ibroadcast 10.130.33.255
Aug 15 21:59:23 iena heartbeat: info: Sending Gratuitous Arp for
10.130.33.33 on eth1:0 [eth1]
Aug 15 21:59:23 iena heartbeat: /usr/lib/heartbeat/send_arp -i 1010 -r 5
-p /var/lib/heartbeat/rsctmp/send_arp/send_arp-10.130.33.33 eth1
10.130.33.33 auto 10.130.33.33 ffffffffffff
Aug 15 21:59:23 iena heartbeat: debug: /etc/ha.d/resource.d/IPaddr
10.130.33.33 start done. RC=0
Aug 15 21:59:24 iena heartbeat: info:
Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /storage ext3 start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/Filesystem /dev/sda1 /storage ext3 start
Aug 15 21:59:24 iena heartbeat:
debug: /etc/ha.d/resource.d/Filesystem /dev/sda1 /storage ext3 start
done. RC=0
Aug 15 21:59:24 iena heartbeat: info: Running /etc/ha.d/resource.d/statd
start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/statd  start
Aug 15 21:59:24 iena heartbeat: debug: /etc/ha.d/resource.d/statd  start
done. RC=0
Aug 15 21:59:24 iena heartbeat: info: Running /etc/ha.d/resource.d/nfsd
start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/nfsd  start
Aug 15 21:59:24 iena heartbeat: debug: /etc/ha.d/resource.d/nfsd  start
done. RC=0
Aug 15 21:59:24 iena heartbeat: ERROR: Cannot locate resource script
coguaro
Aug 15 21:59:24 iena heartbeat: info:
Running /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:24 iena heartbeat: debug: /etc/ha.d/resource.d/MailTo root
start done. RC=0
Aug 15 21:59:24 iena heartbeat: info:
Running /etc/ha.d/resource.d/IPaddr 10.130.33.32 start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/IPaddr 10.130.33.32 start
Aug 15 21:59:24 iena heartbeat: info: /sbin/ifconfig eth1:1 10.130.33.32
netmask 255.255.255.0^Ibroadcast 10.130.33.255
Aug 15 21:59:24 iena heartbeat: info: Sending Gratuitous Arp for
10.130.33.32 on eth1:1 [eth1]
Aug 15 21:59:24 iena heartbeat: /usr/lib/heartbeat/send_arp -i 1010 -r 5
-p /var/lib/heartbeat/rsctmp/send_arp/send_arp-10.130.33.32 eth1
10.130.33.32 auto 10.130.33.32 ffffffffffff
Aug 15 21:59:24 iena heartbeat: debug: /etc/ha.d/resource.d/IPaddr
10.130.33.32 start done. RC=0
Aug 15 21:59:24 iena heartbeat: info:
Running /etc/ha.d/resource.d/Filesystem /dev/sdb1 /dbstorage ext3 start
Aug 15 21:59:24 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/Filesystem /dev/sdb1 /dbstorage ext3 start
Aug 15 21:59:24 iena heartbeat:
debug: /etc/ha.d/resource.d/Filesystem /dev/sdb1 /dbstorage ext3 start
done. RC=0
Aug 15 21:59:25 iena heartbeat: info:
Running /etc/ha.d/resource.d/mysqld  start
Aug 15 21:59:25 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/mysqld  start
Aug 15 21:59:26 iena heartbeat: debug: /etc/ha.d/resource.d/mysqld
start done. RC=0
Aug 15 21:59:26 iena heartbeat: info: Running /etc/ha.d/resource.d/slapd
start
Aug 15 21:59:26 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/slapd  start
Aug 15 21:59:26 iena heartbeat: debug: /etc/ha.d/resource.d/slapd  start
done. RC=0
Aug 15 21:59:26 iena heartbeat[27788]:
ERROR: /usr/lib/heartbeat/ResourceManager takegroup MailTo::root
returned 256
Aug 15 21:59:26 iena heartbeat: info: Acquiring resource group: iena
MailTo::root IPaddr::10.130.33.33 Filesystem::/dev/sda1::/storage::ext3
statd nfsd coguaro MailTo::root IPaddr::10.130.33.32
Filesystem::/dev/sdb1::/dbstorage::ext3 mysqld slapd
Aug 15 21:59:26 iena heartbeat: info:
Running /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:26 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:26 iena heartbeat: debug: /etc/ha.d/resource.d/MailTo root
start done. RC=0
Aug 15 21:59:26 iena heartbeat: info: Running /etc/ha.d/resource.d/statd
start
Aug 15 21:59:26 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/statd  start
Aug 15 21:59:26 iena heartbeat: debug: /etc/ha.d/resource.d/statd  start
done. RC=1
Aug 15 21:59:26 iena heartbeat: ERROR: Return code 1
from /etc/ha.d/resource.d/statd
Aug 15 21:59:26 iena heartbeat: info: Running /etc/ha.d/resource.d/nfsd
start
Aug 15 21:59:26 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/nfsd  start
Aug 15 21:59:26 iena heartbeat: debug: /etc/ha.d/resource.d/nfsd  start
done. RC=1
Aug 15 21:59:26 iena heartbeat: ERROR: Return code 1
from /etc/ha.d/resource.d/nfsd
Aug 15 21:59:26 iena heartbeat: ERROR: Cannot locate resource script
coguaro
Aug 15 21:59:26 iena heartbeat: info:
Running /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:26 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/MailTo root start
Aug 15 21:59:27 iena heartbeat: debug: /etc/ha.d/resource.d/MailTo root
start done. RC=0
Aug 15 21:59:27 iena heartbeat: info:
Running /etc/ha.d/resource.d/mysqld  start
Aug 15 21:59:27 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/mysqld  start
Aug 15 21:59:27 iena heartbeat: debug: /etc/ha.d/resource.d/mysqld
start done. RC=0
Aug 15 21:59:27 iena heartbeat: info: Running /etc/ha.d/resource.d/slapd
start
Aug 15 21:59:27 iena heartbeat: debug:
Starting /etc/ha.d/resource.d/slapd  start
Aug 15 21:59:27 iena heartbeat: debug: /etc/ha.d/resource.d/slapd  start
done. RC=0
Aug 15 21:59:27 iena heartbeat[27788]:
ERROR: /usr/lib/heartbeat/ResourceManager takegroup MailTo::root
returned 256
Aug 15 21:59:27 iena heartbeat[27788]: info: all HA resource acquisition
completed (standby).
Aug 15 21:59:27 iena heartbeat[24933]: info: Standby resource
acquisition done [all].
Aug 15 21:59:27 iena heartbeat[24933]: info: remote resource transition
completed.


thanks


-- 
Luca Corti
PGP Key ID 1F38C091
-- 
Luca Corti
PGP Key ID 1F38C091
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.community.tummy.com/pipermail/linux-ha/attachments/20050815/87891950/attachment.pgp