Keepalived virtual IP won't be transferred back when host master is up again

Question

I am installing keepalived for virtual IP failover using 2 vagrant Centos7 instances but having a problem (I have same problem on production).

Basically, I have 2 instances running keepalived. Below is the configuration keepalived.conf for those 2 servers:

Server1's config:

global_defs {
   notification_email {
      [email protected]
   }
   notification_email_from server1.dev.vagrant.net
   smtp_server localhost
   smtp_connect_timeout 30
}

  vrrp_instance VI_enp0s8 {
      state MASTER
      interface enp0s8
      virtual_router_id 51
      priority 100
      advert_int 1
      authentication {
          auth_type PASS
          auth_pass 1111
      }
      virtual_ipaddress {
                    192.168.65.110
                    192.168.65.111

      }
  }

Server2's config:

global_defs {
   notification_email {
      [email protected]
   }
   notification_email_from server2.dev.vagrant.net
   smtp_server localhost
   smtp_connect_timeout 30
}

  vrrp_instance VI_enp0s8 {
      state BACKUP
      interface enp0s8
      virtual_router_id 51
      priority 99
      advert_int 1
      authentication {
          auth_type PASS
          auth_pass 1111
      }
      virtual_ipaddress {
                    192.168.65.110
                    192.168.65.111

      }
  }

Basically, if I stop keepalived using: sudo service keepalived stop on server1; then the virtual ipaddresses is transferred into server2 successfully. And when I start keepalived back on server1 using "sudo service keepalived start" then the ip addresses is transfered back. Everything is also working correctly when I am doing "sudo reboot" on server1.

However, if I shutdown and then restart server1 using: sudo shutdown -h now (or vagrant halt server1) and then vagrant up server1, then the virtual IP will not be assigned to either one server. (ip addr show enp0s8).

Below is the log of /var/log/messages. It seems to me that for some reason, the virtual IPs have been added, but then removed.

Jun 21 08:19:50 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) Entering MASTER STATE
Jun 21 08:19:50 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) setting protocol VIPs.
Jun 21 08:19:50 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.110 added
Jun 21 08:19:50 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.111 added
Jun 21 08:19:50 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) Sending gratuitous ARPs on enp0s8 for 192.168.65.110
Jun 21 08:19:50 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) Sending gratuitous ARPs on enp0s8 for 192.168.65.111
Jun 21 08:19:52 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.104 removed
Jun 21 08:19:52 oasis4 Keepalived_vrrp[724]: Netlink reflector reports IP 192.168.65.104 removed
Jun 21 08:19:52 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.110 removed
Jun 21 08:19:52 oasis4 Keepalived_vrrp[724]: Netlink reflector reports IP 192.168.65.110 removed
Jun 21 08:19:52 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.111 removed
Jun 21 08:19:52 oasis4 Keepalived_vrrp[724]: Netlink reflector reports IP 192.168.65.111 removed
Jun 21 08:19:52 oasis4 kernel: e1000: enp0s8 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
Jun 21 08:19:52 oasis4 kernel: IPv6: ADDRCONF(NETDEV_UP): enp0s8: link is not ready
Jun 21 08:19:52 oasis4 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp0s8: link becomes ready
Jun 21 08:19:54 oasis4 Keepalived_healthcheckers[723]: Netlink reflector reports IP 192.168.65.104 added
Jun 21 08:19:54 oasis4 Keepalived_vrrp[724]: Netlink reflector reports IP 192.168.65.104 added
Jun 21 08:19:55 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) Sending gratuitous ARPs on enp0s8 for 192.168.65.110
Jun 21 08:19:55 oasis4 Keepalived_vrrp[724]: VRRP_Instance(VI_enp0s8) Sending gratuitous ARPs on enp0s8 for 192.168.65.111
Jun 21 08:19:55 oasis4 systemd: Starting Session 2 of user vagrant.
Jun 21 08:19:55 oasis4 systemd: Started Session 2 of user vagrant.
Jun 21 08:19:55 oasis4 systemd-logind: New session 2 of user vagrant.
Jun 21 08:19:56 oasis4 systemd-logind: Removed session 1.
Jun 21 08:20:00 oasis4 systemd: Starting Session 3 of user vagrant.
Jun 21 08:20:00 oasis4 systemd: Started Session 3 of user vagrant.
Jun 21 08:20:00 oasis4 systemd-logind: New session 3 of user vagrant.
Jun 21 08:20:01 oasis4 systemd: Starting user-0.slice.
Jun 21 08:20:01 oasis4 systemd: Created slice user-0.slice.
Jun 21 08:20:01 oasis4 systemd: Starting Session 4 of user root.

If I log in into server1 and then restart keepalived manually then thing starts to work normally.

Could any one please let me know if there is anything wrong with my configuration?

Also, there is a couple of time I cannot stop the service properly using sudo service keepalived stop; When it happens, virtual IP is not assigned to either servers.

Thanks.

Answer 1

According to the logs, keepalived has started before the network interface enp0s8 is up and ready. If you want keepalived to track the interface, you should add something like this in the vrrp_instance

track_interface {
    enp0s8
}