Overview of Manual Failover Steps When Group Leader Is Unreachable

When the backup group leader cannot reach the group leader, the following happens:
  • There is no access to the NimbleOS GUI.
  • Group leader services are unavailable until a group leader takeover has been performed.
  • The backup group leader can only be accessed from the CLI by using the secondary management IP.

Before you begin the manual takeover, you must make sure there is a common snapshot between the two arrays in the group. If there is not a common snapshot, you will not be able to re-enable synchronous replication.

To perform a manual failover when the group leader is not available, you must complete the following high-level steps. Go to each procedure for a list of the detailed steps it requires.

  1. Confirm that the group leader is unavailable.
    IMPORTANT: If you can reach the group leader using the group --check_takeover command, do not perform a manual takeover.
  2. If you have a clustered configuration, take down the cluster.
  3. From the backup group leader, use CLI commands to transition it to the role of group leader and start the group leader services.

    When the backup group leader becomes the new group leader, the previous group leader will transition to an out-of-sync state. At that time connections to it will be available.

    See Perform a Manual Takeover of the Group Leader Array.

  4. Remove synchronous replication from the downstream partner to allow access to the volume. The following will happen:
    1. Existing ACLs from the upstream volume are cloned to the downstream volume.
    2. The LUNs on the downstream volume get new numbers.

    See Remove Synchronous Replication from the Downstream Partner.

  5. If you are using a clustered configuration, bring the cluster back up.
  6. Set up synchronous replication on the new group leader (the former backup group leader).

    See Add Synchronous Replication

    NOTE: It is a good practice to set up ASO and the Witness.
  7. (Optional) Perform a handover to restore the original direction.

    See Perform a Handover.

  8. (Optional) Migrate the group leader role back to the original group leader.