6.7. Backing Up and Restoring Management Database

Node information, statistics, and configurations are stored in a database on the management node (the one with the admin panel). Backups of this database are created automatically via a daily cron job that starts at 3:00 a.m. If the management node is not highly available, restoring such a backup recovers the node in case of failure or database corruption. Backup files are stored in the /mnt/vstorage/webcp/backup/ directory. The retention policy for management node backups is as follows:

  • All backups created within the last day are kept.
  • From backups created within the last 7 days, the newest for each day is kept.
  • From backups created within the last 7-14 days, the oldest one is kept.
  • From backups created within the last 14-45 days, the oldest for each week is kept.
  • Backups older than 45 days are deleted.

Important

Database backups cannot be restored if management node high availability is enabled.

To back up the database manually, open the SETTINGS > Management node > BACKUP screen and click BACKUP NOW.

../_images/stor_image1_7_ac.png

Once backup is completed, the Last backup date will be refreshed.

Warning

Do not rename the backup file! Otherwise you will not be able to restore the management database from it.

6.7.1. Restoring Management Database from Backup

You can restore a management node database from backup on the same management node or any node in the storage cluster.

Warning

The vstorage-ui-backend service must be running on a single node in the storage cluster. If the management node database is restored on a different node, the previous management node must be re-deployed.

Run the following script on the node where the MN database will be restored:

# /usr/libexec/vstorage-ui-backend/bin/restore-management-node.sh \
-x <public_net_iface> -i <private_net_iface> \
-f /mnt/vstorage/webcp/backup/<backup_file>

Where <public_net_iface> and <private_net_iface> are interfaces assigned the public and private networks, respectively. If required, you can specify the same network interface in both parameters. If the -f option is omitted, the management node database will be restored from the latest backup.

To access the admin panel, use the public IP address of the node with the restored MN database.

6.7.1.1. Restoring with Existing Compute Cluster

If you have the compute cluster deployed, the management node database must be restored only on one of the compute nodes.

When restoring the MN database with the compute cluster, note the following limitations:

  • If you create compute objects after the backup, they will be lost.
  • If you modify or delete compute objects after the backup, they will be restored as follows:
    • Compute objects used as configurations (flavors, storage policies, virtual networks, SSH keys) will be fully restored.
    • All other compute objects (VMs, volumes, images, etc.) will be partially restored. They will be shown in the admin panel but unusable. You will only be able to remove them from the admin panel.

To restore the MN database on a compute node, run the restoration script with the -n option:

# /usr/libexec/vstorage-ui-backend/bin/restore-management-node.sh \
-x <public_net_iface> -i <private_net_iface> -n

where

  • The <public_net_iface> and <private_net_iface> are interfaces assigned the public and private networks, respectively.
  • The -f option is omitted to restore the MN database from the latest backup.
  • The -n option denotes that the compute cluster will be reconfigured to use another management node.

If you restore the MN database on the same node, omit the -n option.

After restoration, virtual machines that resided on the failed management node will be unmanageable in the admin panel and can only be deleted. However, you can rescue them using the vinfra tool as follows:

  • VMs with disabled high availability will appear in the “Active” state: evacuate them using the vinfra service compute server evacuate command;
  • VMs with enabled high availability will appear in the “Rebuild” state: first reset their state with the vinfra service compute server reset-state command and then evacuate them running vinfra service compute server evacuate.
Jul 21, 2020
Print