10.11. Backing up and restoring management database

Node information, statistics, and configurations are stored in a database on the management node (the one with the admin panel). Backups of this database are created automatically via a daily cron job that starts at 3:00 AM. If the management node is not highly available, restoring such a backup recovers the node in case of failure or database corruption. Backup files are stored in the /mnt/vstorage/webcp/backup/ directory. The retention policy for management node backups is as follows:

  • All backups created within the last day are kept.
  • From backups created within the last 7 days, the newest for each day is kept.
  • From backups created within the last 7-14 days, the oldest one is kept.
  • From backups created within the last 14-45 days, the oldest one for each week is kept.
  • Backups older than 45 days are deleted.

Important

Database backups cannot be restored if management node high availability is enabled.

To back up the database manually, open the Settings > Management node > Backup screen, and then click Backup now.

../_images/mn_backup1_ac.png

Once backup is completed, the Last backup date will be refreshed.

Warning

Do not rename the backup file! Otherwise you will not be able to restore the management database from it.

10.11.1. Restoring management database from backup

You can restore a management node database from backup on the same management node or any node in the storage cluster.

Warning

The vstorage-ui-backend service must be running on a single node in the storage cluster. If the management node database is restored on a different node, the previous management node must be re-deployed.

Run the following script on the node where the management node database will be restored:

# /usr/libexec/vstorage-ui-backend/bin/restore-management-node.sh \
-x <public_net_iface> -i <private_net_iface> \
-f /mnt/vstorage/webcp/backup/<backup_file>

Where <public_net_iface> and <private_net_iface> are interfaces assigned the public and private networks, respectively. If required, you can specify the same network interface in both parameters. If the -f option is omitted, the management node database will be restored from the latest backup.

To access the admin panel, use the public IP address of the node with the restored management node database.

10.11.1.1. Restoring with existing compute cluster

If you have the compute cluster deployed, the management node database must be restored only on one of the compute nodes.

When restoring the management node database with the compute cluster, note the following limitations:

  • If you create compute objects after the backup, they will be lost.
  • If you modify or delete compute objects after the backup, they will be restored as follows:
    • Compute objects used as configurations (flavors, storage policies, virtual networks, SSH keys) will be fully restored.
    • All other compute objects (VMs, volumes, images, etc.) will be partially restored. They will be shown in the admin panel but unusable. You will only be able to remove them from the admin panel.

To restore the management node database on a compute node, run the restoration script with the -n option:

# /usr/libexec/vstorage-ui-backend/bin/restore-management-node.sh \
-x <public_net_iface> -i <private_net_iface> -n

where

  • The <public_net_iface> and <private_net_iface> are interfaces assigned the public and private networks, respectively.
  • The -f option is omitted to restore the management node database from the latest backup.
  • The -n option denotes that the compute cluster will be reconfigured to use another management node.

If you restore the management node database on the same node, omit the -n option.

After restoration, virtual machines that resided on the failed management node will be unmanageable in the admin panel and can only be deleted. However, you can rescue them using the vinfra tool. Do the following:

  • VMs with disabled high availability will appear in the “Active” state: evacuate them vy using the vinfra service compute server evacuate command.
  • VMs with enabled high availability will appear in the “Rebuild” state: first reset their state by using the vinfra service compute server reset-state command, and then evacuate them by running vinfra service compute server evacuate.
Oct 06, 2020
Print