Rack-aware conversion using restore
- How-to guide
- 3-min read
Convert a Managed Cluster to a rack-aware deployment using the backup and restore method.
The restore method is universal and works for larger Managed Clusters and more complex topology changes. However, the restoration process is time-consuming, and because backups run daily, some data loss is involved. Transaction storage (distributed traces) isn't contained in the backup. For small Managed Clusters where one node can contain a full replica, use the rack-aware conversion using replication method instead.
Preparation
-
Verify that the Managed Cluster has a recent backup. See Back up and restore a Managed Cluster.
-
Prepare new nodes. Make sure the disk partition allocated for Cassandra storage on the new node is sufficient to contain the entire Cassandra database (with margin for compaction and new data). The disk must be at least twice the combined Cassandra storage of all existing Managed Cluster nodes. Place Cassandra data on a separate volume to avoid disk-space issues from different data types.
Restore the Managed Cluster into new racks
To restore the Managed Cluster into new racks:
-
Stop the existing Managed Cluster to prevent two Managed Clusters with the same ID from connecting to Dynatrace Mission Control.
See Start/stop/restart a node. -
Run the cluster restore procedure (Back up and restore a Managed Cluster), with one modification: run the Dynatrace Managed installer on each node with the rack parameters.
Run the installer in parallel on every node, using the following parameters:
--rack-name: defines the rack to which this node will belong.--rack-dc: defines the data center to which this node will belong.--restore: switches the installer into the restore mode.--cluster-ip: IPv4 address of the node on which you run the installer.--cluster-nodes: the comma-delimited list of IDs and IP addresses of all nodes in the Managed Cluster, including the one on which you run the installer, in the following format<node_id>:<node_ip>,<node_id>:<node_ip>.--seed-ip: IPv4 address of the seed node.--backup-file: the path to the backup*.tarfile, which includes the path to the shared file storage mount, the cluster ID, the node ID, the backup version, and the backup*.tarfile in the following format:
<path-to-backup>/<UUID>/node_<node_id>/files/<backup_version_number>/<backup_file>
Backup path example
In this example path:
/mnt/backup/bckp/c9dd47f0-87d7-445e-bbeb-26429fac06c6/node_1/files/19/backup-001.tarthe parts of the path are as follows:<path-to-backup>=/mnt/backup/bckp/<UUID>=c9dd47f0-87d7-445e-bbeb-26429fac06c6<node_id>=1<backup_version_number>=19
While the backup is in progress, two backup directories may be present with different backup version numbers:
- The directory with the lower version number contains the old backup. This directory is deleted after the backup completes.
- The directory with the higher version number contains the backup that's in progress.
The backup version number increments with each backup execution.
Get the IDs and IP addresses from the inventory you created before you started. For example:
- The IP address of the node to restore:
10.176.41.168. - The node IDs and new IP addresses of all nodes in the Managed Cluster:
1: 10.176.41.168, 3: 10.176.41.169, 5: 10.176.41.170.
sudo ./tmp/backup-001-dynatrace-managed-installer.sh--rack-name rack2--rack-dc datacenter1--restore--cluster-ip "10.176.41.168"--cluster-nodes "1:10.176.41.168,3:10.176.41.169,5:10.176.41.170"--seed-ip "10.176.41.169"--backup-file /mnt/backup/bckp/c9dd47f0-87d7-445e-bbeb-26429fac06c6/node_1/files/19/backup-001.tar
Verify the conversion
The restore process places the loaded data directly in the target racks. After the conversion is complete, the Deployment status page in the Cluster Management Console shows the racks:
