/sys/replication
Vault Enterprise license required
Restricted endpoint
The API path can only be called from the root namespace.Attempt recovery
This endpoint attempts recovery if replication is in an adverse state. For example: an error has caused replication to stop syncing.
| Method | Path |
|---|---|
POST | /sys/replication/recover |
Sample request
$ curl \ --header "X-Vault-Token: ..." \ --request POST \ http://127.0.0.1:8200/v1/sys/replication/recoverSample response
{ "warnings": ["..."]}Reindex replication
This endpoint reindexes the local data storage. This can cause a very long delay depending on the number and size of objects in the data store.
This endpoint requires 'sudo' capability.
| Method | Path |
|---|---|
POST | /sys/replication/reindex |
Parameters
diff(bool: false)– Enables a slower re-indexing which will perform a key level check to diagnose issues. Defaults false.force(bool: false)– Forces a complete re-indexing which only scans data available in the storage. Defaults false.skip_flush(bool: false)– Skips the tree flushing stage of the reindex process. This setting can be used to reduce the amount of time the tree is locked during a reindex process. If this node is killed before the full tree has been asynchronously flushed the reindex may not have applied fully and a new reindex may need to be done. Defaults false.
Sample payload
{}Sample request
$ curl \ --header "X-Vault-Token: ..." \ --request POST \ http://127.0.0.1:8200/v1/sys/replication/reindexSample response
{ "warnings": ["..."]}Check status
This endpoint print information about the status of replication (mode, sync progress, etc).
This is an unauthenticated endpoint.
| Method | Path |
|---|---|
GET | /sys/replication/status |
Sample request
$ curl \ http://127.0.0.1:8200/v1/sys/replication/statusSample response
The printed status of the replication environment. As an example, for a performance primary and DR primary node, it will look something like:
{ "data": { "dr": { "cluster_id": "f2c21cb5-523f-617b-20ac-c913d9154ba6", "known_secondaries": ["3"], "last_wal": 291, "merkle_root": "38543b95d44132138003939addbaf94125ec184e", "mode": "primary", "primary_cluster_addr": "", "secondaries": [ { "api_address": "https://127.0.0.1:49264", "cluster_address": "https://127.0.0.1:49267", "connection_status": "connected", "last_heartbeat": "2020-06-10T15:40:47-07:00", "node_id": "3" } ] }, "performance": { "cluster_id": "1598d434-dfec-1f48-f019-3d22a8075bf9", "known_secondaries": ["2"], "last_wal": 291, "merkle_root": "43f40fc775b40cc76cd5d7e289b2e6eaf4ba138c", "mode": "primary", "primary_cluster_addr": "", "secondaries": [ { "api_address": "https://127.0.0.1:49253", "cluster_address": "https://127.0.0.1:49256", "connection_status": "connected", "last_heartbeat": "2020-06-10T15:40:46-07:00", "node_id": "2" } ] } }}Possible values for connection_status are connected or disconnected. last_heartbeat is the timestamp of the
last time this node exchanged heartbeats with another node.
Sample response from performance secondary & DR primary
The printed status of the replication environment. As an example, for a performance secondary and DR primary node, it will look something like:
{ "data": { "dr": { "cluster_id": "e4bfa800-002e-7b6d-14c2-617855ece02f", "known_secondaries": ["4"], "last_wal": 455, "merkle_root": "cdcf796619240ce19dd8af30fa700f64c8006e3d", "mode": "primary", "primary_cluster_addr": "", "secondaries": [ { "api_address": "https://127.0.0.1:49277", "cluster_address": "https://127.0.0.1:49281", "connection_status": "connected", "last_heartbeat": "2020-06-10T15:40:46-07:00", "node_id": "4" } ] }, "performance": { "cluster_id": "1598d434-dfec-1f48-f019-3d22a8075bf9", "known_primary_cluster_addrs": ["https://127.0.0.1:8201"], "last_remote_wal": 291, "merkle_root": "43f40fc775b40cc76cd5d7e289b2e6eaf4ba138c", "corrupted_merkle_tree": false, "last_corruption_check_epoch": "1694456090", "mode": "secondary", "primaries": [ { "api_address": "https://127.0.0.1:49244", "cluster_address": "https://127.0.0.1:8201", "connection_status": "connected", "last_heartbeat": "2020-06-10T15:40:46-07:00" } ], "primary_cluster_addr": "https://127.0.0.1:8201", "secondary_id": "2", "state": "stream-wals" } }}/sys/replication/merkle-check
The Merkle check endpoint prints information about the corruption status of the Merkle tree. The corruption information provides insight about whether any current replication issues can be resolved by reindexing the tree. We recommend consulting the Merkle check endpoint before reindexing to ensure the process will be useful as reindexing can be time-consuming and lead to downtime.
You must provide an authentication token when calling replication/merkle-check.
| Method | Path |
|---|---|
POST | /sys/replication/merkle-check |
Sample request
$ curl \ --header "X-Vault-Token: ..." \ --request POST \ http://127.0.0.1:8200/v1/sys/replication/merkle-checkSample response
The response provides information about the health of the Merkle composite tree.
The merkle_corruption_report stanza provides information about merkle tree
corruption. The corrupted_root field indicates whether the root of the tree
is corrupted. The corrupted_tree_map field identifies any corruption in the
roots of its subtrees, including replicated and local subtrees. In the event of
corruption within a page or subpage of the tree, corrupted_tree_map includes
the page number along with a list of corrupted subpage numbers. The status details
indicate the extent of corruption within the merkle tree.
{ "request_id": "d4b2ad1a-6e5f-7f9e-edfe-558eb89a40e6", "lease_id": "", "lease_duration": 0, "renewable": false, "data": { "merkle_corruption_report": { "corrupted_root": false, "corrupted_tree_map": { "1": { "corrupted_index_tuples_map": { "5": { "corrupted": false, "subpages": [ 28 ] } }, "corrupted_subtree_root": false, "root_hash": "DyGc6rQTV9XgyNSff3zimhi3FJM=", "tree_type": "replicated" }, "2": { "corrupted_index_tuples_map": null, "corrupted_subtree_root": false, "root_hash": "EXmRTdfYCZTm5i9wLef9RQqyLCw=", "tree_type": "local" } }, "last_corruption_check_epoch": "2023-09-11T11:25:59.44956-07:00" } }}