How to determine if consistency check has truly fixed the data?

[soberLee]

Imagine a common scenario:
A machine has broken a disk, and we need to perform disk replacement and reconstruction for the processes running on this disk.

There are some questions here：

1. If the cluster has a large amount of data, how long can the consistency check discover that the data maintained by these processes is lost and a new replica is added?
2. If the repair time is too long and other data center replicas also fail one after another, does it mean data loss?
3. Can running multiple consistency check processes accelerate checking cluster data status?Or can consistency checks be manually triggered for key values within a certain range?
4. Assuming it has successfully added replicas, how can we determine if there are 3 replicas and their locations for a corresponding key value pair in the cluster?(three_data_hall mode)

Thanks!

[Jingyu Zhou]

Can you repost your questions on the forum? Most of your questions are already answered in old posts there. The github is intended for describing concrete issues or features.

To save your time, short answers for your questions are:

1. The consistency check takes up to a full checking cycle (configurable, default is maybe 2 weeks) to detect corrupted data.
2. If you replaced a disk, I think the behavior is the storage server process using it will crash. So within seconds, data distributor will detect this issue and start repair, which usually can finish within 30 minutes (a rough number, depending on your exact cluster settings).
3. Consistency check doesn't support multiple, cooperative processes. There is a debugging feature in fdbcli to check a certain range, i.e., checkall.
4. There is a debugging feature in fdbcli, i.e., getall and getlocation.