KVM Host Raid Consistency Checking
The Problem
Eventually a disk will fail, but in some cases a sector on a disk just goes bad without being used, and noone knows...
Consider the following.
- A sector on disk A goes bad. The sector is not used yet.
 - A sector on disk B goed bad. The sector on disk B is used.
 - Raid software will take disk B out of the array.
 - A new disk is added to replace disk B.
 - Resyncing from disk A to disk B is started.
 - Resyncing reaches bad sector on disk A,
 - Resyncing is now impossible.
 
The solution
Run regular raid consistency checks. This will detect the bad sectors before they become a problem. I run this on tuesday night. Edit your crontab.
# crontab -e
Add the following to your crontab
0 4 * * 2 echo "check" > /sys/block/md0/md/sync_action 0 4 * * 2 echo "check" > /sys/block/md1/md/sync_action