Hello!
I'm not sure if you've seen this but people assert with some explanation that it is not true that a scrub combined with rotten memory will kill your on-disk data.
https://clusterhq.com/blog/file-systems-data-loss-zfs/
http://www.reddit.com/r/DataHoarder/comments/2suurf/how_true_is_this_statement_if_you_experience_a/
The statement is like this:
If a checksum fails due to a rotten bit being inside either the checksum or the data itself, ZFS assumes that something is wrong with the data.
In that case, if parity or mirroring is used, ZFS will use the parity or the mirror. I understand that if the data is from a mirror, it's just exactly the
same data with the same checksum. Either that data will also be corrupt, due to another rotten bit in memmory, which will just result in a lost file.
How, with parity, I can understand that you can use the parity to reconstruct the data, but how does one validate that data with a RAIDZ1 or higher?
Assuming that we read a chunk of data of one drive and that it is seen as rotten due to a bitflip, by reading all the other blocks and do the parity calculation, it can be reconstructed. If the data itself was at fault, all seems well. If the checksum was at fault, the result of the parity calculation will not match the checksum and you lose the file.
At this point, data is still valid on disk, ZFS just can't serve the data. Replace the memory and all is well.
I'm really trying to find a reasonable scenario where a scrub will actually murder the data-at-rest on your system. I want to understand why.
If the bad ram only affects data or checksums, I would expect nothing permament should happen to your data-at-rest, even with a scrub.
The scenarios where I can consieve that you may lose your entire zpool is this:
1. bitflip hits kernel code, more specifically zfs code
2. bitflip hits ZFS in-memory meta data structures
1 -> sounds like a very remote chance, only a few hundred k of 'static' memory that might get affected.
2 -> sounds way more likely, although less likely than data getting (silently) corrupted.
I wonder what I'm missing here. The risk of not using ECC memory is not just losing a few files you just were transfering, but the statement is that existing data or even the zpool can get corrupted.
What I find interesting is that even the ZFS author Matt Ahrens states that non-ECC memory is not that of an issue for home use. But as seen in the discussion on BSDNOW, it seems that the risk of meta data getting corrupted, is not even acknowledged.
I would very much appreciate it if anyone would help shed light on this and help me understand.
Richard Yao Mats Taraldsvik •
4 months ago
There seems to be a misconception that ZFS has a "destroy data if checksum fails" mode. I will probably address this in a future blog post after finishing background research on how people came to think this. However, it is categorically false. If ZFS detects a bad block, it will automatically heal it by fetching a good block and writing that as a replacement with a report that it fixed a checksum failure. If ZFS cannot find a good block, it will report the file as corrupt for the system administrator to correct. In no way does ZFS ever make a bad situation (checksum failure) worse by replacing good data with bad data.
Something else comes to mind. ZFS calculates parity when using RAIDZ(1|2|3).
So we have a RAIDZ1 array. We have some data. We calculate the parity of those two
data blocks. That parity is also checksummed I assume, BUT before the checksum of
the parity block is calculated, the parity block is already mangled by bad memory.
So in this case, the data is fine, the data will checksum fine. But now there is a small landmine
hidden in your file system as part of the parity. Even if you replace the faulty data at a later date,
In case a drive fails, data is reconstructed based on false parity data, resulting in data corruption.
Long story, but is this in any way plausible? It's not that I've read the ZFS source or anything.