> It was the worst-case scenario for RAM failure: bit flip errors that get written back to the disk. I discovered that several video files that I had been editing had corrupted bits, and were no longer usable.
What a horrible failure mode! Kudos to the OP for even thinking to investigate this.
The mentioned memtest86 tool sounds quite useful - so useful, in fact, that it seems it should be part of the operating system. If my OS is perfectly willing to write corrupted memory back to disk, and it's designed to run on thousands of different OEM hardware configurations that may or may not have ECC RAM, then I would expect it to proactively monitor for faulty RAM and bitflips.
Is there a good reason why this isn't the default behavior of operating systems (maybe it is - I use Mac and don't know much about Windows)? It seems like a trivial diagnostic tool that could prevent a lot of headache. But perhaps the problem is there's no reliable test without some number of false positives, so popping a warning on the screen that your RAM appears unhealthy seems like a good way to confuse the average user. But then again, so does silently corrupting a video file.
Note that memtest86 is a test application, not a means of ensuring integrity: it writes known patterns into memory in order to test it. Windows also has a built-in version which has the same limitations: you reboot into a special mode where it runs the test, it doesn't work online or validate actual application data.
The OS layer is probably the hardest place to do online application data integrity checking: it's too low level to know what data is important and which parts will change and why in a way which allows checksumming to work effectively and efficiently, but too high above the hardware to be able to check the integrity of memory without a massive performance penalty (especially when it comes to how memory is moving in and out of caches). Most solutions work at a higher level in the application itself or lower level with ECC RAM as mentioned in the article.
> The mentioned memtest86 tool sounds quite useful - so useful, in fact, that it seems it should be part of the operating system
Windows has the Memory Diagnostics Tool, and your linux probably has a memory tester option in the boot manager. Of course you have to run them manually, which is a bit bothersome. Thee have been attempts at kernel patches for linux to test memory in the background, going at least back to 2005 [1], but there were probably some before that. The simpler versions just test whatever memory is free, the more sophisticated aproaches try to move stuff in physical memory to free each region periodically so it can be tested. But in the end it's spending CPU cycles on a problem most users never experience.
Windows has a memory test tool built in called the Windows Memory Diagnostic. It runs on the next reboot to fully test system memory.
Apple Diagnostics check memory as well, though maybe not as thoroughly as the Microsoft or memtest86 do (multiple passes).
I've had memory errors that only appeared on two of eight passes through the whole memory test overnight. They certainly caused occasional issues with games though.
> Is there a good reason why this isn't the default behavior of operating systems
Because any equivalent of memtestx86 will flag a LOT of hardware for the buggy piles of shit that they really are.
If Windows started flagging people's crappy cheap hardware, Microsoft would get a lot of grief and have to spend a chunk of money on customer support and PR.
I dunno, I've had overclocked memory where memtest86 wouldn't report errors in multiple passes, yet TestMem5 (weird win32 tool from some .ru site) tells you it's bad in about ten seconds. memtest86 doesn't seem to be that stressful for memory.
memtest86 needs to run alone so it can test the entire physical RAM, and the actual, meaningful tests that actually test RAM and not L1 take forever to run. What would you have the OS do?
It still doesn't have full access it doesn't memtest its own code segment for instance. Memtest can also be run as a usermode program (although it needs permission to lock the pages in memory).
Operating systems zero memory before handing out pages, I wonder what the perf impact would be to run at least a very basic test on a page before allocating it. Obviously allocation would be slower, but maybe it's worth it in some scenarios.
What a horrible failure mode! Kudos to the OP for even thinking to investigate this.
The mentioned memtest86 tool sounds quite useful - so useful, in fact, that it seems it should be part of the operating system. If my OS is perfectly willing to write corrupted memory back to disk, and it's designed to run on thousands of different OEM hardware configurations that may or may not have ECC RAM, then I would expect it to proactively monitor for faulty RAM and bitflips.
Is there a good reason why this isn't the default behavior of operating systems (maybe it is - I use Mac and don't know much about Windows)? It seems like a trivial diagnostic tool that could prevent a lot of headache. But perhaps the problem is there's no reliable test without some number of false positives, so popping a warning on the screen that your RAM appears unhealthy seems like a good way to confuse the average user. But then again, so does silently corrupting a video file.