Silk Road forums
Discussion => Security => Topic started by: astor on January 03, 2013, 03:29 am
-
This is a professional analysis of secure data erasure. It comes from the Center for Magnetic Recording Research. It clearly sates that multiple writes are no more effective than a single write (I highlight the comments in the text). Full analysis is here: http://cmrr.ucsd.edu/people/Hughes/DataSanitizationTutorial.pdf
TL;DR Physical destruction is the most secure data erasure method, but surprisingly, "simply bending a disk makes [data recovery] nearly impossible."
Data Sanitization in Hard Disk Drives
Four basic sanitization security levels can be defined: weak erase (deleting files), block erase (overwrite by external software), normal secure erase (current drives), and enhanced secure erase (see below). The CMRR at UCSD has established test protocols for software secure erase.
Block erase is most commonly used. While it is significantly better than no erase, or file deletion, or drive formatting, it is vulnerable to malware and incomplete erasure of all data blocks. Examples are data blocks reassigned by drives, multiple drive partitions, host protected areas, device configuration overlays, and drive faults.
Normal secure erase is approved by NIST 800-88 for legal sanitization of user data up to Confidential, and enhanced secure erase for higher levels. Enhanced level has only recently been implemented, initially in Seagate drives, and these drives are under evaluation by the CMRR.
These four erasure protocols exist because users make tradeoffs between sanitization security level and the time required. A high security protocol that requires special software and days to accomplish will be avoided by most users, making it little used and of limited practical value. For example, the old data overwrite document DoD 5220 calls for multiple block overwrites of Confidential data, which can take more than a day to complete in today's large capacity drives. So users make tradeoffs between the time required to erase data and the risk that the next drive user may know and use recovery techniques which can access weakly erased data.
For all but top-secret information, users will usually turn to erasure methods that take minutes rather than hours or days. They will select a method that gives them an acceptable level of security in a reasonable time window.
Physical Drive Destruction
To positively prevent data from recovery, disks can be removed from disk drives and broken up, or even ground to microscopic pieces. (Actually, simple disk bending is highly effective, particularly in emergency situations.) Obsolete government document DoD 5220 required physical destruction of the storage medium (the magnetic disks) for data classified higher than Secret. Even such physical destruction is not absolute if any remaining disk pieces are larger than a single 512-byte record block in size, about 1/125" in today's drives. As linear and track densities increases, the maximum allowable size of disk fragments become ever smaller. Destroyed disk fragments of this size have been studied by the CMRR. Magnetic microscopy is used to image stored recorded media bits.
Some storage products are more easily destroyed than hard disk drives, such as magnetic disk data cartridges, tape cartridges, secure USB drives, and optical media.
Disk Drive Degaussing
Degaussers are used to erase magnetic data on disk drives. They create high intensity magnetic fields that erase all magnetic recordings in a hard disk drive, including the sector header information on drive data tracks (information necessary for drive head positioning and data error recovery). In addition, track and disk motor magnets are often also erased by degausser magnetic fields. Like physical destruction, when a disk drive has been successfully degaussed it is no longer useable.
Drive designers continually increase the linear density of magnetic recording to create higher data storage capacity per disk. This raises the disk magnetic coercivity, the field required to write bits on the magnetic media. As the magnetic coercivity increases, the fields required to erase the data on recorded disks increases. Thus an older degausser may not fully erase data on a newer hard disk drive. New perpendicular recording drives may not be erasable by present degaussers designed for past longitudinal recording drives.
Future generations of magnetic recording media may use very high magnetic coercivity disks to achieve areal densities greater than 500 gigabits per square inch. These drives may have technology using laser light in the magnetic write element of the disk drive, to raise the temperature of a spot on the magnetic medium in order to lower the magnetic coercivity to the point where the write element can record a bit on the very high coercivity magnetic media. For disk drives using this Heat or Thermally Assisted Magnetic Recording (HAMR/TAMR) technology the degausser field required to erase the disk drive at room temperatures may be impossible or impractical to achieve. In this case the drive may have to be physically destroyed.
"Hybrid drives" are now being introduced for notebook or laptop computers that have flash memory write cache on hard disk drive circuit boards. Magnetic degaussing would not affect any resident data on such semiconductor memory chips. Data on these non-volatile semiconductors would have to be sanitized using some other technique. For all these reasons degaussing of all the data on hard disk drives will become increasingly impractical.
Nondestructive Data Erasure
Sanitization of data on a hard disk drive is not a simple task. Deleting a file merely removes its name from the directory structure's special disk sectors. The user data remains in the drive data storage sectors where it can be retrieved until the sectors are overwritten by new data. Reformatting a hard disk drive clears the file directory and severs the links between storage sectors, but the user data remains and can be recovered until the sectors are overwritten. Software utilities that overwrite individual data files or an entire hard drive are susceptible to error or malicious virus attack, and require constant modifications to accommodate new hardware and evolving computer operating systems. It is difficult for external software to reliably sanitize user data stored on a hard disk drive.
Many commercial software packages are available using variations of DoD 5220, making as many as 35 overwrite passes. But in today's drives, MULTIPLE OVERWRITES ARE NO MORE EFFECTIVE THAN A SINGLE OVERWRITE. Off-track overwrites could be effective in some drives, but there is no such drive external command for a software utility to move heads offtrack. And even three overwrites can take more than a day to erase a large capacity hard disk drive. In busy IT facilities, such time is often not available and IT personnel are likely to take short cuts.
DoD 5220 overwriting has other vulnerabilities, such as erasing only to a drive's Maximum Address, which can be set lower than its native capacity; not erasing reallocated (error) blocks; or miss extra partitions. External overwrites cannot access the reallocated sectors on most drives, and any data once recorded is left on these sectors. These sectors could conceivably be recovered and decoded by exotic forensics. While enterprise-class drives and drive systems (SCSI/FC/SAS/iSCSI) allow software commands to test all the user blocks for write and read ability, mass market drives (PATA/SATA) cannot read, write, or detect reassigned blocks since they have no logical block address for a user to access.
-
The Secure Erase (SE) command was added to the open ANSI standards that control disk drives, at the request of CMRR at UCSD. The ANSI T13.org committee oversees the ATA interface specification (also called IDE) and the ANSI T10.org committee governs the SCSI interface specification.
Secure erase is built into the hard disk drive itself and thus is far less susceptible to malicious software attack than external software utilities. The SE command is implemented in all ATA interface drives manufactured after 2001 (drives with capacities greater than 15 GB), according to testing by CMRR. A standardized internal secure erase command also exists for SCSI drives, but is optional and not currently implemented in SCSI drives tested.
Secure erase is a positive easy-to-use data destroy command, amounting to "electronic data shredding." Executing the command causes a drive to internally completely erase all possible user data record areas by overwriting, including g-list records that could contain readable data in reallocated disk sectors (sectors that the drive no longer uses because they have hard errors). SE is a simple addition to the existing "format drive" command present in computer operating systems and storage system software, and adds no cost to hard disk drives. Because the Secure Erase command is carried out within hard disk drives, no additional software is required either.
Secure erase does a single on-track erasure of the data on the disk drive. The U.S. NATIONAL SECURITY AGENCY PUBLISHED AN INFORMATION ASSURANCE APPROVAL OF SINGLE PASS OVERWRITE, AFTER TECHNICAL TESTING AT CMRR SHOWED THAT MULTIPLE ON-TRACK OVERWRITE PASSES GAVE NO ADDITIONAL ERASURE. Secure erase has been approved by the U.S. National Institute for Standards and Technology (NIST), Computer Security Resource Center. NIST document 800-88 approves SE at a higher security level than external software block overwrite utilities like as Norton Government Wipe, and it meets the legal requirements of HIPAA, PIPEDA, GLBA, and Sarbanes-Oxley.
Software overwrite utilities running in protected execution environments (e.g. running inside file system hardware like RAID arrays or inside secure computers) could be verified secure under NIST 800-88. For the most sensitive data, the government requires physical destruction of drives. Drive manufacturers today are pursuing higher security secure erase (including secret data), via in-drive data encryption (see below).
Data Encryption Secure Erase
Recently, 2.5-inch hard disk drives for laptop computers have been introduced which encrypt user data before recording -- internal full data encryption. Such drives provide protection of data should the laptop or drive be lost or stolen, and even provide high protection from forensic data recovery. These drives also offer a new, instantaneous way to sanitize data on a hard disk drive -- by securely discarding the encryption key.
Why encrypt data at rest in drives instead of in computers, such as by user application programs that access the data? Because computer level data encryption defeats the purpose of many important data management functions, such as incremental backup, continuous data protection, data compression, de-duplication, virtualization, archiving, content addressable storage, advanced routing, and thin provisioning. Defeating these operations causes significant penalties to enterprise storage companies in data access speed and cost. Each of these operations exploits the structure of user data, and needs to inspect the data. They become inefficient or nonfunctional if the data has been randomized by encryption. For example, data compression ratios may fall from more than 2:1 to less than 1:1, because compressing random data can expand it instead. De-duplication won't find identical data sets if they are encrypted by different users.
Computer level encryption could be employed with in-drive encryption as well, the double encryption does no harm and provides additional security. In-drive encryption can relieve encryption key management problems inherent in removable storage, like laptop disk drives or tape backups. In fact, hardware-based tape drive encryption may become widespread 11 by 2007 due to widely publicized losses of backup tape reels containing identity theft data on millions of people.
Full Disk Encryption (FDE) Enhanced Secure Erase (FDE-SE), securely changes the internal drive encryption key, to render encrypted user data on disk indecipherable. This is enabled via the Enhanced SE command in the present ATA ANSI specs. FDE SE encryption needs to be tested for protection against advanced forensic analysis. The results will determine the erasure security data level -- Confidential, Secret, Top Secret, or higher. The US Commerce Department prohibits most 256-bit and higher encryption export overseas, limiting FDE E-SE to AES-128-bit encryption (since disk
drives are a global industry).
AES-256 bit encryption in FDE drives could allow FDE SE at a somewhat higher security level. Note that a FDE E-SE operation amounts to double AES-128, because the data encrypted by the discarded key is decrypted by the new key, and AES is a symmetric encryption scheme. It would appear that a brute force attack on double AES-128 requires
the same computational effort as single AES-256. For paranoid-level security, the cypt-text in an FDE disk drive could be eliminated by a Normal OW SE done after the FDE E-SE.
An open industry standard for FDE is being worked on by the Trusted Computing Group overall specification (the Storage Working Group in trustedcomputinggroup.org). Drive members of the TCG include Seagate, HGST, Fujitsu and WD. SE via encryption may be included, consistent with the ANSI open standards for ATA drives (t13.org). CMRR has begun testing FDE-SE drives. They take less than 15 milliseconds to complete an Enhanced SE; while a 750 GB ATA-interface HDD can take over an hour to erase using conventional Secure Erase (or many hours using external overwrite software).
Computer Forensics Data Recovery
Forensics recovery uses exotic data recovery techniques by experts with advanced equipment. Its normal purpose is to recover data from failed hard disk drives, and for legal discovery. Forensic companies can successfully recover unerased but protected data in a disk drive using electronic instrumentation. However, the secure erase commands discussed above erase all user data on the disk drive beyond physical disk drive forensic recovery. Drives old enough to permit such attack are too old to have the Secure Erase built-in command.
Paranoid-level recovery concerns based on hypothetical schemes are sometimes proposed by people not experienced in actual magnetic disk recording, claiming the possibility of data recovery even after physical destruction. One computer forensics data recovery company claims to be able to read user data from a magnetic image of recorded bits on a disc, without using normal drive electronics. Reading back tracks from a disk taken out of a drive and tested on a spin stand was practical decades ago, but no longer with today's microinch-size tracks.
The time required by exotic technologies is itself a barrier to data recovery and increases data security. Also, accessing data from magnetic images requires overcoming almost a dozen successive magnetic recording technology hurdles. Even if these hurdles were overcome, about an hour would be required to recover a single user data block out of millions on a disk. Recovering substantial amounts of data in less than months requires that the disk be intact and undamaged, so that heads can be flown over it to obtain data playback signals; then overcoming these technology hurdles. Simply bending a disk makes this nearly impossible, so physical damaging drives to warp their disks makes recovery practically impossible.
Other "experts" claim that limited information can be recovered from unerased track edges. But this has been shown to be false by tests at CMRR. Such recovery also presumes detailed technical knowledge of the drive's magnetic recording design. Charles Sobey at ChannelScience.com wrote an illuminating article on drive-independent data recovery, showing how difficult these hurdles are.
-
Don't forget to read about SSDs and secure data erasure, if you intend to use a modern device with an SSD hard drive.
That is a WHOLE OTHER kettle of fish.
-
Legitimate information regarding the subject.
-
Overwhelming amount of information, but this is good stuff!
-
Bumping this thread, because there seems to be a lot of interest in secure data erasure at the moment.
-
Thank you. There's no use putting extra wear on your HDD by overwriting it 35 times via the infamous Guttman method, unless of course the drive was made in the 90s.
-
how do we determine if our hdd or ssd has secure erase? - i tried searching for it with the computer's search function with zero results, yet none of my drives are more than 3 years old
-
You must have used very specific search terms to get no results. Maybe you included your hard drive's serial number or something.
Boot an Ubuntu/Xubuntu/Lubuntu Live CD and follow these instructions:
https://ata.wiki.kernel.org/index.php/ATA_Secure_Erase
They tell you how to determine if your hard drive has Secure Erase.
Or follow these instructions from the people who wrote the analysis above:
http://cmrr.ucsd.edu/people/Hughes/HDDEraseReadMe.txt
-
Recovering substantial amounts of data in less than months requires that the disk be intact and undamaged, so that heads can be flown over it to obtain data playback signals; then overcoming these technology hurdles. Simply bending a disk makes this nearly impossible
I would not trust bending a disk to keep me secure. It does nothing to actually destroy data on the drive, their assumption is simply that it makes it hard to read the data that is there.
-
To positively prevent data from recovery, disks can be removed from disk drives and broken up, or even ground to microscopic pieces. (Actually, simple disk bending is highly effective, particularly in emergency situations.)
This is a bad method to try unless you really know what you are doing, breaking a platter up only destroys data where the fracture lines are, it can still be put back together and read with spin stand microscopy. I don't know if it will work for bent platters but it certainly will for shattered platters (that is one of the things it is primarily used for, reading data off shattered drive platters). I wouldn't trust bending much either as it doesn't actually destroy any data and only is a physical limitation attempting to prevent forensic tools from reading the data that is still there. Grinding to microscopic bits will leave data behind as well, but it would probably be infeasible although not impossible to perform spin stand microscopy on a platter broken into hundreds of thousands of bits, since it needs to be pieced back together.
Off-track overwrites could be effective in some drives, but there is no such drive external command for a software utility to move heads offtrack.
But there are drive internal programs (firmware) that can do this, namely Secure Erase.
Recently, 2.5-inch hard disk drives for laptop computers have been introduced which encrypt user data before recording -- internal full data encryption. Such drives provide protection of data should the laptop or drive be lost or stolen, and even provide high protection from forensic data recovery. These drives also offer a new, instantaneous way to sanitize data on a hard disk drive -- by securely discarding the encryption key.
Some of the newest generation SSD's have automatic passwordless encryption that seems to exist for the sole purpose of allowing you to Secure Erase by wiping a random key stored in a protected erasable area.
Other "experts" claim that limited information can be recovered from unerased track edges. But this has been shown to be false by tests at CMRR. Such recovery also presumes detailed technical knowledge of the drive's magnetic recording design. Charles Sobey at ChannelScience.com wrote an illuminating article on drive-independent data recovery, showing how difficult these hurdles are.
Obviously the people who made Secure Erase thought the track edges could have data recovered from them, since it uses an off center wipe as well. I did read research showing that they were incapable of pulling data off track edges though.
As for the comment that spin stand microscopy no longer works, well that is news to me. Personally I will tend toward the side of caution and not try to destroy data by smashing my drive platter (as historically this has not worked), nor by bending my platter (as this doesn't actually destroy data just makes it hard to access), and rather will stick with what I currently do, which is Secure Erase followed by a single pass of random data with something like DBAN.