Linux Anti Oops: How To Undelete Files

by Paul Arnote (parnote)

Disaster Scenario #1

You just returned from the vacation of your lifetime. You toured Europe for three weeks, then stopped for a week at a tropical paradise to cap it all off. Over the course of a month, you filled up two 16 GiB SD cards with your high resolution digital SLR camera. Now, it's time to sit down and transfer, catalog and edit your images. You even have plans of creating a travelogue for your church group.

During the process of copying all of your images to your computer's hard drive, you inadvertently delete all the images on the first SD card.

Oh NO! (Or insert your choice of expletive in place of the "no"). Whatever will you do? Those pictures are irreplaceable!

Disaster Scenario #2

You've been up all night working on a presentation that has to be presented to the board of directors of the company for which you work in less than 10 hours. You've been working hard on your presentation all night. Now, at 2 a.m., you decide to raid the refrigerator and grab yourself a late night snack, while taking a short break -- after saving your work. In your absencescared-man.png, the cat decides that the warm keyboard of your laptop would be the PURRFECT place to take a nap. While the cat twirls around to find a comfortable position in the middle of your warm laptop, it steps on a combination of just the right keys and inadvertently manages to delete your presentation.

There's no way you can recreate the presentation in the relatively short amount of time remaining before you have to make your presentation. Whatever will you do? The stakes are high for your career if this presentation goes well.

Never fear ...

Both of these scenarios are scary to even experienced computer users. I'm sure you can even think up some other similarly awful scenarios. It is safe to say that something like this has happened to EVERY computer user out there, at one time or another.

The key to being able to recover deleted files is to act quickly. Stop what you are doing immediately. You will want to be able to prevent the data you're trying to save from being overwritten, so stopping all other computer activity will help prevent that from happening. With today's modern file systems -- and even with file systems from days gone by -- the sooner you minimize drive activity, the greater the chance you will be able to recover any deleted files. Even a small cookie file from your browser could overwrite an important file that you might wish to recover.

Windows users have had the ability to "undelete" files for a very long time -- say, from the time when Windows ran atop MS-DOS. But for Linux users, it's an especially scary proposition. Linux, for all its glory and all that it gets "right," doesn't have a built-in ability to "undelete" files that were mistakenly deleted. With Windows, users have little to no choice over file systems, and are pretty much relegated to using FAT16, FAT32, ExFAT and NTFS file systems. Linux users, however, can use EXT2, EXT3, EXT4, ReiserFS, BtrFS, and a whole host of different file systems. Linux users can even use all of the Windows file systems, too. And this flexibility is most likely at the root of Linux's lack of a simple, built-in "undelete" files capability.

Instead, Linux relies on a some utilities that take care of the task of recovering deleted files, undeleting them, if you will. As far as I know, none of them are GUI, nor do any of them have GUI frontends. Much to the chagrin of those users who use GUI programs only, all four of the utilities are command line only programs. PCLinuxOS has four such utilities in its official repository: ntfsprogs, extundelete, testdisk and photorec. Let's take a look at each one.

ntfsprogs

I figure I'd just start out with this one, and get it out of the way. I have no way of actually testing this one, since I don't have any NTFS formatted drives. Nor do I have access to any. (Well, the hospital where I work does use Windows, but I suspect my employment there wouldn't last too long if I went in there mucking with their computers. My job is to work on people there, not the hospital's computers.) Such are the spoils of declaring yourself "Windows Free." However, if you still dual boot with any version of Windows, you may find yourself in need of this suite of programs specifically designed to work with the NTFS file system.

Here's what Synaptic has to say about ntfsprogs:

It is the last one listed in the suite of utilities that you might be interested in -- at least, when it comes to recovering deleted files.

So, in lieu of actually being able to try it out, the best I can do is refer you to the man page for ntfsundelete. Well, I can also summarize some of the information from there, too, and hit the highlights.

ntfsundelete accepts two command line arguments. You can follow with one of the options (which is optional) and the device to be scanned (required). By default, ntfsundelete simply scans the specified device for any recoverable files (specifying the -s option would also tell ntfsundelete to scan the specified device). It would look like this:

ntfsundelete /dev/sda1

Of course, that's presuming that /dev/sda1 is formatted with NTFS. Substitute your NTFS drive or partition for /dev/sda1. The output might look something like this:

Inode  Flags  %age     Date      Size  Filename
 6038  FN..    93%  2002-07-17  26629  thesis.doc

The scan information reveals the inode of the file, what percentage of the file is recoverable (in this case, 93%, which is a whole lot better than none of the file), the date of the file, the size of the file, and the filename. So, to undelete this file, you would execute the ntfsundelete command as follows:

ntfsundelete -u -m "thesis.doc" -o "thesis.doc" -d ~

Before we explain the command line, just note one thing. Get in the habit of placing the filename(s) in quotes, just in case there are spaces in the filename(s). The Linux command line uses spaces as delimiters between options and command line arguments, so spaces in the filename(s) could cause the command to fail -- unless you put the filename(s) in quotes.

The -u option tells ntfsundelete that you want to undelete a file. The -m option tells it to look for the file that matches the pattern that follows (in this case, the filename that appears in quotes). With the -o option, we tell ntfsundelete to name the output file "thesis.doc" (the same as the "lost" file). Finally, the -d option tells the program to output the file in the user's home directory (~).

There are a number of other options available, and they are explained fairly well on the ntfsundelete man page. For example, you can tell it to scan for files deleted within the last two days (use the -t 2d option), look for specific types of files (-s -m "*.pdf"), and so on.

ntfsundelete only reads from the NTFS volume, and never changes the NTFS volume. It also will not work on compressed or encrypted files. I especially love the "Caveats" section of the man page. It states "Miracles. ntfsundelete cannot perform the impossible," and it then goes on to explain why even files very recently deleted may not be able to be retrieved in a complete state on NTFS volumes.

If you routinely work with NTFS volumes, the ntfsprogs suite of programs deserve a place in your toolbox. Since we're only dealing with how to undelete files under Linux in this article, you can explore the other utility programs in the suite on your own.

extundelete

extundelete is a command line utility specifically for undeleting files from Linux EXT3 and EXT4 file systems. Here is the description from Synaptic:

Before we get to talking about how to run extundelete, note that there is NO "testing.txt" file in /usr/share/doc/extundelete-0.2.4/, as explained in the barebones description in Synaptic. Instead, there are brief instructions on how to run extundelete at the end of the README file. A more complete discussion of how to run it is explained at its non-standard man page, here. Ubuntu has created a more formal man page, which can be viewed here.

Simply entering extundelete on the command line by itself also displays a slimmed down command summary. Here it is:

[parnote-toshiba@parnote-toshiba ~]$ extundelete
No action specified; implying --superblock.
extundelete: Missing device name.

Usage: extundelete [options] [--] device-file

Options:
  --version, -[vV]       Print version and exit successfully.
  --help,                Print this help and exit successfully.
  --superblock           Print contents of superblock in addition to the rest.
                         If no action is specified then this option is implied.
  --journal              Show content of journal.
  --after dtime          Only process entries deleted on or after 'dtime'.
  --before dtime         Only process entries deleted before 'dtime'.

Actions:
  --inode ino            Show info on inode 'ino'.
  --block blk            Show info on block 'blk'.
  --restore-inode ino[,ino,...]
                         Restore the file(s) with known inode number 'ino'.
                         The restored files are created in ./RECOVERED_FILES
                         with their inode number as extension (ie, file.12345).
  --restore-file 'path' Will restore file 'path'. 'path' is relative to root
                         of the partition and does not start with a '/'
                         The restored file is created in the current
                         directory as 'RECOVERED_FILES/path'.
  --restore-files 'path' Will restore files which are listed in the file
'path'. Each filename should be in the same format as
an option to --restore-file, and there should be one
per line.
  --restore-directory 'path'
                         Will restore directory 'path'. 'path' is relative to
                the root directory of the file system.  The restored
                         directory is created in the output directory as
'path'.
  --restore-all          Attempts to restore everything.
  -j journal             Reads an external journal from the named file.
  -b blocknumber         Uses the backup superblock at blocknumber when opening
                         the file system.
  -B blocksize           Uses blocksize as the block size when opening the file
                         system.  The number should be the number of bytes.
  --log 0                Make the program silent.
  --log filename         Logs all messages to filename.
--log D1=0,D2=filename  Custom control of log messages with comma-separated
   Examples below:       list of options.  Dn must be one of info, warn, or
   --log info,error      error.  Omission of the '=name' results in messages
   --log warn=0          with the specified level to be logged to the console.
   --log error=filename If the parameter is '=0', logging for the specified
                         level will be turned off.  If the parameter is
                         '=filename', messages with that level will be written
                         to filename.
   -o directory          Save the recovered files to the named directory.
                         The restored files are created in a directory
                         named 'RECOVERED_FILES/' by default.

While this information is helpful, it really doesn't tell you how to run extundelete. For that, you'll need to refer to the information at the end of the aforementioned README file, or the link to the man page. Or, just follow along with me for a bit longer.

Before you can use extundelete, you will first need to unmount the volume that holds the file(s) you want to recover. For example, if you have your /home directory on a separate partition, issue the mount command without any command line argument.

[parnote-toshiba@parnote-toshiba ~]$ mount
/dev/sda5 on / type ext4 (rw,noatime,acl,commit=0)
/dev/sda6 on /home type ext4 (rw,noatime,acl,commit=0)

That tells me that my /home directory is on /dev/sda6. If the file(s) that I want to undelete are somewhere in my /home directory, I need to unmount the volume first. Alternatively, you can mount the volume as read only. Alternatively, you can remount the drive volume as read-only.

Unmount the drive: umount /dev/sda6
Mount the drive as read-only: mount -o remount,ro /dev/sda6
(Only use ONE of these commands, not both ... unmounting the drive volume is easiest.)

Before proceeding, double check to make sure that the volume is indeed now unmounted. To do so, rerun the mount command. The drive volume should not be listed if it was properly unmounted.

Now, to undelete a specific file, you will need to know the full path and filename of the deleted file. Execute the command like this:

extundelete /dev/sdaX --restore-file path/to/the/deleted/file

You can also do something similar to recover deleted directories. Execute the command like this:

extundelete /dev/sdaX --restore-directory path/to/the/directory

If you don't know or remember the entire path and filename, you can restore all of the deleted files on the drive volume. Execute the command like this:

extundelete /dev/sdaX --restore-all

All recovered files will be saved to a directory named RECOVERED_FILES/, unless you specify a different directory with the -o option, followed by the directory path.

It's also doubtful that extundelete will be of much use to you with SD memory cards, unless you've reformatted the SD card to use the EXT3 or EXT4 filesystem. However, it's unlikely that you would do that, since most devices that use SD cards (cameras, phones, tablets, etc.) expect to see either the FAT32 or ExFAT filesystems in use -- and the filesystem that is used to format nearly all SD cards by default.

Still, extundelete definitely belongs in any Linux user's arsenal of tools, and you should add it to yours.

testdisk

If anyone were to say that testdisk is an easy and intuitive program to use, they would be launched into orbit just by the intense laughter alone. It is neither the easiest nor the most intuitive program to use, and its full range of features will be understood only by the nerdiest of the nerds.

The description in Synaptic doesn't add much to help crack the mystery of how to run testdisk.

Tool to check and undelete partition


Tool to check and undelete partition. Works with the following filesystems:

    * BeFS ( BeOS )
    * BSD disklabel ( FreeBSD/OpenBSD/NetBSD )
    * CramFS, Compressed File System
    * DOS/Windows FAT12, FAT16 and FAT32
    * HFS and HFS+, Hierarchical File System
    * JFS, IBM's Journaled File System
    * Linux Ext2 and Ext3
    * Linux Raid
          o RAID 1: mirroring
          o RAID 4: striped array with parity device
          o RAID 5: striped array with distributed parity information
          o RAID 6: striped array with distributed dual redundancy information
    * Linux Swap (versions 1 and 2)
    * LVM and LVM2, Linux Logical Volume Manager
    * Mac partition map
    * Novell Storage Services NSS
    * NTFS ( Windows NT/2K/XP/2003/Vista )
    * ReiserFS 3.5, 3.6 and 4
    * Sun Solaris i386 disklabel
    * Unix File System UFS and UFS2 (Sun/BSD/...)
    * XFS, SGI's Journaled File System

Not much more light is shed on how to use it, either, by running testdisk --help at the command line:

[root@parnote-toshiba parnote-toshiba]# testdisk --help
TestDisk 7.0, Data Recovery Utility, April 2015
Christophe GRENIER 
http://www.cgsecurity.org

Usage: testdisk [/log] [/debug] [file.dd|file.e01|device]
       testdisk /list  [/log]   [file.dd|file.e01|device]
       testdisk /version

/log          : create a testdisk.log file
/debug        : add debug information
/list         : display current partitions

TestDisk checks and recovers lost partitions
It works with :
- BeFS (BeOS)                           - BSD disklabel (Free/Open/Net BSD)
- CramFS, Compressed File System        - DOS/Windows FAT12, FAT16 and FAT32
- XBox FATX                             - Windows exFAT
- HFS, HFS+, Hierarchical File System     - JFS, IBM's Journaled File System
- Linux btrfs                           - Linux ext2, ext3 and ext4
- Linux GFS2                            - Linux LUKS
- Linux Raid                            - Linux Swap
- LVM, LVM2, Logical Volume Manager     - Netware NSS
- Windows NTFS                          - ReiserFS 3.5, 3.6 and 4
- Sun Solaris i386 disklabel            - UFS and UFS2 (Sun/BSD/...)
- XFS, SGI's Journaled File System      - Wii WBFS
- Sun ZFS

You can only run testdisk as the root user. But, just take a look at the large number of different file systems testdisk works on!

Testdisk is menu driven, but that doesn't make it much easier to run. To better explain how to use testdisk to recover deleted files, we'll resort to using a series of screenshots, with an explanation of each.

To launch testdisk, simply issue the command "testdisk" on the command line as the root user. Once it appears, select the log file you want to use. Your choices are to either create a new log file, append the information to an existing log file, or to not create a log file at all. Press "Enter" when you've made your choice with the cursor keys.

Use the cursor keys to select the drive media that you want to scan with testdisk. Since the laptop I'm writing this on only has one hard drive, the choice is rather easy. Press the "Enter" key to proceed.

Next, select the partition table type for the drive you want to retrieve files from. For most users, it'll be the first choice of "Intel." However, if you have an EFI GPT partition (and I'm fairly certain you would know if you do), use the cursor keys to make that selection. Press the "Enter" key when you have highlighted the proper selection.

Use the up and down cursor keys to select the drive partition that contains the files you want to recover. On my laptop, my /home partition is on /dev/sda6, so this is the reason I've selected "6" from the list. This part initially had me befuddled, until I realized that the numbers correspond with the numbers that PCLinuxOS uses to identify my drive partitions. Once you've highlighted the proper drive partition, use the left and right cursor keys to select the action you want to perform. In our case, it's the "List" option. Press the "Enter" key.

Use the up and down cursor keys to select the directory on the partition that holds the files you want to recover. On my laptop, those are in the "parnote-toshiba" directory. Press the right cursor key to proceed.

Continue using the cursor keys to drill down through your directory structure to select the directory that holds the files you want to recover.

Deleted files will appear in red text. Continue using the cursor keys to highlight the file you want to recover. Press "c" to copy the currently selected file. If you need/want to recover multiple files, highlight the files with cursor key, one by one, press the ":" key to select the files, and then press "C" to copy all the selected files.

Then, testdisk will ask you where you want to save the selected file(s). I created a special directory on my computer, called "TestDel" to contain the files. Press "C" when you've selected the directory where you want your recovered file(s) saved.

The only confirmation you will receive will be some green text six lines down from the top. Even though the confirmation says "1 ok, 0 failed," there is no guarantee that the file was recovered properly. You should ALWAYS check the recovered file, just to be sure that it was recovered properly.

To do that, you will have to go to the directory where you specified the recovered file(s) to be saved. There, you will find the file in a directory structure that mirrors the original path to the file(s) that you recovered. For example, for the highlighted file above, it was stored here: /home/parnote-toshiba/parnote-toshiba/Magazine/Jan2016/SSS/. For some unknown reason, testdisk did not save the selected files in the selected directory. So, you will most likely have to search for your recovered files. Most likely, they are in a similarly named directory in your /home directory.

Also, all the recovered files will belong to the root user. As such, you may need to change ownership of the recovered files back to the proper user.

Testdisk will also work equally well for deleted directories, too, allowing you to recover entire directories that might have been mistakenly deleted. Just as with individual files, deleted directories will also show up as red, and you recover them exactly the same way.

Finally, to quit testdisk, press the "q" key multiple times, until you are finally back at the command line prompt.

Once you've gone through the process with testdisk, it becomes easier. But your first time is likely to be fraught with lots of trial and error -- unless you follow the directions here. I know my first time through testdisk was frustrating, and I found myself becoming easily "lost" in the directory structures and partitions. That is, until I figured things out and until I arrived on familiar turf with the directory structure that I was intimately familiar with from my /home directory.

Testdisk definitely deserves a place in your file recovery toolbox.

photorec

Of all the Linux "undelete" utilities, photorec is the most well known. Like all the others, photorec is a command line only program. The description in Synaptic pretty much tells the whole story.

There aren't a whole lot of options for photorec, but don't let that lead you to underestimating how powerful photorec is. Photorec will often work when all the other undelete utilities fail. As an example, I ran photorec on the hard drive in my laptop. I filled up four (4) directories with recovered files that I had deleted LONG, LONG ago. It came up with PDF files, image files, text files, music files, and a whole assortment of files that I had deleted eons ago. And I didn't even let it finish running, stopping the process after only a couple of minutes. Many of the files photorec was finding and recovering weren't even listed in the other undelete utilities as being available for retrieval. WOW! Plus, even more astounding is that many of the files were complete.

When you think about it, it's actually kind of scary at the same time that so much of what I thought I had deleted was recoverable -- things that I thought were long gone, given the amount of time that had elapsed since I deleted them. If I can recover those files, so can anyone else using the same simple methods described here.

Just as with testdisk, photorec has to be ran as the root user. When you launch it, you should see something like that pictured above. For this article, I inserted a SD card from my hunting trail camera. It's listed as /dev/sdb, and is selected in the image above. Press the "Enter" key to proceed, or use the left and right cursor keys to highlight the "Quit" option, then press the "Enter" key.

Also, if you're thinking that photorec looks quite similar to testdisk, it's because the same people are responsible for both.

Select the partition that stored the files you want to recover.

Now, select the filesystem type where the files were stored. On my SD card, it's a FAT32 filesystem.

You now select whether you want photorec to scan the unallocated space on the device, or if you want to extract files from the whole partition. Selecting "Free" will retrieve only deleted files, while selecting "Whole" will retrieve all files, even those still existing and accessible, as well as deleted files.

Photorec will then proceed to scan your selected drive/partition. It will display the elapsed time since it started the retrieval, as well as an estimate on how long the process will take. It also displays how many of each file type it retrieves. Pressing "Enter" at this stage will stop the scan.

Just in case you pressed "Enter" by mistake, you're given the chance to fix that mistake. Enter "Y" to quit the scan, or "N" to resume the scan/recovery.

Your recovered files will be placed in sequentially numbered directories in your /home directory named recup_dirXX, where XX is a number.

If you want more information on photorec, visit CGSecurity's photorec wiki page, where you'll find a step-by-step guide. You can also visit the photorec man page.

If no other "undelete" utility makes it into your file management toolbox, you should make sure that photorec ends up there. You might not need it today, tomorrow, next week, next month, or even next year -- but eventually, you WILL need it at some point in the future. Like I said earlier, accidental deletion of files occurs to all of us, sooner or later.

Retrieving open files

Occasionally, you might find yourself with a file open in a program, discovered that you have accidentally deleted the file, but without an apparent or obvious way to save the file again (yes, there are programs that don't give you the option to save an open file). If the file is open in a program that allows you to save data, just resave the data to your hard drive. But if it's the first scenario, all is not lost.

Thankfully, there is a way to save a file that is open in a program that doesn't have the option to save data. Like all the other methods listed here, you will have to resort to using the command line.

Let's assume we have a *.ogg file opened up for playback in xmms. Enter the following at the command line:

lsof -c xmms | grep ogg

The "lsof -c xmms" lists all the files that are open by xmms, and pipes that list to grep to list only the *.ogg files. The results will look something like the following:

xmms 9341 wachovia ogg 132r 8,1 349758 1287631 /usr/bin/xmms (deleted)

Now, we have all the information we need to save the file to a folder on our computer, replacing the one that was accidentally deleted. To do that, we use something like this:

cp /proc/9341/fd/132 ~/Music/my-music-file.ogg

The "cp" is the copy command. The 9341 is the process number provided in the output. The 132 is the descriptor shown in the output. Finally, the "~/Music/my-music-file.ogg" is the path and filename that we choose to save the new *.ogg file to.

Now, you have resaved the open file that you accidentally deleted.

Summary

Regardless of how careful you are, you will eventually -- at some point -- accidentally delete a file that you really need or want to keep. One of these tools or methods can help save your assets, so to speak.

There is yet another tool that can help recover deleted files. It is called debugfs. I have skipped it in this article due to its complexity. It makes testdisk look like a walk in Nooby Park. If you have an interest in or a need for debugfs, you can find a rather full discussion of it here. I certainly cannot add anything to what is presented there. The discussion is WAY above my pay grade and far exceeds my knowledge and understanding of such matters.

These tools also demonstrate how vulnerable your data may still be, even after you think that you might have deleted those files. Especially with photorec, I was able to recover a literal ton of files that I thought I had deleted months ago. That vulnerability should be prominent in your mind, especially if you sell or give away one of your used computers.

Next month, we'll take a look at ways to securely delete files so that they cannot be recovered, or to at least minimize the amount of information that can be recovered. With today's heightened concerns about the security of our private and personal data, it pays to exercise caution.