Delete-Me-Not Files- Be Gone!

by Paul Arnote (parnote)

I'm sure this has happened to nearly everyone, at one time or another. You find a file stored on your drive(s) that resists all efforts to be deleted. Nothing you do will send this no-longer-wanted file into the digital afterlife. There is little else that is as annoying to computer users than an unwanted file that refuses to die.

Linux is awesome for its ability to read files on all sorts of file systems. EXT4, EXT3, NTFS, BTFS, FAT, FAT32, EXFAT, HPFS, HFS, HFS+, etc. -- none are really much of a challenge for Linux. However, that comes at a price, sometimes. All of those different file systems have different rules about how to construct a proper filename, and what characters can and cannot be used in file names. This can cause some huge problems for users who trade files with those who use different file systems ... or even for users who make a typo or other mistake when entering a filename in order to save a file.

Illegal Filename Characters

Of course, what's considered an illegal character depends on what file system you are using.

Unsurprisingly, Windows file systems appear to be the most restrictive. Here is a list of illegal characters in Windows file names (in bold): / ? < > \ : * | ". Also illegal is any character you can type with the Ctrl key. Under the FAT file system, the ^ symbol is illegal. FAT file system file names can be up to 255 characters long, while NTFS file system names can be up to 256 characters long. Under FAT and NTFS, the Windows full path may not be more than 260 characters. While the NTFS file system allows a full path of 32,767 characters, with each component of the path limited to 255 characters, many Windows applications will choke on files using such lengthy naming conventions (such as the Windows file manager, Windows Explorer). You also cannot place a space or period at the end of the file names under Windows.

As if all of that isn't enough, Windows also reserves the following file names: com1, com2, com3, com4, com5, com6, com7, com8, com9, lpt1, lpt2, lpt3, lpt4, lpt5, lpt6, lpt7, lpt8, lpt9, clock$, con, nul, and prn.

At the other extreme, there is only one illegal character for the Mac OS9 and OSX: the colon (:). Under HFS on OS9, filenames may be a maximum of 31 characters. Under HFS+ on OSX, filenames may be a maximum of 255 characters.

Nearly as simple and forgiving are *nix type of file systems. The only illegal characters are /, also known as the forward slash, and a null character. Filenames may be a maximum of 255 characters in length.

Go ahead and give it a try. You can name a file some*file.txt, some?file.txt, some\file.txt, and some:file.txt, but not some/file.txt. I wouldn't recommend doing this routinely, especially if you trade/send files to users of other file systems -- like Windows and Mac users. But, it does illustrate the forgiveness of the *nix type of file systems.

The best defense ...

The best way to avoid problems is to adhere to a few simple "rules" when naming your files. First of all, avoid using spaces in filenames and directory names. Sure, the *nix type of file systems can handle them, but you are setting yourself up for plenty of headaches if you are ever forced to use the command line to perform file maintenance duties. You also don't always know which Linux GUI apps are actually acting as a "front" for work that's actually occurring on the command line, behind the scenes. Some GUI applications will handle troublesome files just fine. Others, not so fine. To keep things simple, we'll deal with deleting these stubborn files from the command line.

For example, naming a file "My Yellowstone National Park Vacation 001.jpg" isn't really any more readable than "MyYellowstoneNationalParkVacation001.jpg". Your mind's eye will typically insert the spaces for you. If you find that too hard to read, try using an underscore (_) or dash (-) in place of spaces in your filenames, like this: My-Yellowstone-National-Park-Vacation_001.jpg.

Second of all, adhere to the "rules" of the file system that is most prevalently used. Like it or not, that would be the file system with the most restrictions, or Windows file system restrictions. By sticking with the rules that apply to Windows file systems, you'll avoid any problems when you trade/swap files with Windows users. Otherwise, you may have to rename the files you share with Windows users before they are able to access them.

Third, avoid using a dash (-) or space as the first character of your filenames. The reason for avoiding these two characters as the first character will become clearer when we talk about how to get rid of those stubborn files that resist being deleted.

Deleting those files that refuse to be deleted

Like it or not, the best way to eliminate files that otherwise refuse to be deleted is to use the command line. So, before we get started, let's create a couple of files in our ~/Documents folder (expands to home/username/Documents) that will defy "normal" attempts to delete. First, enter cd Documents at a command line prompt in a terminal session. Next, enter the commands exactly as shown in the command line screenshot below.

This will create two files. One is named -thatfile.txt, and the other is name " thisfile.txt." Notice that the second file starts with a space, while the first file starts with a dash. Both are problematic.

To illustrate, let's attempt to delete -thatfile.txt. Enter rm -thatfile.txt on the command line, and press the Enter key.

Notice that the rm command returns with an error. The dash that starts the filename makes the rm command think that a command line option is being given -- and in this case, an invalid command line option.

There are three ways to work around this problem. I've listed them below.

rm -- -thatfile.txt
rm ./-thatfile.txt
rm "-thatfile.txt"

The first option uses a double dash immediately following the rm command. This tells the command that what follows isn't an option. The second option uses a relative path statement (./), which tells rm to remove the file -thatfile.txt, which exists in the current directory. The third option encloses the filename within double quotes. All three methods will successfully delete the file.

Now, let's try to delete the second file. Enter rm thisfile.txt on the command line in a terminal session.

Notice how the rm command cannot find the file. Quite simply, the file named thisfile.txt does not exist. Instead, you can delete this file by using rm " thisfile.txt". This encloses the filename in double quotes, with the space at the beginning. You can also use the backslash (\) character to escape the next character (a space), like this: rm \ thisfile.txt.

Summary

Your best bet to avoid problematic files is two-fold. First eliminate spaces in your filenames. Second, honor the restrictions imposed by Windows with regard to illegal filename characters. By doing both, you'll go a long way towards avoiding any issues with files that refuse to be deleted.