Files and multi-site and migration, oh my!

Drupal itself does not, in fact, care where the files directory is. When dealing with public files, the files directory can be placed pretty much anywhere under the drupal root as long as it is web-accessible. You then need simply go to Admin > Site configuration > File system and enter the appropriate path, and any properly written module code will then pick up that value and put its files inside that directory as appropriate. No muss, no fuss. (And I can't recall the last time I ran into a module that got that wrong; if you do, delete it from your system immediately until the author figures out how to use Drupal. Or even better, file a patch.)

Before Drupal 6, the default files directory was simply "files", that is, a directory called "files" in the Drupal web root. That was all well and good and worked fine, with one caveat. It meant that when backing up or restoring or upgrading your site there were two directories that were not part of the core files: files and sites. Couldn't we get that down to one directory?

It also runs into a problem with multi-site installations. Drupal will actually function just fine if you have 10 multi-site installs all using the same files directory; new files will get silently renamed if the file name is already in use (true for individual sites as well) and everything keeps on working. However, that is organizationally a disaster, especially if you want to backup or restore just one site.

For those and other reasons, Drupal 6 changed the default location to sites/default/files. That has two advantages. One, everything non-core is in the sites directory for easier backup. Two, it provides a natural way for multi-site installs to split up their files directories: sites/siteone.com/files, sites/sitetwo.com/files, etc. No, muss, no fuss.

Well, actually, there is fuss. Drupal keeps a record of all files it knows about in the files table in the database. Among other things, it tracks the path of the file... relative to Drupal root, not to the files directory. That means you can't move the files directory once you start populating the site with content, because the database will still point to the old location for any existing files.

Why would you want to do that? Well you don't, but you have to with multi-site. Multi-site support in Drupal 6, as with all previous versions, has a direct, literal one-to-one mapping from the requested domain to the sites directory for that site. That is, sites/siteone.com gets used if and only if the request comes in to siteone.com (or a subdomain of it). That makes migrating the site from one domain to another, say from an in-house test server to a live server, quite difficult. The site directory changes, which means the paths to modules and themes stored in the system table change. And, more importantly, the paths in the files table break, too.

Now, that is not an unsolvable problem. The system table can be updated if you can just get to the Modules and Themes admin pages, and a simple PHP script to change all of the records in the files table is not exactly hard to write. We've done it before and the resulting sites worked fine. But really, who wants to have to muck with the files table directly just to migrate a site? You have a good chance of breaking something along the way if you're not careful. Really.

Instead, there are two things we've done to deal with multi-site weirdness. The first is a patch that we wrote for Drupal 7 to allow aliasing of multi-site directories. The second is to go back to a top-level files directory.

But wait, doesn't that mix all the files together? Not at all! Remember, Drupal doesn't care where the files directory is, as long as it doesn't change out from under it. Instead of putting all uploaded files into the files directory, we put them into a subdirectory: files/siteone, files/sitetwo, files/sitethree, etc. Then it doesn't matter if the site lives at siteone.com or at siteone.testserver.palantir.net. The files directory doesn't move, it's still easy enough to back up (just like it was in Drupal 5), it can migrate from server to server without any database trickery, and all sites on the single install have their own easy to manage files directory. And all is right with the world.