Databases, Systems & Networks

Ubuntu 13.10 was released yesterday. As well as other package updates, it includes an upgrade from Apache 2.2 to 2.4, which broke a few things on my dev machine.

After the upgrade I was getting the standard « It works! » Apache message on all of my virtual hosts. Running apache2ctl configtest gave me the familiar warning:

Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1. Set the ‘ServerName’ directive globally to suppress this message

You often get this error in an out-of-the-box Ubuntu setup as you need to tell Apache what hostname to use as the default. I’d previously fixed this by creating a config file at /etc/apache2/conf.d/fqdn that contains only:

ServerName localhost

so first I had to investigate why this was no longer working. It turned that Apache 2.4 no longer reads configs from conf.d. If you check the end of apache2.conf, you’ll see that it now looks in a folder called conf-enabled instead:

# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf

This folder contains symlinked files pointing at conf-available, making the conf setup similar to the approach used for vhosts. So, to fix the error I moved all of my configuration files from conf.d to /etc/apache2/conf-available/, added a .conf file extension to each one, and then ran:

a2enconf fqdn

to setup the symlink.

After restarting Apache, the FQDN error was gone, but I was still getting the « It works! » message for all my vhosts. I had another look at the main Apache config and found:

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

So by default Apache expects the vhost files to have a .conf file extension (which it didn’t before). Mine just used the hostname as the filename. So I renamed each of these and updated the symlinks that point to them.

After that everything was back to normal.

A full 2.2 to 2.4 upgrade guide is available in case you have any other issues. Of particular note is the change to the syntax for ‘Deny from’ and ‘Require’, so if you start getting « 403 Forbidden » errors on things that previously worked you’ll probably need to update those directives.

Many/all of these issues could be mitigaged by either keeping old configs during the upgrade or by installing the provided compatibility module, but in development I prefer to always install package maintainer’s configs when upgrading so that I know how to fix things if they were to go wrong in production.

Every now and then I find myself in a situation where I have a folder (I’ll call it source) of files and nested folders, possibly many levels deep, that I want to copy into another folder (which I’ll call target). target already contains some of the files and folders I’m copying, and it also has files and folders that are not present in source.

Simply copying source to target’s parent folder in the Mac OS X Finder will replace everything in target with the contents of source. This is not always what I want, and in my opinion it’s one of the biggest flaws of the Mac OS X Finder. Not just Mac OS X actually – back in the pre-Mac OS X days there was a utility called Speed Doubler that patched the Finder to add a smart replace option when copying files.

It’s possible to manually open each folder and their subfolders and copy just the files, but it can be very tedious. There are also third party software options that let you merge files when copying, and if you have Apple’s Developer Tools installed there is the FileMerge utility.

However, you can open a Terminal window and copy the files from the command line, which saves you from installing extra software. Since I keep looking up the syntax every time I need to do this I decided to document it here for future reference.

cp

One command line utility that can copy directories without replacing everything in them is cp:

cp -pRv source/ target

The pRv options do the following:

• p preserves timestamps, flags, modes, and ownerships of files
• R copies the entire subtree
• v makes cp output the name of each file that is copied

Note: The / after the name of the source directory is important since it tells cp to copy the contents of the directory and not the directory itself.

rsync

You can also use rsync:

rsync -av source/ target

The av options do this:

• a tells rsync to copy recursively and preserve file attributes
• v makes rsync print information to the terminal window about what was copied

Just as with the cp command, the trailing slash after the source directory is important to make sure only the contents of the directory are copied.

ditto

A third option is ditto:

ditto -V source target

The V option prints information about what was copied.

Deleting files that don’t exist in the source directory

Sometimes you want a merge to delete files that exist in the target directory but not in the source directory. That’s easy with rsync (but be careful as there is no undo):

rsync -av --delete source/ target

But wait! What if the target is under version control? Won’t that delete any .svn or .git directories as well? Yep. That can be avoided by adding filters. Let’s say you want to keep all .htaccess and .svn files or directories in the target directory. This does just that:

rsync -av --delete --filter="- .htaccess" --filter="- .svn" source/ target

A useful tip when you involve delete is to add the n option at first to do a “dry run” and only show what would have been deleted. Again, be careful with delete since there is no undo.

Hoping for Finder integration

It would be great if Apple could make it possible to use the Finder to copy folders like this. It could be a secret option somewhere or invoked when you hold certain modifier keys when copying. It doesn’t matter as long as it’s possible.

Most people probably don’t need this feature every day, but when you do need it it can save lots of time. Having the feature built into the Finder would also reduce the risk of people accidentally deleting files because they don’t realise copying folders replaces everything inside them.