getiblog

This is a quick post to share something I experimented with today to figure out. Perhaps it’ll be helpful to you on your sites.

I finally got fed up with the fact that, no matter what, when I have a blank new site, the browsers (and search engines) all insist on requesting a favicon.ico and a robots.txt file. In general, I want there to be some default files for those two, when I have a blank site. Only if I really care to override do I want a custom file for either.

So, I created a default favicon.ico (taken directly from the awesome FamFamFam Silk Icons set) file, and a default robots.txt file:

# robots.txt
#
User-agent: *
Disallow:

I uploaded these files into a central location on my server (not necessarily web accessible). Now, how do I get all my dozens of sites to use one of these files by default, if the site in question doesn’t already have that file defined? Took some experimentation, but I used some “mod_rewrite” and “Alias” trickery to get it done.

Here’s the relevant excerpt from my Apache configuration:

Alias /default-favicon.ico /path/to/favicon.ico
Alias /default-robots.txt /path/to/robots.txt

<FilesMatch "(favicon\.ico|robots\.txt)$">
        RewriteEngine on
        RewriteCond %{REQUEST_FILENAME} favicon\.ico$ [NC]
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteRule .* /default-favicon.ico [L]

        RewriteCond %{REQUEST_FILENAME} robots\.txt$ [NC]
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteRule .* /default-robots.txt [L]
</FilesMatch>

That’s it! First, I use a “FilesMatch” clause to restrict my URL rewriting to only the two files I care about. Then, I use a mod-rewrite ruleset, only if the requested file isn’t found, to replace the request for “/favicon.ico” with “/default-favicon.ico” (and similarly for robots.txt). Then, notice that I have set up two “Alias” statements (“Alias” comes from the mod-alias module), which map those default-* filenames to the actual default files in the central location. Apache first processes the mod-rewrite ruleset, and then re-passes through URL parsing, where it then picks up on the Alias.

You may ask, why can’t you just use the mod-rewrite “RewriteRule” to map directly to the proper file location? I dunno, I wish I could. But apparently, mod-rewrite insists on paths for “RewriteRule” being relative to the DOCROOT of the site in question, rather than to the filesystem. So “/default-robots.txt” really means “/path/to/site-docroot/default-robots.txt”. The “Alias” takes care of mapping from that new bogus location to the real location of the default files. Bam. Works.

Hope that’s maybe helpful to some of you. No more uncacheable 404′s for favicon.ico and robots.txt. No more bogging down of my error logs. Yay.

Tweet