Second Drawer

Rack map and subdomains

2010-07-03T14:00:00Z

The situation

You are using subdomains for locales such as en.example.com and th.example.com. Your app is Rack based and is behind Nginx with a ‘wildcard’ server_name directive such as:

server {
  server_name .example.com;
  ...
}

You are also using Rack’s map abilities to have two different applications mounted on different paths on your domain. This could be done like this in your config.ru file:

map '/users' do
  run Example::Admin
end

map '/' do
  run Example::Public
end

The problem

If the URL http://th.example.com is hit, Nginx will set ‘HTTP_HOST’ as ‘th.example.com’ but ‘SERVER_NAME’ will be ‘example.com’. What his means is that tests within Rack fail and result in neither of the above ‘maps’ matching so a 404 is returned.

A solution

In Rack apps you are able to modify the environment. This means that one way to fix this is to simply set the environment’s’ SERVER_NAME’ to equal ‘HTTP_HOST’ by including something like this in your config.ru file, before your map calls:

module Rack
  class Blah
    def initialize app
      @app = app
    end

    def call env
      env['SERVER_NAME'] = env['HTTP_HOST']
      @app.call env
    end
  end
end
use Rack::Blah

map '/users' do
...etc.

Optimise your images for performance and consistency

2010-05-20T14:00:00Z

In an effort to reduce your website’s page size, whether it be to just provide a better experience for the user or to work inline with Google’s plans to factor page speed into rankings, you will inevitably get to the point of dealing with your site’s images.

In many sites the images take up a large proportion of the site’s weight (in bytes). So getting the most out of them is very important. One tool that is essential in this regard is pngcrush.

pngcrush

Pngcrush is a tool that lossless reduces the size of PNG files. PNG are used mainly for styling the website and so are vital to the look (and often the readability) of the site itself. JPEGs are great for photographs which need no restriction on the number of colours. Compression of JPEGs are dealt with differently. Few modern designers use GIFs anymore now that IE supports alpha transparency in PNGs, so we will assume that we are only dealing with PNGs.

Pngcrush uses a number of techniques to reduce the size of your PNGs such as reducing the palette or applying different compression algorithms to the image data.

An example

A good example of the size difference is the menu image used on this site. An initial save of the PNG image was 159Kb. This same image when run through pngcrush resulted in a size of 14.5Kb, with no reduction in quality. It is a simple as this:

pngcrush infile.png outfile.png

or trying more methods to reduce size:

pngcrush -brute infile.png outfile.png

For my site, I am already using the sprite technique to combine images for my main menu so together I am able to reduce, not only the total amount to be downloaded, but also the number of HTTP requests needed to create my menu.

Alternatives

There are several other alternatives to pngcrush such as its fork, optiPNG. This does pretty much the same job.

Colour consistency

One thing that pngcrush does that others do not yet seem to do is reduce the file size by removing the colour correction (or colour profile) data from the file. This can sometimes reduce the size by up to 40%. What this also means is that your images will appear more consistently across browsers. How so?

Many image manipulation programs will place a colour profile within the image to enable different systems to display the image as close as possible to the original. Firefox, for instance, will use these profiles to adjust the colours before displaying them. Google Chrome, on the other hand, will not. What this means is that some images on your site will appear differently in Firefox and Chrome. It could also mean that colour values in CSS style sheets will not match those colours, while using identical codes, in PNG files.

The command line to remove all colour profiles from your PNG is as follows:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB infile.png outfile.png

Conclusion

By removing the colour profile data from your images, you not only reduce the file size but you force browsers (such as Firefox) to make do without the profile and so render them more consistently, with other systems and with CSS colour values. A faster and more colour accurate site.

Rack locale setter middleware

2010-04-22T14:00:00Z

For a project I am currently working on I am using the nice i18n library r18n which provides i18n for Rails, Sinatra and most general Ruby projects. It also is reasonably ORM agnostic and so I am using it quite easily with DataMapper.

I have decided to use a subdomain to specify the locale rather than having it be part of the path. So instead of this:

http://example.com/en/foo/

I have this:

http://en.example.com/foo/

I find this makes it easier by being able to specify relative paths everywhere that don’t need to have the locale played with. Switching the locale for the site is also just a matter of changing the domain.

That said, r18n expects the locale to be placed in the ‘params’ array for Sinatra and I thought a simple Rack middleware would do this well. So here it is:

module Rack
  class LocaleSetter
    def initialize(app)
      @app = app
    end

    def call(env)
      req = Rack::Request.new(env)
      if m = req.host.match(/^(?:www\.)?([a-z]{2})\./)
        locale = m[1]
      else
        locale = 'en'
      end
      req.params['locale'] ||= locale
      @app.call env
    end
  end
end

Due to many peoples’ habit of putting ‘www’ at the start of the domain, this should account for en.example.com and www.en.example.com. It also allows for the locale to be specified in the query string. So en.example.com/foo?locale=th will set the locale to ‘th’.

There are probably others out there but due to the simplicity of Rack based apps it is very easy to write, and adjust, your own.

tinydnsdyn - Dynamic DNS for DJBDNSs tinydns

2010-03-10T13:00:00Z

It has been a while since I have touched tinydns-dynamic but some new updates and changes have been made.

Why tinydnsdyn (previously tinydns-dynamic)

I have been using the djbdns DNS server package for many years and I seem to like the ‘DJB way’ (although I use ’runit’ instead of ‘daemontools’). I like knowing my services are monitored and restarted if they die. Almost all my services on each of my services are run this way and I have had no troubles whatsoever with their performance or uptime.

Tinydnsdyn is a simple script to enable dynamic DNS for the DNS server written by Dan Bernstein, tinydns. It filled a small hole that was missing on one of my previous jobs and was written quickly and roughly to fill that void. It has been some years since I have used it and now decided to clean it up and spit it out again.

Features?!

It is pretty simple but here are some of its features:

It uses Python which is available on most *nix servers.
It is designed to be run using daemontools (or similar) which if you are running tinydns then you will most likely already have that up and running.
Some basic security is built in. It uses a simple nonce digest authentication and you can restrict the hosts that can have dynamic IPs.
Simple logging

So see how it fits you.

Getting it

Code can be grabbed from the tinydnsdyn Git repository and you can tell me about all the bugs and mistakes on the tinydnsdyn ticket tracker

Magento 1.4.0 Nginx configuration

2010-03-02T13:00:00Z

UPDATE: I have added my php5.conf file for completeness, by request.

So after my previous post regarding an Nginx configuration for Magento, they have released version 1.4 and are now up to 1.4.0.1 already. I have migrated my Magento installations to the new stable version and this is my current Nginx configuration. Again, comments welcome.

This is in a configuration file called conf.d/magento.conf:

# not sure if this is needed but if the file exists just serve it
if (-f $request_filename) {
  break;
}

# this should catch everything
# if not found then it directs to the named location
location / {
  index index.php;
  error_page 404 = @magento;
}

# this has the catcher for the downloader, the rest to the index
location @magento {
  rewrite ^/downloader/(.*)$ /downloader/index.php$1;
  rewrite ^(.*)$ /index.php$1;
}

I also have a separate configuration file for static domains called conf.d/static_domain.conf that looks a little like this:

# no need to log
access_log  off;

# set a nice long expires for all content and enable caching
location / {
  expires max;
  add_header Cache-Control public;
}

This enables me to reuse the configuration settings nicely. These are then both combined into the virtual host configuration like this:

server {
  server_name example.com www.example.com;
  root /var/www/vhost/example.com/htdocs;
  access_log /var/log/nginx/example.com.access.log main;
  index index.php;

  include conf.d/magento.conf;

  location ~* ^(.+\.php)(.*) {
    include conf.d/php5.conf;
  }
}

server {
  listen 1.1.1.1:443 default ssl;
  server_name secure.example.com;
  root /var/www/vhost/example.com/htdocs;
  access_log /var/log/nginx/example.com.access.log main;
  index index.php;

  ssl_certificate conf.d/your-certificate.pem;
  ssl_certificate_key conf.d/your-certificate-key.pem;

  include conf.d/magento.conf;

  location ~* ^(.+\.php)(.*) {
    include conf.d/php5.conf;
    fastcgi_param HTTPS on;
  }
}

server {
  # have as many 'static' servers as you need
  server_name static.example.com media.example.com;
  root /var/www/vhost/example.com/htdocs;

  include conf.d/static_domain.conf;
}

The included php5.conf file looks a little like this:

include /etc/nginx/fastcgi_params;
fastcgi_index                index.php;
fastcgi_connect_timeout      60;
fastcgi_send_timeout         180;
fastcgi_read_timeout         180;
fastcgi_buffer_size          128k;
fastcgi_buffers              4 256k;
fastcgi_busy_buffers_size    256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors     on;
fastcgi_param                SCRIPT_FILENAME  $document_root$fastcgi_script_name;
fastcgi_split_path_info      ^(.+\.php)(.*)$;
fastcgi_param                PATH_INFO $fastcgi_path_info;
fastcgi_pass                 php5;

Notice the static domain’s root is the same as the main domain but it will only serve files that actually exist, nothing is passed to PHP etc. This obviously then requires that you set up Magento’s verious URL settings to serve your CSS, javascript and images from this static domain.

This cut a couple of seconds off a default install’s page loading time and can further be improved by combining javascript and CSS assets.