Easy Drupal CDN integration for fun and profit

published on April 24, 2012

Speed up your Drupal site with a CDN in a few minutes.

The fun part: it’s nice to learn how to make any Drupal site significantly faster in a few minutes. The profit part: faster websites lead to more users and more revenue.

This article covers the common case: you have a small to medium size (≤1M page views per month), without massive amounts of large images, you’re using Drupal 71 and you only want to spend a few euros or (U.S.) dollars per month on a CDN. (You already know what a CDN is, right?)
So, you want your Drupal site to be faster, only spend a few minutes doing so, don’t want to deal with infrastructure and want to keep the costs very minimal. You’ve come to the right place.

Also: don’t worry about the cost: this little experiment will only cost you a few cents.

We’ll be using Amazon CloudFront and my CDN module for Drupal — I hope you like it. I’ve tried to make it as easy to use as possible. It’s solid, it’s got unit tests where appropriate, it’s used by ±2,000 Drupal sites — http://economist.com and http://worldpressphoto.org amongst others.
My largest deployment is for http://driverpacks.net, where the total CDN bill for well over half a million page views per month is less than USD $10!

Part 1: Create an Amazon CloudFront Distribution with a Custom Origin

We’ll be using Amazon’s CloudFront CDN service. Why?
They’re reliable, have solid performance (though not the best), are affordable (though not the cheapest) and are continuously expanding (i.e. adding more Points of Presence) around the globe. In just the last six months, they added five new edge locations2. So your site will automatically get faster in more locations around the globe, without paying more. Also see their list of edge locations or their map. Amazon is also cutting prices regularly (July 2011 was the last time). You’ll generally also never run into problems — after all, there are bigger fishes out there that help drive the infrastructure forward, you can just get an easy ride along.

You are of course free to use a different CDN, but then you’ll have to make sure you’re using an Origin Pull CDN.

So, go to https://aws.amazon.com/ and sign up for an account if you haven’t already. Go to the AWS Management Console and sign in. Go the CloudFront tab and click the “Create Distribution” in the top left corner.

Amazon recently updated their management console, and now (January 5, 2013) the steps below are outdated: A wizard modal window will pop up (on the first page of the wizard: “Distribution Type”), where you can choose between two types of origins3: Amazon S3 Origin and Custom Origin. Choose Custom Origin; this means Amazon CloudFront’s edge location servers will come to your Drupal site’s web server and retrieve the files it needs to serve to end users. You won’t have to deal with Amazon S3 at all.
In the Origin DNS Name field, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Next: the Protocol Policy field. If your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click “Continue”.

Now you’re on the second page of the wizard (“Distribution Details”). You don’t need to change anything here. Click “Continue” again. We’re now on the “Review” page of the wizard. Click “Create Distribution”. You should get a message stating “You have successfully created a CloudFront Distribution.” Great!

The new steps are: A 2-step wizard will appear. In the first step, you must select the delivery method: “Download” or “Streaming”. Choose “Download” (the default) and click “Continue”. In the second step, there are three groups of settings: “Origin Settings”, “Default Cache Behavior Settings” and “Distribution Settings”. We only care about the first group of settings, the others we can leave at the default (though you may want to change the “Price Class” setting in “Distribution Settings” to indicate that you only want to use U.S. & European edge locations, for example). So, “Origin Settings”: for “Origin Domain Name”, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Now three additional settings will appear (because you’re using a custom origin and not Amazon S3): “Origin Protocol Policy”, “HTTP Port” and “HTTPS Port”. All of them have sane defaults. Only the “Origin Protocol Policy” setting (defaulted to HTTP Only, changeable to Match Viewer) is somewhat likely to be relevant to you. if your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click “Continue”.

At the top of your CloudFront Management Console’s table of distributions, you should now see something like this:

CloudFront Distribution created, status 'InProgress'.

Your newly created distribution’s status will remain at InProgress for a few more minutes, then it will change to Deployed. As soon as it is Deployed, we can actually use it.

If you have more special needs, consult Amazon’s documentation on creating distributions. More often than not, you won’t need that though.

Part 2: integrate your Drupal site with the CDN!

Download the CDN module for Drupal (version 2.5 or later). Install it like you install any other module. After the installation, go to admin/config/development/cdn (admin/settings/cdn on Drupal 6). There’s three tabs:

  1. General
  2. Details
  3. Other

We’ll cover them one by one. Note that you can install the Advanced Help module to get more and better help (it’ll help you explore all features of the CDN module).

First tab: “General”

There’s really only one important setting here: the status of the CDN module. You can either disable it, enable it, or put it in testing mode, which is somewhere in between. In testing mode, none of your visitors will get files from the CDN; only users with the access files on CDN when in testing mode permission will get to see it. This is perfect to test whether the CDN integration is actually working correctly. So, for now, let’s put it in testing mode.

The second (and last) setting on this tab is the Display statistics setting. Users with the access per-page statistics permission will get to see, well, per-page statistics at the bottom of each page: “what percentage of files is served from a CDN”, and so on. Enable this for now, so you get to see the results for each page on your site.

CDN settings: 'General' tab

Second tab: “Details”

At the top of this tab, there’s a Mode setting, which allows you to choose between Origin Pull and File Conveyor. Choose Origin Pull (the default).

Next, there are mode-specific settings. The most important one is the “CDN mapping” setting. Here we define which files are mapped to which CDN (in case you’re using multiple CDNs — or static file servers).
Go back to the CloudFront Management Console and copy the domain name of your CloudFront Distribution. In my case: d67something714.cloudfront.net. Now paste this into the “CDN mapping” setting, but prepend it with http://. So your CDN mapping is now set to http://d67something714.cloudfront.net. This will cause all files to be served from CloudFront.

Read the included Advanced Help documentation to see what else is possible (e.g. serve images from a different CDN or only serve CSS and JS from a CDN), but for us (and for http://wimleers.com), this CDN mapping will do.

Optional, but recommended: Far Future expiration

The last setting on the “Details” tab is the Far Future expiration checkbox. This single checkbox has the capability to make your site much faster than when you were just using a CDN without this setting enabled. It can also reduce your CDN bill significantly.

What does it do? Well, it ensures that all files are served from the CDN in the most optimal way possible: compressed (gzipped) whenever possible and with the most optimal HTTP headers. These HTTP headers tell your visitors’ browsers to cache files “forever”. The results: less requests to the CDN (reducing your bill), and hence an even faster site! Whenever the file changes, its URL is changed automatically, so that your visitors don’t continue using that old buggy JS or CSS file forever, but they get the new files immediately.

There’s a catch though: to be able to set these HTTP headers and automatically4 change file URLs to be unique, we have to serve the files through PHP (Drupal) instead of letting the web server take care of it… This has adverse performance effects: using PHP to serve static files is not efficient!
However, the CDN caches these files for long periods of time (CloudFront edge locations come back to the origin once every 24 hours, no matter what expiration date you configure), so in fact that’s just fine. Hence you should only enable this setting if you’re using a CDN or a reverse proxy such as Varnish. It won’t work when you’re using “your own CDN”, i.e. just a static file server such as nginx or lighttpd. It will work when you’re using the same web server for these alternative domains as you’re using for serving the actual Drupal site, but in that case, very long load times are the result.
Again: no worries, the CDN module checks this automatically for you. After you submit the “Details” form, you should see a status message that confirms everything is a-ok:

Far Future expiration — domain check

Once you’ve enabled the Far Future expiration setting, a new setting appears: Unique file identifier generation. This defines how each file’s unique file identifiers are generated. The default works fine for all sites, but for complex and/or high-traffic sites, you may want to fine-tune this.

CDN settings: 'Details' tab

Third tab: “Other”

We won’t go into the details of the various settings on this tab, but there’s only one we care about for the scope of this article: the CDN supports HTTPS setting. If your site is using HTTPS and you configured your CloudFront distribution’s Protocol Policy to Match Viewer, then you can enable this setting as well. Whenever your site is accessed through HTTPS, your files will still be served from the CDN, via HTTPS!

CDN settings: 'Other' tab

All done!

If you haven’t already, make sure you’ve enabled Drupal’s CSS aggregation, block caching and page caching. The CDN module automatically rewrites image URLs in blocks and nodes, but we don’t want that to happen on every page load if it’s not necessary — hence enable those caching layers.

By now, the status of your CloudFront distribution should have changed to “Deployed”. So let’s give this a try! The following screenshot was taken at http://wimleers.com/blog/facebook-week-12:

Per-page statistics for wimleers.com

If everything is looking good after browsing through your website and confirming that the CDN integration is working correctly everywhere, head back to the “General” tab and change the status from Testing mode to Enabled. Now your site’s actual visitors will also get all files served from the CDN, so they should be experiencing a faster site! Definitely return visits should be significantly faster. Your status report should look similar to this:

Final status report

Congratulations, your site is now accelerated by a CDN!


  1. This will work for Drupal 6, too. ↩︎

  2. October 2011: Sao Paulo, Brazil. December 2011: South Bend, Indiana, U.S.A; San Jose, California, U.S.A.; second edge location in New York City, New York, U.S.A. February 2012: Osaka, Japan; Milan Italy. ↩︎

  3. The “origin” defines what the “origin server” is, i.e. where CloudFront edge location servers around the world will go to get the files they need to serve to your site’s visitors.
    Also see “The Origin Server” in Amazon’s documentation. ↩︎

  4. “Automatically” as in “out of the box”; without modifying Apache’s .htaccess or httpd.conf files and similar web server config files for other web server software. ↩︎

Comments

Wim Leers's picture
Wim Leers

From the article:

If you have more special needs, consult Amazon’s documentation on creating distributions. More often than not, you won’t need that though.

In there, you’ll see this:

Origin ID: A string that will help you distinguish this origin from other origins in this distribution. If you add more origins and cache behaviors to the distribution later in the process, you will link origins with cache behaviors using the origin ID.

I think that answers your question? :)

Remove Stains's picture

Done and done - everything was straight forward and worked right away. Thanks a bunch!

-Matt

Wim Leers's picture
Wim Leers

That’s the goal :)

Erik Halber's picture
Erik Halber

Hi again, I have a new issue. I just launched the following site: athenadoctrine.com. If you scroll over the images in the grid, you’ll notice that all the urls being with cdn-1, 2 or 3. The links are being generated via a view which I’ve done some extensive rewriting off.

Can you tell me why it’s using cdn-N in the beginning of all of these urls instead of the default www?

Thanks!

Erik Halber's picture
Erik Halber

I’ve looked a little more into it and it’s really weird. I am wrapping the title of the article in an anchor whose value is the path to the content. I am wrapping the image with the same token but it’s rendered value uses the cdn-N subdomain. (I have also unchecked the link this image to content checkbox in the image field).

Erik Halber's picture
Erik Halber

Ok, it’s solved, but it feels hackish. I ended up just wrapping the tag in a span.

Previous markup that didn’t work [a][img][/a] - this creates the cdn-N subdomain

Markup that works: [a][span][img][/span][/a]

Is this an issue/bug with the CDN module or views?

Wim Leers's picture
Wim Leers

This is a known bug in the CDN module that has been fixed in the 2.x-dev release, and the bugfix will be included in the upcoming 2.6 release :)

Paul Grimshaw's picture

Hey Wil,

I am trying to setup a Custom Origin but this doesn’t appear to be an option any more - I can only select S3 buckets. As you suggest in the Article I shouldn’t be doing this I am not sure how to proceed - do you have an up to date guide on getting this up and running.

Thanks, Paul.

Wim Leers's picture
Wim Leers

Hi Paul,

First of all: my name is Wim, not Wil. Both derive from “William” (or rather, mine derives from the German “Wilhelm”), but they really are different ;)

Note that Amazon AWS still supports this. It’s just slightly different. I updated the article to explain how you can do it today.

Paul Grimshaw's picture

Huge apologies Wim, bad enough on d.o and I did it on your own site :(.

Thank you for the update, I am progressing through it now.

Kind Regards, Paul.

Pierre's picture
Pierre

Perfect! Thanks a lot! i implemented it at Appsta. Just to avoid duplicate between cdn subdomain and main domain, i added this to settings.php :

<?php if($_SERVER[‘HTTP_USER_AGENT’] == “Amazon CloudFront”) { $file_dir = “sites/default/files”; if (strpos($_GET[‘q’], $file_dir) { } else{ drupal_add_http_header(‘Status’, ‘403 Forbidden’); print ‘403 ForbiddenNot Allowed

You cannot access website via CDN-url.

’; exit; } }  ?>

It prevents your website from being reached from cdn.example.com or from the cloudfront url, except for files under default folder.

Wim Leers's picture
Wim Leers

Thanks Pierre! That feature has already been developed on Drupal.org though, the patch has been sitting there (ready to be committed) for a while now and will be in the upcoming 2.6 release of the CDN module (2.5 will only contain bugfixes).

See http://drupal.org/node/1060358.

Jonathan's picture
Jonathan

Hi Wim,

Thanks for the great tutorial. Before I start getting stuck in, I was wondering how this works for imagecache presets and stuff? I will be using this on a website that utilises different presets for different views etc.

Would be interesting to find out how to make best use of CDN using imagecache. Apologies if this has been answered previously, I couldn’t see it in the comments list.

Many thanks for your time.

Wim Leers's picture
Wim Leers

When the Far Future setting is disabled, nothing special happens: the CDN will hit the origin to retrieve the file, and by doing so, it will cause the file to be generated if it does not already exist.

When the Far Future setting is enabled, the CDN module will perform a HTTP request to generate the file (from within the PHP), so that it can serve the image directly. It needs to do this to be able to generate the UFI.

jonathan's picture
jonathan

Thanks, I appreciate that. Sounds like Imagecache presets are no different then and from your advice, I assume it’ll all work as normal.

Appreciate your time and efforts, thank you! :)

stephen's picture
stephen

Hi Wim

I’m obviously doing something wrong as I’ve followed all the instructions, it works - as such - but for some reason I always lose my front-end styles (using Bartik), this happens in either ‘testing mode’ or ‘Enabled ). Please note that ‘admin’ styles are not lost. Is there anything you can think of that may cause this? I have filed an issue at https://drupal.org/node/2029781

Thank you

Stephen

Sam's picture
Sam

Hello Wim, Thanks for the Nice Tutorial. Can you please let me know where to set up the expiration time. My website has a dynamically changing content every day(It is a news Portal). May be about 140 nodes being created each day.Is it good to have a CDN enabled for a dynamic site?

Wim Leers's picture
Wim Leers

I apparently don’t state it explicitly in the article, but: that’s not what the CDN module does.

The CDN module only makes it possible to serve files (images, videos, CSS, JS, etc.) from a CDN, not the actual HTML pages themselves.

Sam's picture
Sam

Thanks for the response Wim. Can you please let me know How often does CDN fetch from website to serve files (images, videos, CSS, JS, etc.). I upload youtube videos very often daily, so the thumbnails are created very often too. Is it dependent on Cron Run or Caching lifetime specified.

Wim Leers's picture
Wim Leers

I’m afraid you’re still asking poor questions because you’ve not taken the time to properly inform yourself first. The very first link in the article refers to http://wimleers.com/article/key-properties-of-a-cdn. Please go and read that, if it still isn’t clear then, then please continue reading rather than asking.

Bill Riner's picture
Bill Riner

I set up a CloudFront CDN for a site and it works fine. I set a second CDN for a site and when logged in as admin, all pages that aren’t admin pages have no theme applied (e.g. the homepage). They are just a jumble of text and blank placeholders for images. This doesn’t happen to the first site.

Wim Leers's picture
Wim Leers

Please open a support request in the CDN module’s issue queue. I’m pretty sure that the CDN module didn’t cause this though, because I also use two CDN domains, without any problems :)

karamveer's picture
karamveer

Hello Support Team,

Please let me know is there any possibility to stop CDN (links) for a specific content type?

In my website there are some links (download) which are connected with CDN but I don’t want those linked to CDN.

Please suggest me what I have to do?

Thanks, Karamveer Singh

Wim Leers's picture
Wim Leers

Please create a new “Support Request” issue at https://www.drupal.org/project/issues/cdn :)

DH's picture
DH

Hello there, any guides for this module to work with CloudFlare ?

Wim Leers's picture
Wim Leers

CloudFlare is designed to sit in front of your entire site; this module is only designed to serve a site’s assets from a CDN. If you want to use CloudFlare but still serve assets from a different domain, then you can still use this module. But integration with CloudFlare is out of scope.

Suresh Kumar Golkonda's picture
Suresh Kumar Go...

Hi Wim, New images are returning with 404 from https://xxxxxx.cloudfront.net/sites/default/files/styles/article_feature/public/images/article/test3.jpeg?itok=_M-TWQ7b. but the actual file path is https://xxxxxx.cloudfront.net/sites/default/files/images/article/test3.jpeg?

BUT without CDN, if we access the site the new image is loading by dynamically generating the new path https://example.com/sites/default/files/styles/article_feature/public/images/article/test3.jpeg. Why CDN is not generating this new dynamic path?

Ahmad Nawaz's picture
Ahmad Nawaz

I followed your article but when I tried to open my site theming is disturbed. Can you help me what can be the reason so that I can debug and fix it?

Pages