Easy Drupal CDN integration for fun and profit

Published on 24 April, 2012

Speed up your Drupal site with a CDN in a few minutes.

The fun part: it’s nice to learn how to make any Drupal site significantly faster in a few minutes. The profit part: faster websites lead to more users and more revenue.

This article covers the common case: you have a small to medium size (≀1M page views per month), without massive amounts of large images, you’re using Drupal 71 and you only want to spend a few euros or (U.S.) dollars per month on a CDN. (You already know what a CDN is, right?)
So, you want your Drupal site to be faster, only spend a few minutes doing so, don’t want to deal with infrastructure and want to keep the costs very minimal. You’ve come to the right place.

Also: don’t worry about the cost: this little experiment will only cost you a few cents.

We’ll be using Amazon CloudFront and my CDN module for Drupal — I hope you like it. I’ve tried to make it as easy to use as possible. It’s solid, it’s got unit tests where appropriate, it’s used by ±2,000 Drupal sites — http://economist.com and http://worldpressphoto.org amongst others.
My largest deployment is for http://driverpacks.net, where the total CDN bill for well over half a million page views per month is less than USD $10!

Part 1: Create an Amazon CloudFront Distribution with a Custom Origin

We’ll be using Amazon’s CloudFront CDN service. Why?
They’re reliable, have solid performance (though not the best), are affordable (though not the cheapest) and are continuously expanding (i.e. adding more Points of Presence) around the globe. In just the last six months, they added five new edge locations2. So your site will automatically get faster in more locations around the globe, without paying more. Also see their list of edge locations or their map. Amazon is also cutting prices regularly (July 2011 was the last time). You’ll generally also never run into problems — after all, there are bigger fishes out there that help drive the infrastructure forward, you can just get an easy ride along.

You are of course free to use a different CDN, but then you’ll have to make sure you’re using an Origin Pull CDN.

So, go to https://aws.amazon.com/ and sign up for an account if you haven’t already. Go to the AWS Management Console and sign in. Go the CloudFront tab and click the “Create Distribution” in the top left corner.

Amazon recently updated their management console, and now (January 5, 2013) the steps below are outdated: A wizard modal window will pop up (on the first page of the wizard: “Distribution Type”), where you can choose between two types of origins3: Amazon S3 Origin and Custom Origin. Choose Custom Origin; this means Amazon CloudFront’s edge location servers will come to your Drupal site’s web server and retrieve the files it needs to serve to end users. You won’t have to deal with Amazon S3 at all.
In the Origin DNS Name field, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Next: the Protocol Policy field. If your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click “Continue”.

Now you’re on the second page of the wizard (“Distribution Details”). You don’t need to change anything here. Click “Continue” again. We’re now on the “Review” page of the wizard. Click “Create Distribution”. You should get a message stating “You have successfully created a CloudFront Distribution.” Great!

The new steps are: A 2-step wizard will appear. In the first step, you must select the delivery method: “Download” or “Streaming”. Choose “Download” (the default) and click “Continue”. In the second step, there are three groups of settings: “Origin Settings”, “Default Cache Behavior Settings” and “Distribution Settings”. We only care about the first group of settings, the others we can leave at the default (though you may want to change the “Price Class” setting in “Distribution Settings” to indicate that you only want to use U.S. & European edge locations, for example). So, “Origin Settings”: for “Origin Domain Name”, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Now three additional settings will appear (because you’re using a custom origin and not Amazon S3): “Origin Protocol Policy”, “HTTP Port” and “HTTPS Port”. All of them have sane defaults. Only the “Origin Protocol Policy” setting (defaulted to HTTP Only, changeable to Match Viewer) is somewhat likely to be relevant to you. if your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click â€œContinue”.

At the top of your CloudFront Management Console’s table of distributions, you should now see something like this:

CloudFront Distribution created, status 'InProgress'.

Your newly created distribution’s status will remain at InProgress for a few more minutes, then it will change to Deployed. As soon as it is Deployed, we can actually use it.

If you have more special needs, consult Amazon’s documentation on creating distributions. More often than not, you won’t need that though.

Part 2: integrate your Drupal site with the CDN!

Download the CDN module for Drupal (version 2.5 or later). Install it like you install any other module. After the installation, go to admin/config/development/cdn (admin/settings/cdn on Drupal 6). There’s three tabs:

  1. General
  2. Details
  3. Other

We’ll cover them one by one. Note that you can install the Advanced Help module to get more and better help (it’ll help you explore all features of the CDN module).

First tab: â€œGeneral”

There’s really only one important setting here: the status of the CDN module. You can either disable it, enable it, or put it in testing mode, which is somewhere in between. In testing mode, none of your visitors will get files from the CDN; only users with the access files on CDN when in testing mode permission will get to see it. This is perfect to test whether the CDN integration is actually working correctly. So, for now, let’s put it in testing mode.

The second (and last) setting on this tab is the Display statistics setting. Users with the access per-page statistics permission will get to see, well, per-page statistics at the bottom of each page: “what percentage of files is served from a CDN”, and so on. Enable this for now, so you get to see the results for each page on your site.

CDN settings: 'General' tab

Second tab: â€œDetails”

At the top of this tab, there’s a Mode setting, which allows you to choose between Origin Pull and File Conveyor. Choose Origin Pull (the default).

Next, there are mode-specific settings. The most important one is the “CDN mapping” setting. Here we define which files are mapped to which CDN (in case you’re using multiple CDNs — or static file servers).
Go back to the CloudFront Management Console and copy the domain name of your CloudFront Distribution. In my case: d67something714.cloudfront.net. Now paste this into the “CDN mapping” setting, but prepend it with http://. So your CDN mapping is now set to http://d67something714.cloudfront.net. This will cause all files to be served from CloudFront.

Read the included Advanced Help documentation to see what else is possible (e.g. serve images from a different CDN or only serve CSS and JS from a CDN), but for us (and for http://wimleers.com), this CDN mapping will do.

Optional, but recommended: Far Future expiration

The last setting on the “Details” tab is the Far Future expiration checkbox. This single checkbox has the capability to make your site much faster than when you were just using a CDN without this setting enabled. It can also reduce your CDN bill significantly.

What does it do? Well, it ensures that all files are served from the CDN in the most optimal way possible: compressed (gzipped) whenever possible and with the most optimal HTTP headers. These HTTP headers tell your visitors’ browsers to cache files “forever”. The results: less requests to the CDN (reducing your bill), and hence an even faster site! Whenever the file changes, its URL is changed automatically, so that your visitors don’t continue using that old buggy JS or CSS file forever, but they get the new files immediately.

There’s a catch though: to be able to set these HTTP headers and automatically4 change file URLs to be unique, we have to serve the files through PHP (Drupal) instead of letting the web server take care of it
 This has adverse performance effects: using PHP to serve static files is not efficient!
However, the CDN caches these files for long periods of time (CloudFront edge locations come back to the origin once every 24 hours, no matter what expiration date you configure), so in fact that’s just fine. Hence you should only enable this setting if you’re using a CDN or a reverse proxy such as Varnish. It won’t work when you’re using “your own CDN”, i.e. just a static file server such as nginx or lighttpd. It will work when you’re using the same web server for these alternative domains as you’re using for serving the actual Drupal site, but in that case, very long load times are the result.
Again: no worries, the CDN module checks this automatically for you. After you submit the “Details” form, you should see a status message that confirms everything is a-ok:

Far Future expiration — domain check

Once you’ve enabled the Far Future expiration setting, a new setting appears: Unique file identifier generation. This defines how each file’s unique file identifiers are generated. The default works fine for all sites, but for complex and/or high-traffic sites, you may want to fine-tune this.

CDN settings: 'Details' tab

Third tab: â€œOther”

We won’t go into the details of the various settings on this tab, but there’s only one we care about for the scope of this article: the CDN supports HTTPS setting. If your site is using HTTPS and you configured your CloudFront distribution’s Protocol Policy to Match Viewer, then you can enable this setting as well. Whenever your site is accessed through HTTPS, your files will still be served from the CDN, via HTTPS!

CDN settings: 'Other' tab

All done!

If you haven’t already, make sure you’ve enabled Drupal’s CSS aggregation, block caching and page caching. The CDN module automatically rewrites image URLs in blocks and nodes, but we don’t want that to happen on every page load if it’s not necessary — hence enable those caching layers.

By now, the status of your CloudFront distribution should have changed to “Deployed”. So let’s give this a try! The following screenshot was taken at http://wimleers.com/blog/facebook-week-12:

Per-page statistics for wimleers.com

If everything is looking good after browsing through your website and confirming that the CDN integration is working correctly everywhere, head back to the “General” tab and change the status from Testing mode to Enabled. Now your site’s actual visitors will also get all files served from the CDN, so they should be experiencing a faster site! Definitely return visits should be significantly faster. Your status report should look similar to this:

Final status report

Congratulations, your site is now accelerated by a CDN!


  1. This will work for Drupal 6, too. â†©

  2. October 2011: Sao Paulo, Brazil. December 2011: South Bend, Indiana, U.S.A; San Jose, California, U.S.A.; second edge location in New York City, New York, U.S.A. February 2012: Osaka, Japan; Milan Italy. â†©

  3. The “origin” defines what the “origin server” is, i.e. where CloudFront edge location servers around the world will go to get the files they need to serve to your site’s visitors.
    Also see “The Origin Server” in Amazon’s documentation. â†©

  4. “Automatically” as in “out of the box”; without modifying Apache’s .htaccess or httpd.conf files and similar web server config files for other web server software. â†©

Arthur

12 years 8 months ago

Hi,

Don’t forget to adapt sites.php: $sites['www..com'] = 'default'; $sites['d1s*.cloudfront.net'] = 'default';

Wim

12 years 8 months ago

In reply to by Arthur

Actually, no, that’s not necessary at all :)

You see, this is only designed to serve static files, not the entire website. Hence it’s not necessary.

Arthur

12 years 8 months ago

In reply to by Arthur

Okay.

However, I followed your tutorial but it didn’t work without my changes :)

P.S.: Congratulations for your tutorial!

Wim

12 years 8 months ago

In reply to by Arthur

It should work fine without those changes. In fact, it doesn’t make any sense that those changes help!

Awesome! :)

Note that your Drupal theme appears to have some hardcoded references to CSS files. You may want to pass them through file_create_url(), so that they are served from the CDN as well. There’s just been an issue for that as well — that should help you further.

Thank you so much but maybe that issue is talking about missing from CDN, instead of link the node to CDN?

or I may misunderstand

.:(

AppleLife

12 years 8 months ago

Good article to use CDN on Drupal! I followed this to set up my CDN.

BTW, I use a View to show an image on the frontpage, the image links to the corresponding node. However, when I click the image, it linked to http://xxxxxxx.cloudfront.net/node/0000 instead of mydomain.com/node/0000

How can I fix this?

Thank you so much.

Interesting problem. Typically, thumbnails link to the “full images”, i.e. to original image file. You’re instead linking to the “original node”.

The current behavior is hardcoded, so there’s no easy way around this at the moment. Please open a new issue at http://drupal.org/project/issues/cdn.

Wim,

I followed this article to setup a CloudFront CDN Distribution and incorporate the CDN module into our D6 website. Our primary web server is located in Dallas, TX. Before starting this project, our “first view” page load time for Singapore was almost 5 seconds. Now we are seeing 2.1 seconds consistently as you can see with the following link:

http://www.webpagetest.org/result/120501_Z2_46826/

The resulting experience for our users is wonderful – I can’t imagine putting up with a 5 second page load time for long!

The only module we added to achieve these results is CDN module configured with: the core patch, far future expiration, origin pull and the default module settings. This is quite a testimony to your efforts Wim!!

The amazing thing is you can have a dramatic impact on your website user experience and only invest an hour of your time. I’ve been looking at the results of YSlow and similar tools for years that contain poor “grades” for performance. Now due to Wim’s efforts with CDN module, we have straight “A”s when we run YSlow. Needless to say, the technique described above is Highly Recommended!!

I can’t thank you enough Wim, Greg

PS. The results from webpagetest.org may be deleted over time. If this is the case, please feel free to apply the tool to http://www.RayStedman.org/new-testament/ephesians and the Singapore test site using IE8. You will see the resulting page load times, waterfall and connect timing charts with the CDN improvements in place. Great stuff!!

Thanks for this very clear tutorial Wim. I’m more a marketeer than a developer, but configuring the CDN module by following your steps was a breeze! I only needed some developer help with installing the patch and I was a little confused about the Far Future Expiration. Other than that: it just works! Finally we were able to make our site faster. Feel free to check it out: www.thepickwickproject.be

Erik Halber

12 years 7 months ago

Thank you for this wonderful product. My website scores are amazing now on tools.pingdom.com. I have a question though. I followed this tutorial to the letter but I am getting a warning when I’m on the details tab after saving:

Potentially problematic domains: images.herbstprodukt.com uses the same web server as this Drupal site.

Any ideas on how to make this warning go away?

Can you comment on ways to improve performance even more? For example, on this site there are lots of images which makes the first page load pretty slow. Is there away to alternate cdn’s so jpg images could be downloaded from two cdn’s at the same time.

Thank you for again for such a marvelous product!

– In case anyone is wondering, I am using a cname as one of the commentors above recommended. (Thanks for that nice idea!).

You’re seeing that message because images.yourdomain.com is actually just a subdomain that serves the exact same site (; hence it says the same web server is being used. In other words: you’re using a CNAME, as you already said yourself. That means you cannot just enable the Far Future expiration.

Yes, you can balance files over multiple domains/CDNs automatically. Do that as follows. Instead of having the following line in your mapping:

cdn1|jpg jpeg

change it to:

cdn1 cdn2 cdn3|jpg jpeg

That’ll do the trick :) Or just consult the Advanced Help docs for the “CDN mapping” setting, there it’s explained in detail!

I’m actually having the same problem in 7.x-2.x, and I’m not using a CNAME yet. When I use the cloudfront domain and save the “details” form, cdn_admin_details_form_validate() hits ‘cdn/farfuture/reverse-proxy-test’ two times on the Cloudfront domain.

I don’t know why the results aren’t identical for both requests.

Hello there I am so happy I found your weblog, I really found you by accident, while I was searching on Digg for something else, Anyhow I am here now and would just like to say cheers for a tremendous post and a all round interesting blog (I also love the theme/design), I don’t have time to go through it all at the minute but I have book-marked it and also added in your RSS feeds, so when I have time I will be back to read much more, Please do keep up the great jo.

Hi, I’m using a similar CNAME method. I have actually images.yourdomain.com and images2.yourdomain.com to serve files and it seems to work fine.

The trouble is that if I access images.yourdomain.com for example I get the full site. Is there a way to have these subdomains ‘not working’ except to serve the static content? I suspect having these ‘cloned’ sites is not good for SEO nor do I want them to be accessed that way. Any help on configuring this is greatly appreciated.

Syd Barrett

12 years 7 months ago

Hi Mister Wim Leers,

I’m plannin to use CDN for my drupal website and, of course, I’m going to use your tutorial for the first configuration, so then I have time to get more familiar with CDN and customize it around my needs. I’m using Drupal 7 and in some node.tpl.php I use $GLOBALS['base_url']/sites/default/files/podcast/myodcast.mp3 or $GLOBALS['base_url']/sites/all/themes/mytheme/images/blabla.jpg. Now I would like to know if I need to continue to use $GLOBALS['base_url'] or I need to use a new variable for the CDN. If I need a new variable with more configuration, could you please explain it to me or suggest me some tutorial.

Thanks for your Drupal contribution.

Anonymous

12 years 6 months ago

Which one do you think is faster and saves performance for your server? Origin Pull or File Conveyor? Is there a guide for File Conveyor?

Mario Baron

12 years 6 months ago

Hi Wim, excellent instructions :) But


I am one of those “cheap” guys who want to create a fake CDN using subdomains and I tried doing that using a few different instructions I found online but with no success. Here is what I tried: I created 3 subdomains css.mysite, js.mysite img.mysite and placed them inside mysite CDN settings exactly as desribed http://runeforge.net/tutorial/utilizing-an-fake-cdn-to-boost-drupals-pa
 but no joy. CDN on mysite seems to be working and “serving” files from the three subdomains. Or rather it thinks that it is serving files from there because the files are just not there.

Do I need to install something inside those subdomain folders that will pull the files from mysite. or whats the score? I know that when using fake CDN the gains are not as great as they would be from a real CDN but for now I just want to test this and perhaps upgrade to a real CDN service down the track. Can you please post detailed dummy proof instructions on how to setup a fake CDN on shared hosting like godaddy or similar for us poor people :)

Hi Wim, thanks for your response. I actually tried amazon cloudfront but I received the “potentially problematic domains” error for cloudfront domain
..Thats without any CNAME setup

Wim

12 years 6 months ago

In reply to by Mario Baron

The above wasn’t a reply to your comment, it was a separate comment.

Please report the issue you’re having in the issue queue, that’s the appropriate place for questions like these!

Health Conditions

12 years 5 months ago

Very good article, however I am interested in CDN with sub-domains for my health site. I request you if possible can you please write a similar tutorial to help people like me.

Kevin

12 years 3 months ago

When I create a distribution I am presented with fields for “Origin Domain Name” and “Origin ID”. I understand Origin Domain Name is my domain name but what is Origin ID?

From the article:

If you have more special needs, consult Amazon’s documentation on creating distributions. More often than not, you won’t need that though.

In there, you’ll see this:

Origin ID: A string that will help you distinguish this origin from other origins in this distribution. If you add more origins and cache behaviors to the distribution later in the process, you will link origins with cache behaviors using the origin ID.

I think that answers your question? :)

Erik Halber

12 years 1 month ago

Hi again, I have a new issue. I just launched the following site: athenadoctrine.com. If you scroll over the images in the grid, you’ll notice that all the urls being with cdn-1, 2 or 3. The links are being generated via a view which I’ve done some extensive rewriting off.

Can you tell me why it’s using cdn-N in the beginning of all of these urls instead of the default www?

Thanks!

I’ve looked a little more into it and it’s really weird. I am wrapping the title of the article in an anchor whose value is the path to the content. I am wrapping the image with the same token but it’s rendered value uses the cdn-N subdomain. (I have also unchecked the link this image to content checkbox in the image field).

Erik Halber

12 years 1 month ago

Ok, it’s solved, but it feels hackish. I ended up just wrapping the tag in a span.

Previous markup that didn’t work [a][img][/a] - this creates the cdn-N subdomain

Markup that works: [a][span][img][/span][/a]

Is this an issue/bug with the CDN module or views?

Hey Wil,

I am trying to setup a Custom Origin but this doesn’t appear to be an option any more - I can only select S3 buckets. As you suggest in the Article I shouldn’t be doing this I am not sure how to proceed - do you have an up to date guide on getting this up and running.

Thanks, Paul.

Hi Paul,

First of all: my name is Wim, not Wil. Both derive from “William” (or rather, mine derives from the German “Wilhelm”), but they really are different ;)

Note that Amazon AWS still supports this. It’s just slightly different. I updated the article to explain how you can do it today.

Huge apologies Wim, bad enough on d.o and I did it on your own site :(.

Thank you for the update, I am progressing through it now.

Kind Regards, Paul.

Pierre

12 years ago

Perfect! Thanks a lot! i implemented it at Appsta. Just to avoid duplicate between cdn subdomain and main domain, i added this to settings.php :

<?php if($_SERVER[‘HTTP_USER_AGENT’] == “Amazon CloudFront”) { $file_dir = “sites/default/files”; if (strpos($_GET[‘q’], $file_dir) { } else{ drupal_add_http_header(‘Status’, ‘403 Forbidden’); print ‘403 ForbiddenNot Allowed

You cannot access website via CDN-url.

’; exit; } }  ?>

It prevents your website from being reached from cdn.example.com or from the cloudfront url, except for files under default folder.

Jonathan

11 years 10 months ago

Hi Wim,

Thanks for the great tutorial. Before I start getting stuck in, I was wondering how this works for imagecache presets and stuff? I will be using this on a website that utilises different presets for different views etc.

Would be interesting to find out how to make best use of CDN using imagecache. Apologies if this has been answered previously, I couldn’t see it in the comments list.

Many thanks for your time.

Wim

11 years 10 months ago

In reply to by Jonathan

When the Far Future setting is disabled, nothing special happens: the CDN will hit the origin to retrieve the file, and by doing so, it will cause the file to be generated if it does not already exist.

When the Far Future setting is enabled, the CDN module will perform a HTTP request to generate the file (from within the PHP), so that it can serve the image directly. It needs to do this to be able to generate the UFI.

Thanks, I appreciate that. Sounds like Imagecache presets are no different then and from your advice, I assume it’ll all work as normal.

Appreciate your time and efforts, thank you! :)

stephen

11 years 6 months ago

Hi Wim

I’m obviously doing something wrong as I’ve followed all the instructions, it works - as such - but for some reason I always lose my front-end styles (using Bartik), this happens in either ‘testing mode’ or ‘Enabled ). Please note that ‘admin’ styles are not lost. Is there anything you can think of that may cause this? I have filed an issue at https://drupal.org/node/2029781

Thank you

Stephen

Sam

11 years 6 months ago

Hello Wim, Thanks for the Nice Tutorial. Can you please let me know where to set up the expiration time. My website has a dynamically changing content every day(It is a news Portal). May be about 140 nodes being created each day.Is it good to have a CDN enabled for a dynamic site?

Wim

11 years 6 months ago

In reply to by Sam

I apparently don’t state it explicitly in the article, but: that’s not what the CDN module does.

The CDN module only makes it possible to serve files (images, videos, CSS, JS, etc.) from a CDN, not the actual HTML pages themselves.

Thanks for the response Wim. Can you please let me know How often does CDN fetch from website to serve files (images, videos, CSS, JS, etc.). I upload youtube videos very often daily, so the thumbnails are created very often too. Is it dependent on Cron Run or Caching lifetime specified.

Bill Riner

10 years 11 months ago

I set up a CloudFront CDN for a site and it works fine. I set a second CDN for a site and when logged in as admin, all pages that aren’t admin pages have no theme applied (e.g. the homepage). They are just a jumble of text and blank placeholders for images. This doesn’t happen to the first site.

karamveer

10 years 8 months ago

Hello Support Team,

Please let me know is there any possibility to stop CDN (links) for a specific content type?

In my website there are some links (download) which are connected with CDN but I don’t want those linked to CDN.

Please suggest me what I have to do?

Thanks, Karamveer Singh

CloudFlare is designed to sit in front of your entire site; this module is only designed to serve a site’s assets from a CDN. If you want to use CloudFlare but still serve assets from a different domain, then you can still use this module. But integration with CloudFlare is out of scope.

Suresh Kumar G


6 years 6 months ago

Hi Wim, New images are returning with 404 from https://xxxxxx.cloudfront.net/sites/default/files/styles/article_feature/public/images/article/test3.jpeg?itok=_M-TWQ7b. but the actual file path is https://xxxxxx.cloudfront.net/sites/default/files/images/article/test3.jpeg?

BUT without CDN, if we access the site the new image is loading by dynamically generating the new path https://example.com/sites/default/files/styles/article_feature/public/images/article/test3.jpeg. Why CDN is not generating this new dynamic path?

Ahmad Nawaz

6 years 5 months ago

I followed your article but when I tried to open my site theming is disturbed. Can you help me what can be the reason so that I can debug and fix it?