Easy Drupal CDN integration for fun and profit
Speed up your Drupal site with a CDN in a few minutes.
The fun part: it’s nice to learn how to make any Drupal site significantly faster in a few minutes. The profit part: faster websites lead to more users and more revenue.
This article covers the common case: you have a small to medium size (≤1M page views per month), without massive amounts of large images, you’re using Drupal 71 and you only want to spend a few euros or (U.S.) dollars per month on a CDN. (You already know what a CDN is, right?)
So, you want your Drupal site to be faster, only spend a few minutes doing so, don’t want to deal with infrastructure and want to keep the costs very minimal. You’ve come to the right place.
Also: don’t worry about the cost: this little experiment will only cost you a few cents.
We’ll be using Amazon CloudFront and my CDN module for Drupal — I hope you like it. I’ve tried to make it as easy to use as possible. It’s solid, it’s got unit tests where appropriate, it’s used by ±2,000 Drupal sites — http://economist.com and http://worldpressphoto.org amongst others.
My largest deployment is for http://driverpacks.net, where the total CDN bill for well over half a million page views per month is less than USD $10!
Part 1: Create an Amazon CloudFront Distribution with a Custom Origin
We’ll be using Amazon’s CloudFront CDN service. Why?
They’re reliable, have solid performance (though not the best), are affordable (though not the cheapest) and are continuously expanding (i.e. adding more Points of Presence) around the globe. In just the last six months, they added five new edge locations2. So your site will automatically get faster in more locations around the globe, without paying more. Also see their list of edge locations or their map. Amazon is also cutting prices regularly (July 2011 was the last time). You’ll generally also never run into problems — after all, there are bigger fishes out there that help drive the infrastructure forward, you can just get an easy ride along.
You are of course free to use a different CDN, but then you’ll have to make sure you’re using an Origin Pull CDN.
So, go to https://aws.amazon.com/ and sign up for an account if you haven’t already. Go to the AWS Management Console and sign in. Go the CloudFront tab and click the “Create Distribution” in the top left corner.
Amazon recently updated their management console, and now (January 5, 2013) the steps below are outdated:
A wizard modal window will pop up (on the first page of the wizard: “Distribution Type”), where you can choose between two types of origins3: Amazon S3 Origin and Custom Origin. Choose Custom Origin; this means Amazon CloudFront’s edge location servers will come to your Drupal site’s web server and retrieve the files it needs to serve to end users. You won’t have to deal with Amazon S3 at all.
In the Origin DNS Name field, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Next: the Protocol Policy field. If your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click “Continue”.
Now you’re on the second page of the wizard (“Distribution Details”). You don’t need to change anything here. Click “Continue” again. We’re now on the “Review” page of the wizard. Click “Create Distribution”. You should get a message stating “You have successfully created a CloudFront Distribution.” Great!
The new steps are:
A 2-step wizard will appear. In the first step, you must select the delivery method: “Download” or “Streaming”. Choose “Download” (the default) and click “Continue”. In the second step, there are three groups of settings: “Origin Settings”, “Default Cache Behavior Settings” and “Distribution Settings”. We only care about the first group of settings, the others we can leave at the default (though you may want to change the “Price Class” setting in “Distribution Settings” to indicate that you only want to use U.S. & European edge locations, for example). So, “Origin Settings”: for “Origin Domain Name”, enter the domain name of your Drupal site: www.yoursite.com. In my case, I entered wimleers.com (my site is accessible both with and without the www). Now three additional settings will appear (because you’re using a custom origin and not Amazon S3): “Origin Protocol Policy”, “HTTP Port” and “HTTPS Port”. All of them have sane defaults. Only the “Origin Protocol Policy” setting (defaulted to HTTP Only, changeable to Match Viewer) is somewhat likely to be relevant to you. if your site is only accessible via HTTP and not via HTTPs, then just go with the default HTTP Only option. If your site supports https (or you want to support this in the future), then select Match Viewer. Finally, click “Continue”.
At the top of your CloudFront Management Console’s table of distributions, you should now see something like this:
Your newly created distribution’s status will remain at InProgress for a few more minutes, then it will change to Deployed. As soon as it is Deployed, we can actually use it.
If you have more special needs, consult Amazon’s documentation on creating distributions. More often than not, you won’t need that though.
Part 2: integrate your Drupal site with the CDN!
Download the CDN module for Drupal (version 2.5 or later). Install it like you install any other module. After the installation, go to admin/config/development/cdn (admin/settings/cdn on Drupal 6). There’s three tabs:
- General
- Details
- Other
We’ll cover them one by one. Note that you can install the Advanced Help module to get more and better help (it’ll help you explore all features of the CDN module).
First tab: “General”
There’s really only one important setting here: the status of the CDN module. You can either disable it, enable it, or put it in testing mode, which is somewhere in between. In testing mode, none of your visitors will get files from the CDN; only users with the access files on CDN when in testing mode permission will get to see it. This is perfect to test whether the CDN integration is actually working correctly. So, for now, let’s put it in testing mode.
The second (and last) setting on this tab is the Display statistics setting. Users with the access per-page statistics permission will get to see, well, per-page statistics at the bottom of each page: “what percentage of files is served from a CDN”, and so on. Enable this for now, so you get to see the results for each page on your site.
Second tab: “Details”
At the top of this tab, there’s a Mode setting, which allows you to choose between Origin Pull and File Conveyor. Choose Origin Pull (the default).
Next, there are mode-specific settings. The most important one is the “CDN mapping” setting. Here we define which files are mapped to which CDN (in case you’re using multiple CDNs — or static file servers).
Go back to the CloudFront Management Console and copy the domain name of your CloudFront Distribution. In my case: d67something714.cloudfront.net. Now paste this into the “CDN mapping” setting, but prepend it with http://. So your CDN mapping is now set to http://d67something714.cloudfront.net. This will cause all files to be served from CloudFront.
Read the included Advanced Help documentation to see what else is possible (e.g. serve images from a different CDN or only serve CSS and JS from a CDN), but for us (and for http://wimleers.com), this CDN mapping will do.
Optional, but recommended: Far Future expiration
The last setting on the “Details” tab is the Far Future expiration checkbox. This single checkbox has the capability to make your site much faster than when you were just using a CDN without this setting enabled. It can also reduce your CDN bill significantly.
What does it do? Well, it ensures that all files are served from the CDN in the most optimal way possible: compressed (gzipped) whenever possible and with the most optimal HTTP headers. These HTTP headers tell your visitors’ browsers to cache files “forever”. The results: less requests to the CDN (reducing your bill), and hence an even faster site! Whenever the file changes, its URL is changed automatically, so that your visitors don’t continue using that old buggy JS or CSS file forever, but they get the new files immediately.
There’s a catch though: to be able to set these HTTP headers and automatically4 change file URLs to be unique, we have to serve the files through PHP (Drupal) instead of letting the web server take care of it… This has adverse performance effects: using PHP to serve static files is not efficient!
However, the CDN caches these files for long periods of time (CloudFront edge locations come back to the origin once every 24 hours, no matter what expiration date you configure), so in fact that’s just fine. Hence you should only enable this setting if you’re using a CDN or a reverse proxy such as Varnish. It won’t work when you’re using “your own CDN”, i.e. just a static file server such as nginx or lighttpd. It will work when you’re using the same web server for these alternative domains as you’re using for serving the actual Drupal site, but in that case, very long load times are the result.
Again: no worries, the CDN module checks this automatically for you. After you submit the “Details” form, you should see a status message that confirms everything is a-ok:
Once you’ve enabled the Far Future expiration setting, a new setting appears: Unique file identifier generation. This defines how each file’s unique file identifiers are generated. The default works fine for all sites, but for complex and/or high-traffic sites, you may want to fine-tune this.
Third tab: “Other”
We won’t go into the details of the various settings on this tab, but there’s only one we care about for the scope of this article: the CDN supports HTTPS setting. If your site is using HTTPS and you configured your CloudFront distribution’s Protocol Policy to Match Viewer, then you can enable this setting as well. Whenever your site is accessed through HTTPS, your files will still be served from the CDN, via HTTPS!
All done!
If you haven’t already, make sure you’ve enabled Drupal’s CSS aggregation, block caching and page caching. The CDN module automatically rewrites image URLs in blocks and nodes, but we don’t want that to happen on every page load if it’s not necessary — hence enable those caching layers.
By now, the status of your CloudFront distribution should have changed to “Deployed”. So let’s give this a try! The following screenshot was taken at http://wimleers.com/blog/facebook-week-12:
If everything is looking good after browsing through your website and confirming that the CDN integration is working correctly everywhere, head back to the “General” tab and change the status from Testing mode to Enabled. Now your site’s actual visitors will also get all files served from the CDN, so they should be experiencing a faster site! Definitely return visits should be significantly faster. Your status report should look similar to this:
Congratulations, your site is now accelerated by a CDN!
-
This will work for Drupal 6, too. ↩︎
-
October 2011: Sao Paulo, Brazil. December 2011: South Bend, Indiana, U.S.A; San Jose, California, U.S.A.; second edge location in New York City, New York, U.S.A. February 2012: Osaka, Japan; Milan Italy. ↩︎
-
The “origin” defines what the “origin server” is, i.e. where CloudFront edge location servers around the world will go to get the files they need to serve to your site’s visitors.
Also see “The Origin Server” in Amazon’s documentation. ↩︎ -
“Automatically” as in “out of the box”; without modifying Apache’s
.htaccessorhttpd.conffiles and similar web server config files for other web server software. ↩︎
Comments
Hi,
Don’t forget to adapt
sites.php:$sites['www.******.com'] = 'default'; $sites['d1s*******.cloudfront.net'] = 'default';Actually, no, that’s not necessary at all :)
You see, this is only designed to serve static files, not the entire website. Hence it’s not necessary.
Okay.
However, I followed your tutorial but it didn’t work without my changes :)
P.S.: Congratulations for your tutorial!
It should work fine without those changes. In fact, it doesn’t make any sense that those changes help!
Great write-up and great module. Goes straight to the top position of modules to check out!
Installed it, with a CNAME so it looks prettier. It works like a charm!
http://www.kdv-vergelijker.nl
Any instruction to set up CNAME ?? I may need it but no idea about CNAME
A quick internet search yields many tutorials :) ;)
Awesome! :)
Note that your Drupal theme appears to have some hardcoded references to CSS files. You may want to pass them through
file_create_url(), so that they are served from the CDN as well. There’s just been an issue for that as well — that should help you further.Thank you so much but maybe that issue is talking about missing from CDN, instead of link the node to CDN?
or I may misunderstand…….:(
Thanks for the tip! Will look into that. The theme has to be redone anyways (and the upgrade to D7 too!)
Good article to use CDN on Drupal! I followed this to set up my CDN.
BTW, I use a View to show an image on the frontpage, the image links to the corresponding node. However, when I click the image, it linked to http://xxxxxxx.cloudfront.net/node/0000 instead of mydomain.com/node/0000
How can I fix this?
Thank you so much.
Interesting problem. Typically, thumbnails link to the “full images”, i.e. to original image file. You’re instead linking to the “original node”.
The current behavior is hardcoded, so there’s no easy way around this at the moment. Please open a new issue at http://drupal.org/project/issues/cdn.
Thanks for your reply. I did that, but some of them just don’t work that way I want…
I will go open a new issue. Thanks again!
The easiest solution is to just link to the full image instead of the node :)
Wim,
I followed this article to setup a CloudFront CDN Distribution and incorporate the CDN module into our D6 website. Our primary web server is located in Dallas, TX. Before starting this project, our “first view” page load time for Singapore was almost 5 seconds. Now we are seeing 2.1 seconds consistently as you can see with the following link:
http://www.webpagetest.org/result/120501_Z2_46826/
The resulting experience for our users is wonderful – I can’t imagine putting up with a 5 second page load time for long!
The only module we added to achieve these results is CDN module configured with: the core patch, far future expiration, origin pull and the default module settings. This is quite a testimony to your efforts Wim!!
The amazing thing is you can have a dramatic impact on your website user experience and only invest an hour of your time. I’ve been looking at the results of YSlow and similar tools for years that contain poor “grades” for performance. Now due to Wim’s efforts with CDN module, we have straight “A”s when we run YSlow. Needless to say, the technique described above is Highly Recommended!!
I can’t thank you enough Wim, Greg
PS. The results from webpagetest.org may be deleted over time. If this is the case, please feel free to apply the tool to http://www.RayStedman.org/new-testament/ephesians and the Singapore test site using IE8. You will see the resulting page load times, waterfall and connect timing charts with the CDN improvements in place. Great stuff!!
Awesome! It’s great to see such massive improvements! :)
Thanks for this very clear tutorial Wim. I’m more a marketeer than a developer, but configuring the CDN module by following your steps was a breeze! I only needed some developer help with installing the patch and I was a little confused about the Far Future Expiration. Other than that: it just works! Finally we were able to make our site faster. Feel free to check it out: www.thepickwickproject.be
That’s great to hear! :) Sounds like I’ve reached my goal then :)
Thank you for this wonderful product. My website scores are amazing now on tools.pingdom.com. I have a question though. I followed this tutorial to the letter but I am getting a warning when I’m on the details tab after saving:
Potentially problematic domains: images.herbstprodukt.com uses the same web server as this Drupal site.Any ideas on how to make this warning go away?
Can you comment on ways to improve performance even more? For example, on this site there are lots of images which makes the first page load pretty slow. Is there away to alternate cdn’s so jpg images could be downloaded from two cdn’s at the same time.
Thank you for again for such a marvelous product!
– In case anyone is wondering, I am using a cname as one of the commentors above recommended. (Thanks for that nice idea!).
You’re seeing that message because
images.yourdomain.comis actually just a subdomain that serves the exact same site (; hence it says the same web server is being used. In other words: you’re using aCNAME, as you already said yourself. That means you cannot just enable the Far Future expiration.Yes, you can balance files over multiple domains/CDNs automatically. Do that as follows. Instead of having the following line in your mapping:
change it to:
That’ll do the trick :) Or just consult the Advanced Help docs for the “CDN mapping” setting, there it’s explained in detail!
I’m actually having the same problem in 7.x-2.x, and I’m not using a CNAME yet. When I use the cloudfront domain and save the “details” form, cdn_admin_details_form_validate() hits ‘cdn/farfuture/reverse-proxy-test’ two times on the Cloudfront domain.
I don’t know why the results aren’t identical for both requests.
Strange. Please report this in the issue queue and we’ll figure it out :)
Hello there I am so happy I found your weblog, I really found you by accident, while I was searching on Digg for something else, Anyhow I am here now and would just like to say cheers for a tremendous post and a all round interesting blog (I also love the theme/design), I don’t have time to go through it all at the minute but I have book-marked it and also added in your RSS feeds, so when I have time I will be back to read much more, Please do keep up the great jo.
Hi, I’m using a similar CNAME method. I have actually images.yourdomain.com and images2.yourdomain.com to serve files and it seems to work fine.
The trouble is that if I access images.yourdomain.com for example I get the full site. Is there a way to have these subdomains ‘not working’ except to serve the static content? I suspect having these ‘cloned’ sites is not good for SEO nor do I want them to be accessed that way. Any help on configuring this is greatly appreciated.
Yes, that is possible — please see See http://drupal.org/node/1060358.
Hmm, are you referring to the patch found in http://drupal.org/node/1060358#comment-6676810? There’s mentions earlier that this code doesn’t work with image styles. 95% of my images are using that so does it mean this won’t help me that much at all?
It should be working fine now. But in any case this is not the right place to discuss such a feature. The issue I pointed you to is.
I followed your steps and the site seems to work. However when I checked the log, a lot of message like the following were generated:
“Nested HTTP request to generate read-write: failure (URL: http://engworld.org/read-write, time: 1337962668).”
Do you know why?
Please report problems in the issue queue.
I like the valuable information you provide in your articles. I will bookmark your weblog and check again here frequently. I am quite sure I’ll learn lots of new stuff right here! Best of luck for the next!
Hi Mister Wim Leers,
I’m plannin to use CDN for my drupal website and, of course, I’m going to use your tutorial for the first configuration, so then I have time to get more familiar with CDN and customize it around my needs. I’m using Drupal 7 and in some
node.tpl.phpI use$GLOBALS['base_url']/sites/default/files/podcast/myodcast.mp3or$GLOBALS['base_url']/sites/all/themes/mytheme/images/blabla.jpg. Now I would like to know if I need to continue to use$GLOBALS['base_url']or I need to use a new variable for the CDN. If I need a new variable with more configuration, could you please explain it to me or suggest me some tutorial.Thanks for your Drupal contribution.
If you use
file_create_url(), it’ll switch to using the CDN whenever you’ve activated it :)Thanks for your answer, do I need any patch to make it work?
No, as you can see in the link: it’s part of Drupal 7 core.
Brilliant! Thanks a lot!
Which one do you think is faster and saves performance for your server? Origin Pull or File Conveyor? Is there a guide for File Conveyor?
If you have to ask that question, I recommend using Origin Pull.
Hi, I do believe this is an excellent site. I stumbledupon it ;) I may return yet again since I book-marked it. Money and freedom is the greatest way to change, may you be rich and continue to help others.
Hi Wim, excellent instructions :) But…
I am one of those “cheap” guys who want to create a fake CDN using subdomains and I tried doing that using a few different instructions I found online but with no success. Here is what I tried: I created 3 subdomains css.mysite, js.mysite img.mysite and placed them inside mysite CDN settings exactly as desribed http://runeforge.net/tutorial/utilizing-an-fake-cdn-to-boost-drupals-pag… but no joy. CDN on mysite seems to be working and “serving” files from the three subdomains. Or rather it thinks that it is serving files from there because the files are just not there.
Do I need to install something inside those subdomain folders that will pull the files from mysite. or whats the score? I know that when using fake CDN the gains are not as great as they would be from a real CDN but for now I just want to test this and perhaps upgrade to a real CDN service down the track. Can you please post detailed dummy proof instructions on how to setup a fake CDN on shared hosting like godaddy or similar for us poor people :)
I still need to write a tutorial much like the one we’re commenting on for that use case. For now, please follow the relevant issue.
On June 19, Amazon added an edge location in Sydney, Australia.
Hi Wim, thanks for your response. I actually tried amazon cloudfront but I received the “potentially problematic domains” error for cloudfront domain…..Thats without any CNAME setup
The above wasn’t a reply to your comment, it was a separate comment.
Please report the issue you’re having in the issue queue, that’s the appropriate place for questions like these!
Very good article, however I am interested in CDN with sub-domains for my health site. I request you if possible can you please write a similar tutorial to help people like me.
Correct; please see the relevant issue on Drupal.org.
Thank you very much for your link, however my site is as http://example.com instead of http://www.example.com and CNAME setting and enabling CDN module generates errors.
Whether it is possible to serve static content from non www domain’s subdomain, please advice.
CNAMEs only apply to subdomains by definition.
The only way to serve CDN traffic from your main (non-sub)domain, is by pointing it directly to the CDN. But that’s only feasible for very advanced set-ups (at least today).
On Sep 12, 2012, Amazon added an edge location in Madrid, Spain.
When I create a distribution I am presented with fields for “Origin Domain Name” and “Origin ID”. I understand Origin Domain Name is my domain name but what is Origin ID?
Pages