Symphony.

I’ve recently found myself looking at ways to optimize my sites and make them run faster and one of the things I’ve found is that browsers can only download certain files from a domain one at a time and that they way around this is to serve your images from a sub-domain. In theory you can carry this pretty far and serve all your javascript, css, images, video and so forth all from different sub-domains.

Overboard, I know but it’s a thought.

So my question becomes one of would there be a way to incorporate this into Symphony somehow? Where images and so forth were automatically served from the sub-domain http://media.yourwebsite.com/ to help speed up site load times?

You should be able to do this using an .htaccess in each subdomain root. Or you might be able to do it directly in your Apache Hosts configuration. Either way, you’d be pointing to maindomain.com/js, etc. Interesting idea, seems like it could seriously improve pageload.

Speaking of taking it overboard, imagine having a serverside calculation of the volume of all the resource on a page, then balancing the load by modifying resource urls on the page to point to different subdomains all aliasing to the same group of resources. Madness!

I think the solution depends on the “domain construction”, using different folders or just one for your main domain and your subdomains. You may find some useful hints in my Ninja Domains article, especially regarding symbolic links. The htaccess rules described there are pre-2.0.6 rules, but should be easy to adopt for Symphony 2.0.6.

I did something similar about a year and a half ago, and found using the htaccess file was the best way to accomplish this. Simliar to michael-e’s article.

On a side note take a look at Steven Souders website. He has written a lot on the subject of speeding up websites by using better UI development practices – including two books and the original YSlow Firebug extension. what dougoftheabaci is mentioning is one of his core pieces of research.

I’ve found is that browsers can only download certain files from a domain one at a time and that they way around this is to serve your images from a sub-domain.

Are you sure this is the reason for subdomains? I thought you’d put a heavy Apache on your primary domain(s), serving the content, and several proxies or Lighttpds on your subdomains, serving the images.

Are you sure this is the reason for subdomains?

Absolutely — the performance benefits are enough for the big guns to implement this (refer to Steve Souders’ work for rationale, his High Performance Websites book is excellent). It’s why eBay loads images off a domain something like “ebayimages.com” or something.

We sometimes use Amazon S3 to host large media assets (JPEGs and FLVs) and use an .htaccess rule to remap their original location to subdomained location elsewhere. Not only does it allow more simultaneous HTTP requests during the page load, but you have the added benefit of faster downloading of the assets from the CDN.

One thing to be aware of with the approach Nick describes is that there is a DNS lookup time associated with the secondary domains. This can add a second or so to the page load time, so you have to weigh up the benefits of using a rewrite rule carefully.

It makes sense for video, audio and large image galleries. Probably not for a standard web build.

I’m not really a developer so forgive what is likely a stupid question, but would the DNS lookup time lag apply to sub-domains as well or would it only affect sites where the actual location was a different URL entirely?

My guess would be yes, it’s still effected but I have learned never to assume with the web.

I’m working on a rebuild of one of my sites and I might take this route, if only as a theoretical exercise. If I can make it work it’d be a cool trick to have up my sleeve.

Thanks for the links and resources! I’m sure they’ll all come in handy!

Yes, it applies to subdomains as well.

Use of a single additional domain on a page is not going to hamper performance in a noticeable way and will open the door to the use of a CDN or cloud host for cheap distributed media hosting. It also enhances parallel downloading (most browsers limit the number of simultaneous connections to a single server [2 for IE for example]; using multiple domains increases the number of files that can be downloaded at once).

However use of multiple additional domains (separating out your media by domain [video.domain.com, images.domain.com, etc.]) can cause considerable slow down, not only because of the lookup time associated with each domain, but also because of the amount of parallel downloading that splitting domains enables (effectively flooding a users connection).

You need to strike a balance. I generally split site components across at least two but no more than four domains. This is a good compromise between reducing DNS lookups and allowing a high degree of parallel download.

If you’re going to be syncing to the cloud under Linux, take a look at s3cmd (a small rsync replicator for Amazon S3 and Cludfront). We funded its development specifically for use with Symphony to enable near real-time content replication to Cloudfront.

Good explanation Joe. I went ahead and set this up for one of my own sites yesterday because we were hoovering up our 20GB monthly allowance in about 5 or 6 days. I moved a ton of assets to Amazon S3 (.swf, some large static .jpegs and all .pdf downloads), added a Cloudfront distribution on top of it and pointed a media subdomain to it using a CNAME record.

Took about an hour to set up and configure in total (after reading documentation etc), bandwidth consumption on my primary hosting account is way down, and the asset are downloading super-fast for almost no expense at all. A noticeable speed increase.

In addition to s3cmd I’d like to point out S3Hub — a tiny OSX client for managing files on Amazon S3 (although Transmit can do this too).

As others have noted, it is easiest to handle the redirects with mod_rewrite via your .htaccess file.

Sample rules:

### Cloudfront RULES
RewriteRule ^assets/css/(.*)$ http://media.your-domain.net/assets/css/$1 [L]
RewriteRule ^assets/js/(.*)$ http://media.your-domain.net/assets/js/$1 [L]
RewriteRule ^assets/ui/(.*)$ http://media.your-domain.net/assets/ui/$1 [L]
RewriteRule ^assets/audio/(.*)$ http://media.your-domain.net/assets/audio/$1 [L]
RewriteRule ^assets/video/(.*)$ http://media.your-domain.net/assets/video/$1 [L]

@Nick: it looks like this is what you’re doing on Bekonscot - correct?

Note: beware cross domain issues with Flash files. You need a crossdomain.xml file on your CDN and your web host to be able to load content between the domains.

Sample crossdomain.xml file:

<?xml version="1.0"?><!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
    <allow-access-from domain="your-domain.com" />
    <allow-access-from domain="*.your-domain.com" />
    <allow-access-from domain="*.your-media-domain.net" />
</cross-domain-policy>

it looks like this is what you’re doing on Bekonscot - correct?

Yep. Data transfer on the primary hosting account has decreased by 60% in the last 24 hours. Perfecto!

@Joseph: Cool, thank you.

@Nick: I guess one could find a way to disable bucket listing on S3? This might be a security issue in some cases:

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>bekonscot.co.uk</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>files/bekonscot-annual-tickets.pdf</Key>
        <LastModified>2009-08-15T11:56:31.000Z</LastModified>
        <ETag>"bbdfecbd233d71ca6f6363fb12f28471"</ETag>
        <Size>649677</Size>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>files/bekonscot-car-pack.pdf</Key>
        <LastModified>2009-08-15T11:56:31.000Z</LastModified>
        <ETag>"60ff99f60ada389208b9b2dbb46b9b0a"</ETag>
        <Size>126575</Size>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    ...    
</ListBucketResult>

Answering my latest post: Yes, it’s super-simple if you are using Transmit. You can simply set different “permissions” for your bucket and your files using Transmit’s information window.

[EDIT]: And I’d request this, because I was even able to browse Nick’s S3 bucket using Transmit.

@michael-e: s3cmd supports the complete underlying permissions system, so as long as you are setting the ACL as part of the command that you pass the app, the process is secure.

I love the idea of using S3 to load my media but I’m not sure this would be a viable solution except for my own website.

Correct me if I’m wrong, but wouldn’t I be unable to use Symphony’s upload field and have it upload to S3? If so, that’d likely necessitate me uploading the files beforehand then setting up Symphony to link to those files specifically.

While that’d be fine for my own stuff I don’t think I could get away with it for a client site.

Unless I’m completely getting this wrong.

@dougoftheabaci: Unless I too am mistaken, you are correct. This post has had me thinking that an “Upload to S3” field might not go astray.

@dougoftheabaci: no, you can use Symphony’s upload field as per usual. The point of s3cmd is that it can be scheduled as a cron job to automatically mirror your content from /workspace/uploads/# to S3 and Cloudfront.

A sample cron job can be seen below:

*/15 * * * *    s3cmd sync --acl-public --guess-mime-type /symphony/workspace/uploads/video/ s3://media.your-domain.net/assets/video/

I love the idea of using S3 to load my media but I’m not sure this would be a viable solution except for my own website.

I’ve moved just static content (UI assets, CSS, JavaScript, PDF downloads and large Flash SWFs) but yes, it need this additional step for dynamic content.

I don’t personally have SSH access to my shared hosting environment so s3cmd isn’t a workable solution for me. However Max’s idea of an Amazon S3 Upload field is rather a good one for those like me without root access.

There looks to be a decent open source PHP class for this as well as Amazon sample code. I imagine this would actually be an extremely easy field extension — inherit the standard File Upload field class and override the upload method with the additional call to the S3 library. This is how the Hashed Upload File works. Oh, and of course provide the S3 bucket and authentication details when configuring the field.

The downside is that if the push to S3 fails then the user can’t upload the file. The benefit of s3cmd running as a cron is that the upload process is decoupled from the syncing to the CDN. We use it at Airlock for video uploads where we also need to convert to FLV on the server, which needs to run asynchronously to the user upload process.

Again I feel like I’m asking a stupid question…

I thought cron jobs happened at set intervals? So either you’d have to hide a post until after the cron job had run or you’d have to post it with the data not fully live as the files might not be fully available.

Though, I suppose you could get around this by testing for the files on S3 and if they’re not there just using the local files.