The more files you include on a page the longer it takes to download everything. Even when all you have is a lot of tiny files like JS, there's still a large limitation in the form of browsers limiting the number of HTTP connections they'll make to a single website at once. This limit is normally 2 connection to a domain. So only two files at once can be downloaded, and there's a certain amount of negotiation overhead when moving to the next file.
Since page rendering is held up by all of the scripts and CSS in the head, that means you really want to have as few files as possible load in the head. Otherwise your viewers are left watching a blank page for precious fractions of a second while the 20 files in your head are downloaded two at a time.
deviantART has a lot of CSS and JavaScript. I counted right now (ack -G ".js$" -f | wc -l), and we have 560 JavaScript files and 310 CSS files. Not all of them are needed on every page, of course... but our core set of JS that gets loaded everywhere consists of 53 files, and the equivalent CSS is 34 files.
Back when we were a young site we just stuck all the JS into the head, because we didn't know better, and also because we didn't have much JS back then. But then we added more functionality, and we noticed just how slow it was making us. So we set out to develop a way to not suck.
Nowadays we use a system of automatically bundling up our JS and CSS into big files, so a single HTTP connection can fetch them all at once. So the 34 CSS files I mentioned become this big file. In the case of the JS it's a bit more complicated, and we also minify all of the files using the YUI compressor, resulting in something like this.
We define all of these bundles in "list files". These are simple text files listing the other files that should be combined. So v6core.js.list contains a bunch of file names, and it gets bundled together as v6core.js.
The bundling occurs in a svn commit hook. So whenever a developer makes a commit that touches a .css or .js file, it triggers a rebuild of the .list file that contains those files. The rebuild happens on our staging server, and the files get copied out to production when we do a release.
Dependencies
Now, because we know this .list system exists, we get to make lots of small JS files that contain single pieces of functionality. These components wind up having dependencies on each other... jQuery is used almost everywhere; lots of code creates modal windows; etc. So now we're faced with the problem of only including the .list files that contain the code we need for the current page.
We used to have to manually declare all of these dependencies in PHP when adding JS/CSS to a page, like so:
$gWebPage->addJSDependency("lib/json2.js");
$gWebPage->addJSDependency("lib/difi.js");
$gWebPage->addJSDependency("lib/events.js");
$gWebPage->addJSDependency("pages/awesome.js");
This is obviously somewhat unwieldy, and is prone to us forgetting a dependency but having it work because at the moment the missing dependency is in a .list file that's being included anyway. Then breaking later because we rearrange the .list files so that less commonly used code is only loaded when needed.
So now what we do is have some special comments at the top of our JS/CSS files which look a little like this:
/* This is the hypothetical pages/awesome.js
@require jms/lib/difi.js
@require jms/lib/events.js
*/
Then in the PHP we just have to do:
$gWebPage->addModule("jms/pages/awesome.js", MODULE_FOOTER)
...and it'll take care of the rest without us having to think about it. It guarantees that the dependencies (and their dependencies) will be loaded before the requested file. The second argument ("MODULE_FOOTER") is a priority; the caller can say whether they need this JS to be output in the head, the top of the body, or the end of body. This makes sure that the only JS in the head is the JS that really needs to be there.
The dependency mapping is built in the same commit hook that I mentioned earlier, and is serialized out into a file that's loaded when we need to resolve dependencies.
DWait dwhat?
When we were trying to remove as much JS as possible from the head, because it blocks rendering, we encountered the problem of JS in the head that controls behavior on the page. It obviously needs to be there as soon as possible, because otherwise a user who quickly clicks somewhere might see an error, or just have nothing happen. But in the vast majority of cases people won't click really quickly, and if we put it in the head we'll have delayed rendering for nothing.
Our solution to this problem is called DWait. It's way for our JS to request that an action be delayed until a dependency has loaded. This lets us stick a lot of code in the very footer of the page, without worrying about whether some link in the page depends on it.
So you'll see a lot of code like this on dA:
<a onclick="return DWait.readyLink('jms/pages/gruzecontrol/gmframe_gruser.js', this, function () { GMI.query('GMFrame_Gruser', {match: {typeid: 62 }})[0].loadView('submit') } )" href="#" id="blog-submit-link" class="gmbutton2 gmbutton2plus">
This says that the click handler for the link depends on gmframe_gruser.js. If the file is already loaded in a .list then it'll execute the handler immediately. Otherwise it'll remember the click and run the handler as soon as the load has happened.
To detect the loading every bundle file created by our commit hook gets a line of JS added to the end which tells DWait that the individual files within it have been loaded.
There are also a few JS files that have a special command in their header called "@fastcall". This means that the file is so important to the page that it has to be output directly in the head of the page as an inline script. We cache a minified version of the JS in the dependency map so that this case doesn't involve extra file reads on the webservers.
There's one more trick that DWait has for cutting down load time, and it goes back to the priority argument to addModule that I mentioned earlier. We can tell it to use the priority "MODULE_DOWNLOAD" which means that dependency information is passed to DWait, but that the JS file itself isn't loaded. Instead it waits until a DWait.ready call asks for it and then dynamically loads the file. This is fantastic for rarely used functionality, with the tradeoff being a slight delay when the user first uses it.
Takeaway
These techniques are important for any website, no matter how small. Page load speed has a major effect on how people perceive your site, and there's a lot you can do to to improve it. As a first step, just get as much as possible bundled up and out of the head of your page, and see how much effect it has.
