From an interview with Steve Souders published on ☞ O’Reilly Radar:
The first thing I recommend website owners do is get a handle, get an idea of the overall page load time. And then the second thing I say is break that into two parts: the back-end and the front-end. And if you find that like most websites, your back-end time is less than 10 or 20 percent, then you’re correct in focusing on these front-end best practices. If your back-end time is 30 to 50 percent or more, then you should really start on looking at your back-end architecture.
And in case you are running a media company, you should also check the part of the interview focused on the degradation of performance due to non-optimized ad servers.