Hello ,
I have some suggestion. Hope it’s useful in your case.
1. Switch on both cache (Joomla + Zite template).
2. Check how many foreign site is referred from your content (DB query) it could be a surprise (i.e. pics)
3. Reconsider the usage of the “important” plugins and modules, sometimes switching them off is better (i.e. google ajax library and so)
4. Check the log in the cpanel it tells a lot (if it is not empty)
5. Server settings and a good htaccess could help also.
6. Your articles need to be archived regularly (choose a number of days/weeks to keep “current” or un-archived) so that upon a homepage hit the template will be quicker.
Regards.