For SQL, I use SQL in a nutshell 2nd Ed. (O'Reilly), and the MySQL Pocket Reference (O'Reilly). For CSS I have a book called (I think) Practical CSS for the World Wide Web (it has a picture of a waterfall on the cover, IIRC). I don't use it much, though; I find it easier to just do a quick google search, and click on the W3C Schools results
I don't have any HTML/XHTML references (I don't really do XHTML yet, haven't had a need for it, and I've been doing HTML for about 8 years now). I learned PHP/MySQL from Larry Ullman's "PHP and MySQL for Dynamic Websites" (it is a Visual Quickpro Guide), but I don't have any PHP references (I really need to pick up one).
I also find it useful to flip through cookbook style books when I'm interested in certain things covered by them.
Another good reference is going to be sample code. Here is a easy little trick to get some.. Find a book on Amazon that has good reviews, especially regarding the sample code, and then look for the book website (or the website of the author). Usually you can download the sample code that is in the book for free.