I have created a nav list for my website but the html validator is showing errors for html4. I haven't been able to resolve it. Can you help me?

The issue is that an error message is given for using <nav>. If I remove this the <li> tags show as an error. If I remove them I might as well go for a pint and forget about it..

This is the html code.

HTML Code:
<div id="menu">
  <li><a href="index.html">Home</a></li>
  <li><a href="">Studies</a>
    <li><a href="study_what_christians_believe.html">What do Christians believe about God?</a></li>
    <li><a href="study_accepted.html">Accepted as you are</a></li>
    <li><a href="study_reconciliation.html">Reconciliation</a></li>
    <li><a href="study_baptism.html">What is Baptism?</a></li>
    <li><a href="study_forgiveness.html">Forgiveness</a></li>
    <li><a href="study_the_holy_spirit.html">The Holy Spirit</a></li>
    <li><a href="study_the_devil.html">Satan, the Devil</a></li>
    <li><a href="study_1_kings_and_elijah.html">1 Kings and Elijah</a></li>
  <li><a href="">Books</a>
    <li><a href="book_christians_believe.html">What do Christians believe about God, Jesus and the Holy Spirit?</a></li>
    <li><a href="book_all_studies.html">All 8 Bible Studies</a></li>
    <li><a href="both_books.html">Both Books Combined</a></li>
  <li><a href="author.html">Author</a><li>
  <li><a href="http://what-do-christians-believe.co.uk/wp/category/food-for-thought-blogs/">Food For Thought Blog</a></li>
  <li><a href="">Contact Us</a>
    <li><a href="contact-what-do-christians-believe.html">Simple Form - preferred method</a><li>
    <li><a href="MAILTO:info@what-do-christians-believe.co.uk">By email</a><li>
  <li><a href="sitemap.html">Site Map</a></li>
For studies and books I have use <li><a href="">Studies</a> as they don't do anything except reveal a sub menu. Is this the correct way to stay on page/url?

What do I need to do to get it to validate?