Valid web pages are accessible. When pages are accessible, they might be usable. Validity is a formal criterion, which can be checked automatically by a validation process.
There are good reasons for checking the validity of your web pages with respect to the HTML specifications. Pages that can pass a validity test are readable with any browser, and are thus accessible by the widest possible audience. An invalid page, that is tuned to display well in current browsers, usually breaks when a new browser version is released. On the contrary, valid pages survive browser updates. Such pages are also better suited for automatic analysis, which can improve your search engine positioning.
There are many definitions of validity. For some, it just means that the page exists on the web. Others include a test for "dead links", but this is still not sufficient. What is needed, is to check that your HTML codes conform to the specified DTD (Document Type Definition). For this you can only use a "real validator". While there are many "fluff validators" out there, "real validators" are more scarce.
The World Wide Web Consortium (W3C) is developing the standard specifications for coding web pages, and they have made freely available a reliable HTML Validator that assesses the formal validity of any HTML (or XHTML) page. You can use it to test your web-site, and see for yourself whether your pages need a make-over!
Another, equally recommendable "real validator" is the Web Design Group's WDG HTML Validator, with its excellent error messages, developed by Liam Quinn. Both Validators are very similar, and built on top of the same underlying parser, written by James Clark.
Since no commercially available authoring software is yet capable of producing pages that conform to the specifications, most web pages contain invalid coding, resulting in various incompatibilities, and even a few browser crashes. Understandably, readers tend to dislike pages that are loaded with junk, and not to revisit such sites. A report by the Gartner group has even shown that many corporate sites are so bad that they actually harm the reputation of the companies involved.
Though there is much more to it, usability is clearly related to validity: to be usable, web pages need to be accessible, since it is not possible to use a resource when it cannot be accessed. To be accessible, pages must be valid, so validity is transitively a prerequisite of usability.
There are two main accessibility challenges: alleviating human disabilities, and overcoming the different computer platforms' diverging capabilities. The first aspect is addressed in a Technical Recommendation published by the W3C, on how to make web pages accessible to people with disabilities. Following this advice results in far better pages, that are actually much more usable by everyone. The second challenge is addressed in W3C's recently launched device-independence activity. Improving validity helps on both accounts.
You can test the accessibility of your web pages at the Bobby site, which also provides links to extremely valuable explanations about each issue raised. Fairly enough, you might even want to check Bobby's accessibility rating for the present page. Fortunately, it is quite good, since it qualifies as level 3 (the highest possible).
Once pages are valid and accessible, they might still require significant changes before readers find them usable. Jakob Nielsen's Useit site contains a lot of healthy, though often quite polemical insight on these questions.