In the past 2 weeks, I have met two clients that have SharePoint internet sites and that are exposing system pages (under _layouts) to anonymous users.
If you allow anonymous access to your site, test the following urls:
By not securing your site properly, you will expose list and libraries that are not officially published.  When people will go to _layouts/viewlsts.aspx, they will more then likely see a different branding, because these pages are using a different masterpager.  The anonymous user could also see the name of the users who have published some of the documents,  and more.
When using MOSS you can lock down your site by using the lockdown mode. 
"The Lockdown mode is a feature that you can use to secure published sites. When lockdown mode is turned on, fine-grain permissions for the limited access permission level are reduced.  Consider using lockdown mode on published sites if greater security on these sites is a requirement. Additionally, if you applied the Publishing Portal site template, determine if lockdown mode is the desired configuration for these sites. If not, use the Stsadm.exe command-line tool to turn off lockdown mode."

This text is taken from this article:

You can ture on the lockdown mode for a site collection by using stsadm.exe:

stsadm -o activatefeature -url <site collection url> -filename ViewFormPagesLockDown\feature.xml

Have a look at these article if you want to learn more about how to use the lockdown feature and how to secure internet access sites: