Pages with very complex structure: deeply nested tables, many frames, or unusually complex html
Lengthy JavaScript or css code at the top of the page html code listing: crawlers give up on a page that seems to contain no content
Pages with many broken links: crawlers abandon pages with many broken links, and they can’t follow the broken links to find new pages
Content with keyword spamming (repeating keywords many times in hidden text, alternate image text, or meta tags): search engines now ignore these primitive relevance-biasing schemes, and your page may even be banned from the search index if you use these techniques
Server-side or meta-refresh redirects that are used to move a user from an old url to a new one: many crawlers don’t follow the redirect link to the new page
Long, complex urls with special characters (&, ?, %, $) that are often generated by dynamic programming or databases
Slow-loading pages with inefficient dynamic links to content management systems or databases: if the page doesn’t load in a few seconds, many crawlers give up and move on
Pages that use frames or iframes: crawlers often ignore pages with complex frame schemes because they can’t make sense of the individual html files that make up each framed “page” (avoid frames where possible, and never use frames for navigation purposes)
Some dynamic pages that are assembled on request by a web application and database; be sure your developers know how you want to handle the search visibility of your content before they choose a development technology or content management tool for dynamic web sites