I noticed that several of my apps with content are not being generated in the site map for search engines. Content from automatically generated content, like spywall never reaches search engines. While generic static content like privacy, terms of use, about and help files tends to easily reach their eyes. Even forum posts are being lost in the shuffle, while profiles are easily found but have no bearing on site content. Now I know how to disallow in the robot.txt to eliminate some of the more indexed non essentials like search.php, browse.php, search and browse, but how would I get the sitemap to index these other pages.
With SEO being so important to the success of a social website, I find it surprising that the software has not provided optimization of sitemap. There should be a section on admin solely dedicated to daily steps to be taken to increase ranking.
