Any page you add to the robots.txt file won't be indexed by many of the search engines (a few SE, robots, spiders, do not use robots.txt.). If you want some pages, but not others listed, only put the ones you don't want indexed at all on the file.
Some people want all visitors to enter through the homepage and restrict all the others. You'd be better off letting the engines follow most links. Different pages may be keyword rich and rank higher in other searches. A lot of your interior pages may rank higher than your homepage. Unless their are a few pages you really don't want them to enter on, like a privacy policy, page of credits, etc. I'd let it follow most links and let the viewer click around once they get there. It's better to get a visitor to enter on an interior page than not at all.
|