After being spidered....

I'm using the VB forum on a site I'm developing and the spiders having been running mental on the site for a while now... everything is fine apart from the forum links... there are pages that don't need to be indexed, like usercp.php in VB which are getting multiple hits because each URL like has a different session ID in it.... Inktomi seems to be the worst.

I've added the pages that the spiders don't need to index and put them in the robots.txt file.

Because the spiders have been going mental for a few months now, 90% of the existing links are irrelevant pages that all have different URL parameters due to the session ID.

Now that the spiders know about previous links, and now they are being informed to ignore them.

Q: Would the spiders eventually forget about these links never to return on old previously spidered links?

Q: Does the disallow in the robots file inform them to remove the any existing content over time?

Or simply put, what would the net effect be by doing this?

Sessionids are a bad idea when it comes to spiders, it is what we would like to call a spider trap. Because of the sessionid changes with each session, you will have spiders indexing the same content over and over and over again.

The best id is not to serve spiders sessionids or not to pass the session variables
as a part of the URL string. I am not sure if I followed your question about robots.txt so I will not address it.