I tried PHPBB several years ago for a closed forum, and wanted to change the name of the key modules so the bots out there that search for those known files (like the admin files, etc) wouldn't be able to find them. But each time they updated the application, it was a nightmare to go back and find where I needed to make the modifications, and I finally just gave up and took down the forum. (It was a good thing that the clients never used it.)

Have you had any experience with this, and how did you solve it? That's my main concern with the free, open source applications - everyone knows the module names, which lessens their security.