If you run Windows or any MS application you will find that if you some how manage to remove IE, doing so will cause some usability issues with Windows itself. Help pages, applications will all fail to work correctly.

So best thing to do is get a hardware firewall, set it to block http traffic. This will remove any operating system issues.

Although i find the web so useful I would find it impossible to do my job without it.