From my experience based on the dozens of occasions in which clients have had me amalgamate names-and-addresses lists of various sorts (Outlook, Excel, Access, Word, ACT! and so on), I doubt that CRM software will remove duplicates satisfactorily.

People keep their names and addresses lists in different ways. For example not everyone allocates addresses rigorously to same first name, last name, address line 1, address line 2, address line 3, town/city, county/state, postcode/zip code field pattern.

On top of that there is the problem that most lists are flat file. Usually, this means that in business contacts lists there is one complete row for each individual. So, if there is more than one contact at a company address, then the company name and address information is repeated. If you de-dupe on the basis of company name, then you are in danger of losing individual contacts.

My approach to de-duping has been to mix a programmatic approach with manual adjustments. The process is never easy. I believe you will find there is no easy way.

I work in Microsoft Access so that I can use queries as the engine room for the earliest sweeps. The first sweeps start with postcode/zip code as the target field. Then I work from there. On occasions I found the need to write text-handling functions with which to process some entries.

I make tables of the results of each major sweep so that I return to any point of the process if later on it is determined that things have gone wrong somewhere.

I aim to produce finally one table of company names and address and another table of contacts with a one-to-many link between the tables to associate individuals with companies.

Without exception I am left with records that have not passed throught the sieves and for which I have to ask the client what I am to do with them.

It is likely that you will not get the de-duping right to everyone’s satisfaction. So prepare yourself for the complaints.

On the up side, I am pretty certain that my services represented better value to the client than putting the work out to a bureau. By doing it yourself you will have complete control and will know where uncertainties have arisen during the de-duping/cleansing activity.