The Data Cleansing & Deduplication Blog

Merging Duplicate Records – The Easy Way

Posted by:

Merging duplicate records, if done correctly, can be one of the most rewarding of data cleansing tasks. To merge duplicate records manually can take a considerable amount of time and mistakes can easily happen. Saving money on duplicate mailing and increasing customer satisfaction are just a few benefits from merging duplicate records using advanced merge purge software.

In the following example, we use the powerful matching and merging features from WinPure Clean & Match 2012 to demonstrate how simple and effective this process can really be when using specialised data cleansing and deduplication software.

The first thing we do is to perform a data deduplication on the list.  For this exercise we used fuzzy matching (75% Threshold) on firstname, surname, company and add1 to achieve the following results set.

Merging Duplicate Records

We can clearly see that from this set of results that there are just 3 people, meaning we should only have 3 records instead of 8! Now, rather than removing the duplicated records, what if we want to populate the missing gaps with data from the duplicated records?

Using WinPure Clean & Match to merge these records into 1 record per person, we do just 3 things:

  1. Select the master record from each duplicate group.
  2. Click the “Merge/Update” button.
  3. Choose the “Merge” option and then click “Execute”

Lets take a closer look at each:

1. Selecting the master record from each duplicate group.

Each group of duplicates provide the ability to select a Master record.  This Master record will be the record from each duplicate that we will keep. See below on the 1st duplicate group, we have selected the record of Tom Rose as the master record. However this master record has missing values in Add2 and Postcode.

 

The 2nd duplicate group is shown below and in this instance we will choose Rebecca Turner as the master record, this again is showing there is data missing in Add2.

 

The 3rd duplicate group we will choose Joanne Wood as the master record.

 

So, for each of these duplicate groups we want to remove the duplicate AND also want to populate all the missing gaps in the master record, to give us a more accurate and populate record for each person.

2. Click the “Merge/Update” button..

Once we have chosen the master records in each group we simply click the following button:

3. Choose the “Merge” option and then click “Execute”

The following merge options are available, but in this instance we will use the Merge option. WinPure Clean & Match offers other powerful merge options that we will discuss on a separate blog.

After clicking Execute, the software will automatically remove all the duplicated records and populate the missing gaps.

As you can see below, those 8 records have now been reduced to just 3 unique records, and each of the columns have now populated from their duplicate groups….easy eh? :)

Merging duplicate records is only available on WinPure Clean & Match 2012 and you can download a free trial of the software today and perform some merging of duplicates from your own data set!

Happy merging!

 

1


About the Author: