What is a duplicate?
When Donorfy detects that a constituent has very similar details to another constituent, Donorfy will raise this as a possible duplicate. This means they might be the same person or organisation.
Enabling or disabling the checks
The duplicate check function can be enabled/disabled via Settings > Configuration > Duplicates Settings
Only those users who have access to Data Management will be able to see the Duplicates option when enabled.
Duplicate checks
There are two triggers for the duplicate check:
Weekly automated check
The weekly automated duplicate check is run for everyone at the weekend. If duplicate constituents are detected, emails are sent to users who have chosen to receive them - see 'Manage your Notifications' accessed from the profile button, top right of screen. Note: if no users have opted to receive these emails, they will instead be sent to all administrators.
Manually triggered check
The duplicate detection can be triggered by going to Data Management > Duplicates
Then select the Refresh button in the top right corner.
The process to check duplicates is a background job due to the large amounts of data it has to process and it may take some time to run depending on how large your data set is.
How the checks are run
Each constituent checked against every other constituent using a set of rules, each rule is given a confidence score.
- Personal addresses are used for individuals
- Work addresses are used for organisations/groups
- Constituents that have a linked address will not be matched
- A group will not be matched with its main contact
- Constituents which are of different types will have their score reduced to 2 even if they match a rule
|
Match method |
Confidence score |
|
National ID + First name + Last name
|
15 |
|
Email (where there is no First Name, Last Name, First line of address, Mobile or Phone)
|
15 |
|
External Key
|
14 |
|
Email + First name + Last name
|
14 |
|
Email + First name
|
13 |
|
Email + Last name
|
12 |
|
Email (where profiles feature conflicting data - i.e in both the First and Last name)
|
10 |
|
Mobile (where there is no Email, First Name or Last Name, First line of address or Phone)
|
10 |
|
Phone (where there is no Email, First Name, Last Name, First line of address or Mobile)
|
10 |
|
First line of address + Postal Code
|
8 |
|
National ID (where profiles feature conflicting data - i.e in both the First and Last name)
|
7 |
|
Postcode + First + Last Name
|
6 |
|
Postcode + First Name
|
5 |
|
Postcode + Last Name
|
5 |
|
First name + Last name (where is contradictory content in any of their First line of address, Mobile, Phone or Email fields)
|
4 |
Resolving duplicates
Duplicates can be checked and resolved in the Data Management > Duplicates menu.
There are two sections, Single Matches and All Matches, there will be a note about how many potential duplicates found within the brackets:
You can increase the number of matches displayed on the screen via the number drop-down box
You also can use the filter to refine your results by Confidence score by using the sliders at each end of the matches bar:
Once you have selected the Confidence Score you want to work within, click the filter button:
The Duplicates matches are separated out in to two sections
- Single Matches - where a 1 to 1 match has been found
The link to the article can be found here: Duplicates - Single Matches - Multiple Matches - where numerous potential matches are found
The link to the article can be found here: Duplicates - All Matches
To take some of the manual work out of management of the Duplicates process an option is provided to Auto merge items. This process uses the Confidence score of 13 or more.
The steps to set this up can be found here: Duplicates - Auto merge
Logs
Not Duplicates
Constituent pairs which have been marked as not duplicates can be viewed under Data Management > Duplicates > Not Duplicates
Deleting a match from here will result in them being reconsidered in the next duplicate check.
Merged
The merged log can be viewed under Data Management > Duplicates > Merged
All merges which we performed using the duplicates tool or merged via the constituent profile will be shown here.
Comments