What is a duplicate?
When Donorfy detects that a constituent has very similar details to another constituent, Donorfy will flag them as possible duplicates. This means they might be the same person or organisation.
Enabling or disabling the checks
The duplicate check function can be enabled/disabled via Settings > Configuration > Duplicates Settings
Only those users who have access to Data Management will be able to see the Duplicates option when enabled.
How to check for duplicates
When adding constituents to Donorfy
There are numerous ways that constituents can be added into your Donorfy. Manually via the keyboard, automatically via the integrations, imported using the File Uploader. Donorfy minimise the chance of duplicates by assessing whether the constituent being added already exists in Donorfy, or not.
Manually via the keyboard
When you add a new constituent via the keyboard (by clicking the orange Add button at the top of the screen), Donorfy will attempt to find matches - or near matches - based on the information you enter, as you type it. These are displayed on the right, so the user can make a choice to go ahead and add a new one, or select one of the suggested constituents if they are in fact already in the system.
Donorfy's standard integrations
Donorfy will use the information provided and attempt to find a match, using the algorithm below. If a match is found, with a confidence score of 13 or above, Donorfy will not add a new constituent, and it will use the constituent found. If multiple safe matches are found, Donorfy will use the oldest one (the one with the earliest Date Added).
Imported via the File Uploader
Donorfy uses the same criteria as described above for the standard integrations.
Checking for duplicates across all of your constituents
There are two ways to get Donorfy to check your entire constituent database for duplicates:
Weekly automated check
This happens at weekends. If duplicate constituents are detected, emails are sent to users who have chosen to receive them - see 'Manage your Notifications' accessed from the profile button, top right of screen. Note: if no users have opted to receive these emails, they will instead be sent to all administrators.
Manually initiated check
The duplicate detection can be started by going to Data Management > Duplicates
Then select the Re-scan for Duplicates button. This initiates a background job so you don't have to wait for it to finish - you can get on with other work in Donorfy. The process can take several hours to complete, depending on the volume of constituent records in your Donorfy. You will receive an email when it is complete.
How the duplicate check algorithm works
Each constituent checked against every other constituent using a set of rules, each rule is given a confidence score.
- Personal addresses are used for individuals
- Work addresses are used for organisations/groups
- Constituents that have a linked address will not be matched
- A group will not be matched with its main contact
- Constituents which are of different types will have their score reduced to 2 even if they match a rule
|
Match method |
Confidence score |
|
National ID + First name + Last name
|
15 |
|
Email (where there is no First Name, Last Name, First line of address, Mobile or Phone)
|
15 |
|
Email + First name + Last name
|
14 |
|
Email + First name
|
13 |
|
First name + Last Name + Address line 1 + postal code (all 4 fields must contain data)
|
13 |
|
Phone or Mobile (where there is no data in the email, first name, last name or address)
|
13 |
|
External Key
|
12 |
|
Email + Last name
|
12 |
|
Email (where profiles feature conflicting data - i.e in both the First and Last name)
|
10 |
|
Phone (where there is no Email, First Name, Last Name, First line of address, Mobile or Phone; or where those fields may be populated but don't match)
|
10 |
|
First line of address + Postal Code (exact)
|
9 |
|
First line of address + Postal Code (fuzzy)
|
8 |
|
National ID (where profiles feature conflicting data - i.e in both the First and Last name)
|
7 |
|
Postcode + First + Last Name
|
6 |
|
Postcode + First Name
|
5 |
|
Postcode + Last Name
|
5 |
|
First name + Last name
|
4 |
Email matching requires the emails to be of the same type - Personal, Work, or Other. If two constituents have the same first name, last name, and email address, but one has it as Personal Email and the other has it as Work Email, they will only match on first name + last name.
Resolving duplicates
Duplicates can be checked and resolved in the Data Management > Duplicates menu. There are two sections, Single Matches and All Matches, there will be a note about how many potential duplicates found within the brackets:
You can increase the number of matches displayed on the screen via the number drop-down box
You also can use the filter to refine your results by Confidence score by using the sliders at each end of the matches bar:
The Duplicates matches are separated out into two sections
- Single Matches - where a 1 to 1 match has been found
The link to the article can be found here: Duplicates - Single Matches - Multiple Matches - where numerous potential matches are found
The link to the article can be found here: Duplicates - All Matches
The Search function allows you to search for constituents. It works on the view you are currently displaying (Single Matches or Multiple Matches):
Auto merge
To take some of the manual work out of management of the Duplicates process an option is provided to Auto merge items. This process uses the Confidence score of 13 or more.
The steps to set this up can be found here: Duplicates - Auto merge
Logs
Not Duplicates
Constituent pairs which have been marked as not duplicates can be viewed under Data Management > Duplicates > Not Duplicates
Deleting a match from here will result in them being reconsidered in the next duplicate check.
Merged
The merged log can be viewed under Data Management > Duplicates > Merged
All merges which we performed using the duplicates tool or merged via the constituent profile will be shown here.
Preferences
Communication preferences will take the most recently updated value (excluding the yellow "unknown" state) from either the target constituent or source constituent.
Comments