Get Your Data Ready for an Integration

A reconciled appointment-type list, clean provider availability, a deduplicated patient set, normalised phone numbers, and a written record of what each system may write.

  • Muhammad Qasim Hammad
  • 17 min read
7
Lessons
17
Minutes
6
Figures

Before you start

Before you start

  • + Admin access to your practice management system, or somebody who has it
  • + A clinician or practice manager who can rule on appointment types
  • + The ability to export a patient list and an appointment-type list
  • + About 6 hours, best spread across 2 weeks

What you will end up with

  • + A reconciled appointment-type list where every surviving type has the duration it actually takes.
  • + Availability rules that match how providers currently work, including leave entered far enough ahead.
  • + A counted and merged duplicate set, with the count kept because it measures your intake process.
  • + Phone numbers in one canonical format, with landlines flagged separately if you intend to text.
  • + A written decision about which system is the authority on each fact, and what any integration may write.

Integrations rarely fail because the software could not connect. They fail because the practice had four appointment types meaning the same thing, three records for the same patient, and phone numbers stored six different ways.

the appointment type list

Consult or Consultation?

the integration, day one

A receptionist picks the right one without noticing there was a choice. Software cannot.

There is a version of an integration failure that gets blamed on the vendor and is not their fault. The connection works, the credentials are right, data moves in both directions, and the result is wrong appointments, duplicated patients, and messages sent to numbers that do not exist.

What happened is that a system with no tolerance for ambiguity met a dataset that had been quietly accumulating ambiguity for years, handled entirely by people who resolved it from memory without noticing they were doing it.

This is the cleanup that prevents that. Seven lessons, about 6 hours, best done across 2 weeks and before you sign anything, because every one of these is cheaper to fix without a vendor waiting on you.

Why this comes before the vendor

A practice management system is tolerant because humans operate it, and humans are extraordinary at resolving ambiguity without noticing. A receptionist seeing two appointment types called "Consult" and "Consultation" knows which is which, and picks correctly every time without ever registering that a choice was made.

Software has none of that. It matches on the value it is given, so the ambiguity a person resolves silently becomes a defect the moment anything automated touches it, and the defect appears as wrong behaviour rather than as an error message.

There is a second reason the cleanup earns its keep regardless of what you buy. Almost everything in these 7 lessons is something a practice would benefit from having done anyway: appointment types that mean what they say, a diary that reflects who actually works when, and patient records that are not duplicated. Nobody schedules that work on its own, because it is nobody's job and it never becomes urgent. An impending integration is simply the occasion that finally makes it happen.

Doing this before selecting a vendor also removes a genuine confound from your evaluation. A pilot run on messy data measures your data, and you will not be able to tell a weak product from a clean product fed bad inputs.

Humans resolve ambiguity without noticing. Software cannot, so every ambiguity becomes a defect the day something connects.

There is a measured cost to leaving this undone. The 2023 CAQH Index puts a published price on each administrative transaction by mode, and the gap between the manual and the electronic version of the same transaction is the recurring bill a practice pays for data nobody reconciled [2]. An integration built on top of unreconciled data does not remove that bill. It automates it.

01Reconcile your appointment types

This is the single highest-value lesson and it is the one practices resist most, because the list has grown organically and every entry made sense to somebody at the time.

  1. Export every appointment type in your system, including inactive ones.
  2. Count how many were actually used in the last 6 months, and how often.
  3. Group the ones that mean the same thing, which is usually more of them than expected.
  4. For each group, pick one survivor and mark the rest for retirement.
  5. Check the duration on every survivor is the duration actually used, not the one set at installation.
  6. Have a clinician confirm the surviving list before anything is changed.

Step 5 is where the operational damage hides. An appointment type set to 30 minutes that everybody knows really takes 45 works perfectly while humans are booking it and produces a schedule that runs late all day the moment anything books it literally.

An appointment type set to 30 minutes that everybody knows takes 45 is a schedule that runs late all day, the moment anything books it literally.

Step 3 usually finds duplicates created by different staff at different times, plus types for services the practice no longer offers, plus one or two created for a single unusual case in 2022 and never removed.

What the type list usually contains
  • ✓Two or three names for the same appointment, created by different staff
  • ✓Types for services the practice no longer offers
  • ✓One created for a single unusual case years ago and never removed
  • ✓Durations set at installation that nobody has used since
  • ✓Inactive types that still appear in booking screens
Lesson 1. Every entry made sense to somebody at the time it was created.

Expect this lesson to be contentious in a way the others are not, because appointment types encode how people think about their own work. A clinician who has used a particular type for years will experience its retirement as a change to their practice rather than as a tidy-up. Frame it as reconciliation rather than reduction, and be willing to keep a type that only one person uses if that person genuinely uses it.

The count in step 2 does most of the persuading. A type used 4 times in 6 months is easy to discuss; a type nobody can defend on usage usually retires itself once the number is on the table.

Retire rather than delete. Historical appointments reference these types and deleting them damages your own reporting, so mark them inactive and stop them appearing in new bookings.

02Fix the availability rules behind the diary

The second lesson is about the rules that decide when a slot exists, which are usually accurate for the way the practice worked when they were configured and not for how it works now.

What to checkThe failure it prevents
Each provider's actual working days and hoursBookings offered on a day somebody does not work
Admin, catch-up and lunch blocksPatients booked into time reserved for something else
Which appointment types each provider actually doesA patient booked with a clinician who does not do that treatment
Room or chair constraints, where they existTwo bookings for one physical space
Holiday and leave, for the next 3 monthsThe most common cause of a booking that has to be undone

The third row is the one that produces the worst patient experience, because the appointment exists, is confirmed, and is wrong in a way nobody notices until the patient arrives.

Check the second row against reality rather than against the configuration. Catch-up time and admin blocks are frequently configured once and then informally used for something else, or informally not used at all, and a connected system will fill whatever the configuration says is free. The question to ask each provider is not what their diary says but what actually happens in that slot on a normal Tuesday.

The fifth row needs a process rather than a fix. Leave entered 2 weeks late is leave that a connected system will happily book over, so agree who enters leave and how far ahead, and treat that as part of the integration work rather than as an HR matter.

03Find the duplicate patient records

Every practice has duplicates, and the number is always higher than the practice thinks. They matter here because anything automated will pick one record and update it, and a 50% chance of picking the right one is not a system anybody wants.

  1. Export your patient list with name, date of birth, phone and email.
  2. Sort by date of birth and look for matching or near-matching names.
  3. Separately, sort by phone number, which catches spelling variants and married names.
  4. Separately, sort by email, which catches the same person entered twice from different forms.
  5. Count what you find before merging anything, because the count tells you how bad the underlying capture process is.
  6. Merge according to your system's supported process, never by editing records manually.

Do the passes in that order, because each one shrinks the work for the next. Merging the obvious birthday matches first removes rows from the phone-sorted pass, and by the third pass you are usually looking at a short list rather than the whole file.

Step 3 finds the most. A patient entered as Kathryn once and Katherine another time will never match on name and will match immediately on a phone number, which is why the phone-sorted pass is worth doing even though it feels redundant.

Step 5 matters more than the merge itself. If you find 300 duplicates in 4,000 records, you have a capture problem that will regenerate them, and merging without fixing that is a task you will repeat every year.

Three passes over the patient list
  1. 1
    Sort by date of birth
    Catches near-matching names on the same birthday.
  2. 2
    Sort by phone number
    Catches spelling variants and married names. Finds the most.
  3. 3
    Sort by email
    Catches the same person entered twice from different web forms.
  4. 4
    Count before merging
    The count measures your intake process. The merge deletes the evidence.
Lesson 3. Each pass catches duplicates the others miss, which is why all three are worth doing.

Count the duplicates before merging them. The count is a measurement of your intake process, and the merge deletes the evidence.

Leave the ambiguous ones alone rather than guessing. Two records for the same name and birthday with different addresses might be a duplicate or might be a parent and child sharing a name, and merging those is a much worse outcome than leaving 2 records that a human will disambiguate. Flag them, and let somebody who knows the patients look.

Step 6 exists because manual merging loses data. Practice management systems have a merge function for a reason, and editing one record then deleting the other discards history that may matter clinically.

04Normalise the phone numbers

Phone numbers are where integrations fail most visibly, because a number stored in a format the sending system does not recognise simply fails, usually without a useful error.

  1. Export every patient phone number and look at the distinct formats present.
  2. Expect at least 5 variants: with and without country code, with spaces, with dashes, with brackets, with extensions appended.
  3. Pick one canonical format, which for US numbers should be the full 11 digit form with country code.
  4. Convert in bulk if your system supports it, and by hand if the count is small.
  5. Flag anything that is not a valid number: internal extensions, obvious typos, and the digits somebody typed to skip a required field.
  6. Flag landlines separately if you intend to text, since a text to a landline is a silent failure.

A text to a landline does not bounce. It simply never arrives, and the delivery report stays green while a share of your list hears nothing.

Step 6 is the one that surprises practices after go-live. A message sent to a landline generally does not bounce in any visible way, so a practice can be failing to reach 15% of its list for months with a delivery report that looks entirely healthy.

Country code is the choice that catches practices out later. Storing numbers without it works perfectly while everything is local and breaks the moment anything is sent through a system that assumes international format, which is most messaging platforms. Adding it now is a bulk operation; adding it after go-live means reconciling against records something else has already touched.

Step 5's last category is real and common. Required fields produce fake data, and a phone column with a scattering of 1111111111 entries tells you something about a form somewhere as well as giving you rows to remove.

One thing worth doing while you are in here. Look at how many records have no phone number at all, as distinct from a bad one, because that count tells you something about your intake forms rather than about your data. A practice with 8% of records missing a mobile has a form that does not require one, and fixing the form stops the number growing while you clean what is already there.

If you intend to send messages at all, the consent and registration questions are separate from data hygiene and have to be settled independently, per the patient texting guide.

05Decide which system is the authority on what

When two systems hold the same fact, one of them has to be right, and deciding which one after they have disagreed is much harder than deciding in advance.

FactUsual authorityWhy
Patient demographicsPractice management systemIt is the clinical record
Appointment slots and availabilityPractice management systemThe diary is the operational truth
Contact preferences and opt-outsWhichever system every sender can readUselesss if a sender cannot check it
Enquiry and lead recordsThe front-office or CRM systemThese are not patients yet
Consent recordsOne system, always, without exceptionTwo consent stores is the failure mode

The third and fifth rows are the ones that cause real problems, because they are the two facts most likely to be held in several places by systems that were installed years apart and cannot see each other.

The second row is worth a moment even though it looks obvious. Practices with an online booking widget frequently have availability rules in 2 places, one in the practice management system and one in the widget's own configuration, set up at different times by different people. They agree until somebody changes their hours, and then they disagree silently until a patient turns up to a closed building.

Write the decision down and tell whoever administers each system. An authority that exists only as an understanding between two people is not an authority, and it will be contradicted by the first person who was not in that conversation.

One fact, one authority
Decided in advanceLeft to work itself out
When two systems disagreeOne of them is definitively wrongAn argument about which to trust
Consent recordsOne store, every sender reads itTwo stores, one of them stale
A new administrator arrivesReads the written decisionInvents their own
Cost to establishOne conversation and a noteA data reconciliation project
Lesson 5. Decide in advance, because deciding after they disagree is much harder.

Authority is also a legal question, not only an operational one. The information blocking rules administered by ONC restrict a practice or a vendor from unreasonably preventing access, exchange or use of electronic health information [3]. That does not oblige you to let a phone vendor write to your chart, and it is worth knowing which side of the line your answer sits on before a vendor tells you.

06Establish what each system may write, and to where

Reading is low risk. Writing is where an integration can do damage, and the scope of what any connected system may change should be a deliberate decision rather than whatever the default integration offers.

  • List every field any connected system will be able to write
  • Remove anything it does not need, rather than accepting the default scope
  • Decide explicitly whether it may create patient records, or only match existing ones
  • Decide whether it may modify existing appointments, or only create new ones
  • Decide what happens when it cannot match a patient confidently
  • Confirm every write is attributable to the integration in your audit trail
  • Confirm you can revoke its access in one action, and know who can do that

The third line is the most consequential. A system permitted to create patient records will create duplicates, because it cannot know that Kathryn Smith with a new mobile number is the same person as Katherine Smith, and it will resolve that uncertainty by creating a record.

The fourth line is worth thinking about separately from the third. Creating an appointment is additive and its failures are visible: a wrong slot appears and somebody notices. Modifying an existing appointment is destructive, and its failure mode is a patient who turns up at the old time because the change was never communicated. Many practices should permit the first and withhold the second, at least until the system has earned some trust.

Narrowing write scope is also a privacy discipline rather than only an operational one. The minimum necessary standard requires covered entities to limit uses and disclosures of protected health information to the minimum needed for the purpose [1], and an integration granted every field because that was the default is not a considered application of it.

The fifth line deserves an explicit answer rather than a default. A system that cannot confidently match a patient has three options: create a new record, attach to the closest match, or stop and ask a human. Only the third is safe, and it is rarely the default, because the first two let a vendor demonstrate a smoother flow.

The last two lines are the ones nobody checks until they need them, which is the worst possible time. Confirm both before go-live, and write down who holds the ability to revoke.

Some of the plumbing is arriving whether or not you plan for it. CMS-0057-F, finalised in January 2024, requires affected payers to expose prior authorisation and patient access APIs, with the main API requirements taking effect on 1 January 2027 [4]. If your data is not reconciled by then, the new interfaces will simply carry the same inconsistencies faster.

07Take a snapshot you can compare against

The final lesson costs 20 minutes and is the only thing that will let you distinguish an integration problem from a coincidence in month 2.

  1. Record your patient record count, immediately before anything connects.
  2. Record your count of appointments in the next 30 days.
  3. Record your duplicate count from lesson 3, after merging.
  4. Record your unreachable phone count from lesson 4.
  5. Export the appointment-type list as it stands.
  6. Date all of it, and keep it somewhere that is not the system being integrated.

Take it immediately before rather than a fortnight before. A snapshot from 2 weeks earlier includes 2 weeks of ordinary change, and separating that from anything the integration did is exactly the problem the snapshot was supposed to solve.

Step 1 is the one that catches the most common go-live failure. A patient count that grows by 40 in a week when you saw 30 new patients is an integration creating duplicates, and without the before number that growth is invisible.

Step 2 needs a stated window rather than a vague one. Thirty days is the useful choice because it is long enough to contain a normal booking pattern and short enough that a comparison is not swamped by seasonality, and because anything an integration does wrong to your diary will show up inside it.

Step 6's last clause is not paranoia. A snapshot stored inside the system it describes is not much use if that system is the thing behaving oddly.

The pre-go-live snapshot
  • ✓Patient record count, taken immediately before anything connects
  • ✓Count of appointments in the next 30 days
  • ✓Duplicate count after merging
  • ✓Unreachable and landline phone counts
  • ✓The appointment-type list as it stands, exported
  • ✓All of it dated, and stored outside the system being integrated
Lesson 7. Twenty minutes, and the only way to tell a problem from a coincidence later.

Keep the snapshot somewhere a person will find it, which in practice means telling somebody other than yourself where it is. A file on one laptop is not a control, and the month you need it is disproportionately likely to be a month when the person who made it is unavailable.

Re-take the same 5 numbers 30 days after go-live and compare. If they have moved in ways you cannot explain, you have found something early, which is the entire purpose of the exercise and the reason it belongs in the same discipline as running a 30 day pilot.

What this does not prepare you for

Clean data does not make an integration work. What it does is remove the most common class of reasons that an integration does not work, which is a narrower and more honest claim. The reasons that remain are architectural, and none of them is improved by anything in the 7 lessons above.

Whether your practice management system exposes a supported write interface at all is the largest of them, and several widely used systems expose reading far more readily than writing. That question decides which category of product you can actually buy, which is worked through in what "the AI books the appointment" actually means.

What is left after the data is clean
This guide removes
  • Wrong appointment types and durations
  • Bookings on days providers do not work
  • Duplicate records created by automated matching
  • Messages failing silently to landlines and bad numbers
Still to establish
  • Whether your system exposes a supported write interface
  • What happens when the system is uncertain
  • Who owns the queue that captures land in
  • Whether the vendor's integration is maintained and in live use
These are architectural and none of them is fixed by anything in this guide.

A second architectural question that survives this cleanup is whether the vendor's integration with your specific system is actually maintained and in live use by other practices today, as opposed to existing in their codebase. Integrations decay quietly when a vendor's other customers move off a platform, and there is no external signal that has happened. Ask when it was last used in production, and by how many practices.

Nor does clean data resolve what should happen when a connected system is uncertain. That is a policy question your practice answers, not a data question, and answering it is the subject of writing your call handling rules.

The honest summary is that this guide removes a class of failure that is otherwise very likely, and leaves you facing the smaller set of problems that are actually about the software. One last sequencing note. If you are doing this alongside a vendor evaluation rather than before one, do lessons 1 and 3 first regardless of how far the evaluation has got, because those two are the ones whose absence will most distort a pilot. The rest can run in parallel with the conversations.

That is a good trade for 6 hours, and every hour of it is useful even if you never integrate anything, because a practice with reconciled appointment types and no duplicates simply runs better.

FAQs

Questions about this

About 6 hours of work spread across 2 weeks, and the spread matters more than the total. Appointment-type reconciliation needs a clinician's judgement, which arrives in fragments, and duplicate merging is better done in short sessions than in one long one where attention degrades and mistakes get made.
Before, for two reasons. Everything here is cheaper to fix without a vendor waiting on you, and a pilot run on messy data measures your data rather than their product. If you evaluate on a dirty dataset you will not be able to tell a weak product from a good one fed bad inputs.
Work the three sorted passes rather than trying to review everything, since duplicates cluster and those passes surface most of them for a fraction of the effort. Merge the clear matches, leave the ambiguous ones flagged rather than guessing, and record the count so you know whether the problem is regenerating.
Because a text to a landline usually fails silently rather than bouncing, so your delivery report stays healthy while a share of your list receives nothing. Practices commonly discover after go-live that they have been failing to reach a tenth or more of their patients for months with no visible error anywhere.
Then the category of product you can buy is narrower, and knowing that before you shop saves a wasted evaluation. Reading-only integrations still support capture, routing and messaging, and those are the majority of the value for most practices. What they cannot support is anything writing directly into your diary.

Written by

Muhammad Qasim Hammad

Founder, Velaire Health

Builds AI front-office systems for medical and aesthetic practices. Posts here start from published sources rather than vendor claims, and every number links back to where it came from.

Keep reading

More from the journal.

Start here

What is an AI front office?

The plain-language explainer: what it does, what it deliberately doesn't, and how it compares with voicemail, an answering service, a phone tree, and hiring.

Read the guide
Let’s make room for better care

Your next caller
could be your next patient.

See how Velaire would handle the calls your practice misses. A personal walkthrough, built around your questions.

Let’s talk about your practice