The structured de-identification pass ran without errors.
Names: removed. SSNs: removed. DOBs: truncated to year. ZIP codes: 3-digit prefix, population-verified.
Then a reviewer opened the clinical_notes column.
Row 47:
"Patient is a 47-year-old female referred by Dr. Sarah Chen from UCSF Medical Center. She presented with symptoms consistent with the recent outbreak in her San Jose workplace. Previous records from her 2019 treatment at Valley Medical Group were reviewed."
Provider name. Institution name. City. Workplace cluster reference. Institution name from a prior record. All in free text. None of it touched by the structured de-identification pass.
This is the free-text PHI problem. It is not rare. It is the rule.
TL;DR
Structured de-identification tools remove PHI from named columns — patient_name, ssn, zip_code. They do not read the content of free-text fields like clinical_notes, visit_summary, reason_for_visit, or comments. Clinical free-text routinely contains provider names, institution names, geographic references, dates in prose form, and contextual details that re-identify patients even without any of the 18 Safe Harbor identifiers appearing explicitly. Automated scanning — pattern matching for names, dates, locations, and known entities — reduces residual risk but cannot detect all contextual PHI. Manual review by a qualified reviewer remains the recognized standard for high-stakes external data releases.
Methodology note (May 2026): HIPAA de-identification standards are at 45 CFR §164.514(b). Guidance on free-text de-identification comes from HHS publications and the clinical NLP literature. Manual review recommendations reflect standard practice in healthcare privacy; no automated tool should be represented as providing complete PHI removal from clinical narratives.
For the complete Safe Harbor workflow across all 18 identifiers, see our complete PHI de-identification guide.
⚖️ NOT LEGAL ADVICE — This post covers HIPAA Safe Harbor free-text PHI de-identification for informational purposes only. Whether your specific data and process satisfy Safe Harbor depends on your data, use case, and organizational context. Consult qualified legal and compliance counsel before making de-identification or sharing decisions for regulated data.
Table of Contents
- Why Structured De-Identification Misses Free-Text PHI
- Common PHI Patterns in Clinical Notes
- Reality: A Row That Passes Structured De-Id and Still Contains PHI
- Categories of Free-Text PHI Risk
- How Automated Scanning Works — and Where It Fails
- Step-by-Step: Scan Free-Text Columns with SplitForge
- When Manual Review Is Non-Negotiable
- Limitations
- FAQ
- Conclusion
Why Structured De-Identification Misses Free-Text PHI
Structured de-identification works by column. The tool sees patient_name → removes it. It sees date_of_birth → strips month and day. It sees zip_code → truncates.
The tool does not read the value inside clinical_notes and apply judgment about whether that prose contains a person's name. Most de-identification pipelines, including automated Safe Harbor tools, are column-based, not content-based.
This is by design. Structured de-id is fast, auditable, and effective against the 18 Safe Harbor identifier types when those identifiers appear in dedicated columns. The problem is that clinical text was not written to respect column boundaries. Clinicians write notes in natural language. Natural language contains names, places, dates, and context — woven together in ways no column-level tool is built to detect.
Common PHI Patterns in Clinical Notes
Direct identifiers in prose:
- Provider names: "per Dr. Martinez's recommendation," "referred by Dr. Sarah Chen"
- Institution names: "records from UCSF Medical Center," "previous admission at Valley Medical Group"
- Dates in prose: "presented in March of last year," "started treatment on the 14th"
- Geographic references: "lives near the Riverside County facility," "workplace in San Jose"
Quasi-identifier combinations in narrative:
- "47-year-old Hispanic female with a history of..." — age, sex, ethnicity in one sentence
- "resides in a rural area served by the 895 prefix" — explicit geographic hint
- "recently discharged from the city's only Level I trauma center" — facility identifier by implication
Cross-record linkage hooks:
- "previously treated at [named clinic] in 2019" — institution + year creates a linkable data point
- "referred by the same cardiologist who treated her sister" — relationship information
- "long-term patient of this practice since childhood" — duration implies approximate DOB range
Condition-specific identifiers:
- "attending the Parkview HIV clinic" — diagnosis + institution
- "enrolled in the county methadone program" — diagnosis + program type + county
- Names of conditions that affect small enough populations to re-identify by diagnosis alone
Reality: A Row That Passes Structured De-Id and Still Contains PHI
Input row (before any de-identification):
patient_id | name | dob | zip | clinical_notes
PT-7829 | Jane Doe | 1977-04-12 | 94110 | 47-year-old female referred by Dr. Sarah Chen
| | | | from UCSF Medical Center. Workplace exposure
| | | | in San Jose. Prior records from Valley Medical
| | | | Group 2019 reviewed.
After structured de-identification (column-level only):
patient_id | name | dob | zip | clinical_notes
PT-7829 | [redact]| 1977 | 941 | 47-year-old female referred by Dr. Sarah Chen
| | | | from UCSF Medical Center. Workplace exposure
| | | | in San Jose. Prior records from Valley Medical
| | | | Group 2019 reviewed.
The name, dob, and zip columns are compliant. The clinical_notes column still contains:
- A provider name (
Dr. Sarah Chen) - An institution name (
UCSF Medical Center) - A city (
San Jose) - A second institution name (
Valley Medical Group) - A year in prose (
2019) — linked to treatment history
A structured de-id pass against column definitions will not touch any of this. The row looks "done." It is not.
Categories of Free-Text PHI Risk
High confidence detectable (pattern matching works well):
- Social Security numbers in text (
123-45-6789,SSN: 123456789) - Phone numbers in prose (
call at 555-203-4812) - Email addresses (
[email protected]) - Dates in standard formats (
March 14, 2023,3/14/2023) - ZIP codes in prose (
living in the 94107 area)
Moderate confidence detectable (pattern matching partially effective):
- Common provider name patterns (
Dr. [Name],Dr. [Last Name]) - Major institution names (known entity lists)
- City and county names (geographic entity lists)
Low confidence detectable (automated tools frequently miss):
- Provider names without "Dr." prefix
- Facility references by implication (
the county's only pediatric hospital) - Dates in prose form (
last spring,the year before diagnosis) - Quasi-identifier combinations without explicit identifiers
- Contextual re-identification hooks (
long-term patient referred by her sister's oncologist)
How Automated Scanning Works — and Where It Fails
Automated free-text PHI scanning uses pattern matching, named entity recognition (NER), and rule-based detection.
What pattern matching can do:
- Flag text matching regex patterns for SSNs, phone numbers, email addresses, and numeric dates
- Flag known entity types: sequences matching
Dr. [Capitalized Name] - Flag geographic terms from known entity lists (major city names, state names)
- Flag years and date-like sequences
Where pattern matching fails:
- Provider names without prefixes (Jane Smith who happens to be a clinician, referenced by full name in a note)
- Facility references by description rather than name ("the rural critical access hospital serving the eastern valley")
- Contextual combinations that are only identifying together — each term benign in isolation
- Dates expressed in relative or colloquial form ("earlier this year," "two winters ago")
- Rare names or minority-language names that fall outside training data
The honest baseline: Automated scanning reduces the density of detectable PHI in free-text. It does not eliminate it. A note that passes automated scanning has fewer obvious PHI patterns — not zero PHI risk.
Step-by-Step: Scan Free-Text Columns with SplitForge
1. Identify all free-text columns
Audit column names for: notes, comments, summary, description, narrative, reason, text, history, findings, assessment. These warrant content-level review regardless of whether structured de-id passed.
2. Load the CSV in SplitForge Data Validator
Open SplitForge Data Masking and load your file locally. No upload — processing happens in the browser.
3. Flag PHI-pattern matches
Run a scan against free-text columns. The tool will flag:
- Sequences matching SSN, phone, email, and date patterns
- Common provider name patterns
- Known geographic entity matches
4. Review flagged rows
For each flagged row, read the full text value. Determine whether the flagged term is:
- A direct PHI element → redact or replace
- A false positive (common word or name that is not PHI in context) → retain
5. Redact or replace identified PHI
Replace PHI tokens with category labels: [PROVIDER_NAME], [INSTITUTION], [CITY], [DATE]. Do not delete notes entirely unless the content cannot be safely de-identified — losing clinical context has research and care quality costs.
6. Document residual risk
For data destined for external release, document:
- Which free-text columns were scanned
- What scanning method was used
- Whether manual review was performed and by whom
- The residual risk assessment
When Manual Review Is Non-Negotiable
Automated scanning reduces risk. It does not satisfy the review standard for high-stakes external releases.
Manual review by a qualified privacy reviewer is the recognized standard when:
- The dataset is intended for publication or public repository
- The dataset includes clinical narratives for rare diseases or small populations
- The dataset includes notes from high-risk specialty areas (HIV, substance use, mental health, oncology)
- The de-identified data will be shared with external research partners without a BAA
- The IRB or privacy officer has flagged the free-text columns as sensitive
"We ran automated scanning" is not equivalent to "we reviewed the free-text columns." For high-stakes releases, both are expected. Document what was done, who performed the review, and the date.
Limitations
Satisfying Safe Harbor removes data from HIPAA's definition of PHI; it does not make the data anonymous. De-identified ≠ anonymized — residual re-identification risk can remain, and "actual knowledge" that the remaining data could identify someone re-activates HIPAA obligations.
Automated tools cannot detect all contextual PHI. Contextual identifiers — references that are only identifying in combination, or that require world knowledge to interpret — are outside the reach of pattern-matching tools. This is a fundamental limitation, not a gap that will be closed by a better algorithm.
SplitForge's scanning flags patterns, not PHI with certainty. A flagged row may be a false positive. An unflagged row may still contain PHI. The scanning output is a starting point for review, not a certification of compliance.
Removing notes entirely is not always the right choice. Stripping all free-text eliminates the re-identification risk from those fields, but also destroys clinical research value. The better approach is selective redaction of identifiable terms — which requires human judgment.
Scanning does not replace Expert Determination for high-value datasets. For datasets where re-identification risk must be formally quantified, Expert Determination by a qualified statistician is the appropriate method. See Safe Harbor vs Expert Determination.
FAQ
Conclusion
Structured de-identification solves a column problem. Clinical notes are a content problem. Those two problems require different tools and different standards of review.
The scenario at the top of this guide — a row that passes structured de-id and still contains a provider name, an institution, a city, and a prior treatment site — is not a rare edge case. It is representative of what clinical text contains. Every dataset with a free-text notes column warrants content-level review.
Automated scanning helps. It catches SSNs in prose, dates in standard formats, common provider name patterns. It does not catch everything. A note that contains no flagged patterns is not guaranteed PHI-free — it is only pattern-free.
For external releases, especially of data involving small populations, rare conditions, or specialty care, manual review by a qualified privacy reviewer is the expected standard. Document what was done, who did it, and what risk assessment was reached.
Next: Pre-Sharing PHI CSV Checklist → | Validate PHI Removal in Patient CSV Files →