Portfolio
Area C—Data Foundation
DICOM De-Identification Pipeline
HIPAA Safe Harbor · DICOM PS3.15
Removing identity without destroying research value — the legal gate for every model.
- parallel, resumable
- 32-way
- parallel, resumable
- files provenance-stamped
- 100%
- files provenance-stamped
01/Problem
Clinical DICOM files are saturated with PHI — in tags, private vendor fields, and sometimes burned into pixels. The hard part is removing identity without destroying research utility.
02/Approach
A versioned, recipe-driven pipeline aligned to DICOM PS3.15 and HIPAA Safe Harbor. Deterministic pseudonymization keeps cross-study patient linkage without exposing identity; birth dates generalize to month and ages bin to 5 years. Defense in depth: regex sweeps over private tags, overlay removal, and burned-in pixel screening flagged for human review. Runs 32-way parallel with idempotent resume and per-file provenance.
03/Outcome
- Produced the compliant datasets underpinning the entire model-development program.
- Longitudinal linkage and analytic value preserved while direct identifiers are eliminated.
- Auditable by design: versioned recipe plus per-file provenance.
04/My role
- Introduced HIPAA-compliant ML data governance as a prerequisite for enterprise onboarding — a commercial unblock, not only compliance.
- Required deterministic pseudonymization over simple redaction to keep longitudinal data usable.
- Set burned-in pixel PHI handling as flag-and-review.
Stack
pydicomdeidPython multiprocessing