MA
Portfolio

Area CData Foundation

DICOM De-Identification Pipeline

HIPAA Safe Harbor · DICOM PS3.15

Removing identity without destroying research value — the legal gate for every model.

parallel, resumable
32-way
parallel, resumable
files provenance-stamped
100%
files provenance-stamped

01/Problem

Clinical DICOM files are saturated with PHI — in tags, private vendor fields, and sometimes burned into pixels. The hard part is removing identity without destroying research utility.

02/Approach

A versioned, recipe-driven pipeline aligned to DICOM PS3.15 and HIPAA Safe Harbor. Deterministic pseudonymization keeps cross-study patient linkage without exposing identity; birth dates generalize to month and ages bin to 5 years. Defense in depth: regex sweeps over private tags, overlay removal, and burned-in pixel screening flagged for human review. Runs 32-way parallel with idempotent resume and per-file provenance.

03/Outcome

  • Produced the compliant datasets underpinning the entire model-development program.
  • Longitudinal linkage and analytic value preserved while direct identifiers are eliminated.
  • Auditable by design: versioned recipe plus per-file provenance.

04/My role

  • Introduced HIPAA-compliant ML data governance as a prerequisite for enterprise onboarding — a commercial unblock, not only compliance.
  • Required deterministic pseudonymization over simple redaction to keep longitudinal data usable.
  • Set burned-in pixel PHI handling as flag-and-review.

Stack

pydicomdeidPython multiprocessing