ML Engineer Journey: Clinical Trial Patient Matching at Scale

Persona: Jordan Park, ML Engineer at Pinnacle Health System Goal: Build an automated system to match eligible patients to active clinical trials, improving enrollment rates from 8.3% to 15% Primary Workbenches: ML Workbench, Pipeline Service

Background

Clinical trial enrollment is a critical bottleneck in healthcare. Pinnacle Health runs 500 active clinical trials across its 12 hospitals, but only 8.3% of eligible patients are ever identified and enrolled. Most trial matching is done manually by clinical research coordinators who review patient charts one at a time -- a process that misses the vast majority of eligible candidates.

Jordan Park has been tasked with building an automated matching engine that screens every new admission against active trial eligibility criteria, scores patient-trial matches, and alerts research coordinators to high-confidence matches in real time.

Stage 1: Ingestion

Jordan begins by connecting the clinical trial management system and supplementary data sources through the Data Workbench ingestion panel.

Clinical Trial Management System (CTMS)

The CTMS runs on PostgreSQL and contains trial definitions, eligibility criteria, site assignments, and enrollment records:

{
  "source_type": "postgresql",
  "connection_name": "pinnacle_ctms",
  "config": {
    "host": "ctms-db.pinnaclehealth.org",
    "port": 5432,
    "database": "clinical_trials",
    "replication_method": "standard",
    "tables": [
      "trials",
      "eligibility_criteria",
      "trial_sites",
      "enrollments",
      "screening_logs",
      "protocol_amendments"
    ],
    "schema": "public"
  },
  "schedule": {
    "frequency": "daily",
    "time": "02:00"
  },
  "destination": {
    "schema": "ctms_data"
  }
}

ClinicalTrials.gov API

Jordan also pulls external trial listings to ensure Pinnacle Health is aware of all trials accepting patients at their sites:

{
  "source_type": "http_api",
  "connection_name": "clinicaltrials_gov",
  "config": {
    "base_url": "https://clinicaltrials.gov/api/v2/studies",
    "auth_type": "none",
    "request_params": {
      "query.locn": "Pinnacle Health",
      "filter.overallStatus": "RECRUITING",
      "fields": "NCTId,BriefTitle,Condition,InterventionName,EligibilityCriteria,Phase,EnrollmentCount,LocationFacility",
      "pageSize": 100
    },
    "pagination": {
      "type": "token",
      "token_field": "nextPageToken"
    }
  },
  "schedule": {
    "frequency": "weekly"
  },
  "destination": {
    "schema": "external_trials"
  }
}

De-Identification Pipeline

All patient data flowing into the trial matching system passes through a de-identification layer before being used for matching. Only after a match is confirmed does a coordinator access the identified record:

Data Source Flow for Trial Matching:
====================================

  EHR Data (PHI)              CTMS Data (non-PHI)
       │                            │
       ▼                            │
  ┌─────────────────┐              │
  │ De-Identification│              │
  │ Pipeline         │              │
  │                  │              │
  │ - Tokenize MRN   │              │
  │ - Hash names     │              │
  │ - Generalize DOB │              │
  │ - Retain clinical│              │
  │   codes (ICD-10, │              │
  │   LOINC, NDC)    │              │
  └────────┬─────────┘              │
           │                        │
           ▼                        ▼
  ┌──────────────────────────────────────┐
  │     Matching Engine (de-identified)  │
  │     Patient tokens + clinical codes  │
  │     matched against trial criteria   │
  └──────────────────┬───────────────────┘
                     │
                     ▼ match alerts (token only)
  ┌──────────────────────────────────────┐
  │  Coordinator Re-Identification       │
  │  (authorized access, HIPAA audited)  │
  └──────────────────────────────────────┘

Source	Records	Sync Status
CTMS (trials, criteria)	500 trials, 12K criteria	Daily at 02:00
CTMS (enrollments)	50K records	Daily at 02:00
ClinicalTrials.gov	1,200 regional trials	Weekly refresh
EHR (de-identified clinical data)	200K patients	Continuous (15 min)

Stage 2: Discovery

Jordan explores the trial and patient data in the Data Workbench Catalog to understand data quality and mapping requirements.

Eligibility Criteria Analysis

Trial eligibility criteria are stored as semi-structured text. Jordan profiles the criteria to understand what clinical data elements are referenced:

-- Top referenced clinical concepts in eligibility criteria
SELECT
    concept_type,
    concept_code,
    concept_name,
    COUNT(DISTINCT trial_id) AS trials_referencing,
    SUM(CASE WHEN criterion_type = 'INCLUSION' THEN 1 ELSE 0 END) AS inclusion_count,
    SUM(CASE WHEN criterion_type = 'EXCLUSION' THEN 1 ELSE 0 END) AS exclusion_count
FROM ctms_data.parsed_eligibility_criteria
GROUP BY concept_type, concept_code, concept_name
ORDER BY trials_referencing DESC
LIMIT 10;

Concept Type	Code	Name	Trials	Inclusion	Exclusion
Diagnosis	ICD-10	Malignant neoplasm categories	187	142	45
Age	--	Age range criterion	423	423	0
Lab Value	LOINC	eGFR (renal function)	156	34	122
Medication	NDC	Immunosuppressants	98	12	86
Lab Value	LOINC	Hemoglobin A1c	87	45	42
Diagnosis	ICD-10	Heart failure (I50.x)	76	62	14
Procedure	CPT	Prior surgery categories	72	8	64
Lab Value	LOINC	Platelet count	68	12	56
Medication	NDC	Anticoagulants	64	18	46
Diagnosis	ICD-10	Diabetes mellitus (E11.x)	61	48	13

NDC Code Inconsistency Discovery

Jordan discovers that medication NDC codes are inconsistent across the two EHR systems:

Data Quality Finding: NDC Code Inconsistencies
================================================

Epic hospitals:   NDC-11 format (5-4-2):  00006-0061-31
Cerner hospitals: NDC-10 format (no dashes): 0000606131

Impact: 34% of medication matching queries return different results
        depending on which EHR system the patient is in.

Resolution: Built NDC normalization mapping in the catalog:
  - Standardize to NDC-11 format
  - Map to RxNorm CUI for cross-system matching
  - 98.7% of NDC codes successfully mapped
  - 1.3% flagged for manual review (discontinued drugs)

Ontology Mapping

Jordan builds a mapping between trial eligibility concepts and available EHR data elements using the Ontology Service:

Trial Criterion	EHR Data Element	Mapping Confidence
"Age 18-75"	`patients.birth_date` (computed)	Exact
"Diagnosis of HFrEF"	`conditions.icd10_code IN ('I50.2x')`	Exact
"eGFR >= 30"	`lab_results WHERE loinc_code = '33914-3'`	Exact
"No prior immunotherapy"	`medications WHERE drug_class = 'immunotherapy'`	Fuzzy (85%)
"ECOG performance status 0-2"	`clinical_notes` (NLP extraction)	NLP (78%)
"Life expectancy > 6 months"	Not directly available	Proxy required

Stage 3: Query

Jordan builds the eligibility matching queries that translate trial criteria into SQL predicates against the unified clinical data.

Criteria-to-SQL Translation

-- Example: Match patients for NCT-2025-CARDIO-001
-- "Phase III Trial of Novel Heart Failure Treatment"
-- Inclusion: Adults 18-80, diagnosis of HFrEF (LVEF <= 40%),
--            NYHA Class II-III, eGFR >= 30
-- Exclusion: Prior cardiac transplant, current immunosuppressant use,
--            hemoglobin < 9 g/dL
 
WITH eligible_patients AS (
    SELECT DISTINCT
        p.patient_token,
        p.age,
        p.gender,
        e.facility_id,
        lvef.result_value AS lvef_pct,
        egfr.result_value AS egfr_value,
        hgb.result_value AS hemoglobin
    FROM clinical_deidentified.patients p
    -- Active encounter in the last 90 days
    JOIN clinical_deidentified.encounters e
        ON p.patient_token = e.patient_token
        AND e.admit_date >= CURRENT_DATE - INTERVAL '90' DAY
 
    -- Inclusion: Diagnosis of HFrEF
    JOIN clinical_deidentified.conditions c
        ON e.encounter_id = c.encounter_id
        AND c.icd10_code LIKE 'I50.2%'
 
    -- Inclusion: LVEF <= 40% (most recent)
    JOIN LATERAL (
        SELECT result_value
        FROM clinical_deidentified.lab_results lr
        WHERE lr.patient_token = p.patient_token
          AND lr.loinc_code = '10230-1'  -- LVEF
        ORDER BY lr.collected_at DESC LIMIT 1
    ) lvef ON TRUE
 
    -- Inclusion: eGFR >= 30
    JOIN LATERAL (
        SELECT result_value
        FROM clinical_deidentified.lab_results lr
        WHERE lr.patient_token = p.patient_token
          AND lr.loinc_code = '33914-3'  -- eGFR
        ORDER BY lr.collected_at DESC LIMIT 1
    ) egfr ON TRUE
 
    -- For exclusion check: hemoglobin
    LEFT JOIN LATERAL (
        SELECT result_value
        FROM clinical_deidentified.lab_results lr
        WHERE lr.patient_token = p.patient_token
          AND lr.loinc_code = '718-7'  -- Hemoglobin
        ORDER BY lr.collected_at DESC LIMIT 1
    ) hgb ON TRUE
 
    WHERE
        -- Inclusion: Age 18-80
        p.age BETWEEN 18 AND 80
        -- Inclusion: LVEF <= 40%
        AND CAST(lvef.result_value AS DOUBLE) <= 40.0
        -- Inclusion: eGFR >= 30
        AND CAST(egfr.result_value AS DOUBLE) >= 30.0
),
 
excluded_patients AS (
    -- Exclusion: Prior cardiac transplant
    SELECT DISTINCT patient_token
    FROM clinical_deidentified.procedures
    WHERE cpt_code IN ('33945', '33940')  -- Heart transplant CPT codes
 
    UNION
 
    -- Exclusion: Current immunosuppressant use
    SELECT DISTINCT patient_token
    FROM clinical_deidentified.medications
    WHERE drug_class = 'IMMUNOSUPPRESSANT'
      AND end_date >= CURRENT_DATE
)
 
SELECT
    ep.*,
    CASE WHEN ep.hemoglobin IS NOT NULL
              AND CAST(ep.hemoglobin AS DOUBLE) < 9.0
         THEN 'EXCLUDED_LOW_HEMOGLOBIN'
         ELSE 'ELIGIBLE'
    END AS final_status
FROM eligible_patients ep
WHERE ep.patient_token NOT IN (SELECT patient_token FROM excluded_patients);

Federated Query Across EHR and CTMS

-- Match eligible patients to all recruiting trials at their facility
SELECT
    t.trial_id,
    t.nct_number,
    t.brief_title,
    t.phase,
    t.therapeutic_area,
    ts.facility_id,
    COUNT(DISTINCT ep.patient_token) AS eligible_patients,
    t.target_enrollment - COALESCE(enrolled.count, 0) AS enrollment_gap,
    ROUND(100.0 * COUNT(DISTINCT ep.patient_token)
        / NULLIF(t.target_enrollment - COALESCE(enrolled.count, 0), 0), 1)
        AS pipeline_coverage_pct
FROM ctms_data.trials t
JOIN ctms_data.trial_sites ts ON t.trial_id = ts.trial_id
JOIN matching_results.eligible_patients ep
    ON ep.facility_id = ts.facility_id
    AND ep.trial_id = t.trial_id
LEFT JOIN (
    SELECT trial_id, COUNT(*) AS count
    FROM ctms_data.enrollments
    WHERE status = 'ENROLLED'
    GROUP BY trial_id
) enrolled ON t.trial_id = enrolled.trial_id
WHERE t.status = 'RECRUITING'
GROUP BY t.trial_id, t.nct_number, t.brief_title, t.phase,
         t.therapeutic_area, ts.facility_id, t.target_enrollment,
         enrolled.count
ORDER BY enrollment_gap DESC;

Stage 4: Orchestration

Jordan builds the matching pipeline using the Pipeline Service with daily batch processing for new admissions and a weekly full refresh.

Trial Matching Pipeline

{
  "pipeline_name": "clinical_trial_matching",
  "schedule": "0 5 * * *",
  "description": "Daily patient-trial matching for new admissions and updated criteria",
  "steps": [
    {
      "step_id": "parse_eligibility_criteria",
      "type": "python_transform",
      "script": "criteria_parser.py",
      "description": "Parse free-text criteria into structured predicates",
      "config": {
        "nlp_model": "clinical_criteria_parser_v2",
        "output_format": "structured_predicates"
      },
      "destination": "matching.parsed_criteria"
    },
    {
      "step_id": "screen_new_admissions",
      "type": "sql_transform",
      "depends_on": ["parse_eligibility_criteria"],
      "query_ref": "sql/patient_screening.sql",
      "description": "Screen patients admitted in last 24h against all active trial criteria",
      "destination": "matching.daily_screen_results",
      "write_mode": "append"
    },
    {
      "step_id": "compute_match_scores",
      "type": "python_transform",
      "depends_on": ["screen_new_admissions"],
      "script": "match_scoring.py",
      "description": "Score patient-trial matches on completeness and confidence",
      "config": {
        "scoring_model": "trial_match_scorer_v3",
        "min_confidence": 0.60
      },
      "destination": "matching.scored_matches"
    },
    {
      "step_id": "quality_validation",
      "type": "data_quality",
      "depends_on": ["compute_match_scores"],
      "suite": "trial_matching_quality",
      "checks": [
        { "type": "not_null", "columns": ["patient_token", "trial_id", "match_score"] },
        { "type": "range", "column": "match_score", "min": 0.0, "max": 1.0 },
        { "type": "referential_integrity", "column": "trial_id", "reference": "ctms_data.trials.trial_id" },
        { "type": "row_count_change", "max_pct_change": 50, "alert_on_breach": true }
      ],
      "on_failure": "alert_and_quarantine"
    },
    {
      "step_id": "notify_coordinators",
      "type": "notification",
      "depends_on": ["quality_validation"],
      "config": {
        "channel": "ehr_inbox",
        "filter": "match_score >= 0.75",
        "template": "trial_match_alert",
        "group_by": "facility_id"
      }
    },
    {
      "step_id": "audit_log",
      "type": "audit_event",
      "depends_on": ["notify_coordinators"],
      "event_type": "trial_matching_run",
      "details": {
        "pipeline": "clinical_trial_matching",
        "data_accessed": ["patients", "encounters", "conditions", "medications", "lab_results"],
        "purpose": "clinical_trial_enrollment_optimization",
        "fda_21cfr11": true
      }
    }
  ],
  "weekly_full_refresh": {
    "schedule": "0 1 * * SUN",
    "description": "Full refresh of all patient-trial matches",
    "override_step": "screen_new_admissions",
    "override_config": {
      "query_ref": "sql/patient_screening_full.sql",
      "write_mode": "overwrite"
    }
  }
}

Pipeline Monitoring

Pipeline: clinical_trial_matching
Run: 2025-11-15 05:00:00 UTC  (Daily)
Status: COMPLETED

  parse_eligibility_criteria  ████████████████████ 100%  [2m 14s]  487 trials parsed
  screen_new_admissions       ████████████████████ 100%  [8m 37s]  1,842 new admissions screened
  compute_match_scores        ████████████████████ 100%  [4m 12s]  312 matches scored
  quality_validation          ████████████████████ 100%  [0m 28s]  0 failures
  notify_coordinators         ████████████████████ 100%  [0m 15s]  47 alerts sent
  audit_log                   ████████████████████ 100%  [0m 03s]  1 event logged

Summary:
  New admissions screened:   1,842
  Patients matched to >= 1 trial: 312  (16.9%)
  High-confidence matches (>= 0.75):  47
  Alerts sent to coordinators:  47 (across 9 facilities)

Stage 5: Analysis

Jordan validates the matching engine's accuracy against manual chart review.

Accuracy Validation

Jordan runs a validation study where 200 patient-trial matches are independently reviewed by clinical research coordinators:

Matching Engine Validation Study
==================================

Ground truth: Manual chart review by 3 clinical research coordinators
Sample: 200 randomly selected patient-trial matches

                    Manual Review
                    Eligible    Not Eligible
  Engine    ┌──────────────┬──────────────┐
  Match     │  TP = 178    │  FP = 22     │  Precision = 0.89
            ├──────────────┼──────────────┤
  No Match  │  FN = 48     │  TN = 1,574  │
            └──────────────┴──────────────┘
                Recall = 0.79

  F1 Score:    0.84
  Precision:   0.89  (89% of engine matches confirmed by chart review)
  Recall:      0.79  (79% of truly eligible patients identified)

Common Exclusion Reason Analysis

SELECT
    exclusion_reason,
    COUNT(*) AS patients_excluded,
    ROUND(100.0 * COUNT(*) / SUM(COUNT(*)) OVER (), 1) AS pct_of_exclusions
FROM matching.exclusion_log
WHERE run_date = CURRENT_DATE
GROUP BY exclusion_reason
ORDER BY patients_excluded DESC;

Exclusion Reason	Patients	% of Exclusions
Age outside range	4,218	28.4%
Missing required lab value	3,102	20.9%
Concurrent medication conflict	2,456	16.5%
Prior treatment exclusion	1,834	12.3%
Diagnosis not confirmed	1,502	10.1%
eGFR below threshold	987	6.6%
ECOG status unavailable	772	5.2%

Demographic Bias Check

Demographic	Eligible Population	Matched Rate	Enrolled Rate	Notes
White	58%	17.2%	9.1%	Baseline
Black	22%	16.8%	7.4%	Enrollment gap -- investigate coordinator bias
Hispanic	14%	15.9%	6.8%	Language barrier in consent process
Asian	6%	17.1%	8.9%	Within range
Male	52%	17.4%	8.8%	Baseline
Female	48%	16.2%	7.6%	Lower matching -- some cardiac trials male-skewed

Stage 6: Productionization

Jordan deploys the matching engine to Ray Serve through the ML Workbench and integrates with the EHR workflow.

Ray Serve Deployment

{
  "deployment_name": "trial_matching_engine",
  "model_name": "trial_match_scorer_v3",
  "serving_config": {
    "runtime": "ray_serve",
    "num_replicas": 2,
    "max_concurrent_queries": 50,
    "ray_actor_options": {
      "num_cpus": 2,
      "memory": 4096
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 4,
      "target_num_ongoing_requests_per_replica": 10
    }
  },
  "endpoints": [
    {
      "path": "/api/v1/trial-match/batch",
      "method": "POST",
      "description": "Batch match patients against all active trials",
      "input_schema": {
        "patient_tokens": ["string"],
        "facility_id": "string"
      }
    },
    {
      "path": "/api/v1/trial-match/realtime",
      "method": "POST",
      "description": "Real-time match for a single patient on admission",
      "input_schema": {
        "patient_token": "string",
        "encounter_id": "string",
        "facility_id": "string"
      },
      "sla": {
        "p50_latency_ms": 200,
        "p99_latency_ms": 1000
      }
    }
  ],
  "health_check": {
    "path": "/health",
    "interval_seconds": 30
  }
}

EHR Integration Flow

Patient Admission Workflow with Trial Matching
================================================

  Patient admitted ──▶ ADT message (HL7v2)
                            │
                            ▼
                  ┌──────────────────┐
                  │ Real-time match  │
                  │ API endpoint     │
                  │ (< 1 sec SLA)    │
                  └────────┬─────────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
          No match    Low confidence   High confidence
          (do nothing) (0.60 - 0.74)   (>= 0.75)
                           │            │
                           ▼            ▼
                     Weekly batch   Immediate alert
                     review list   to coordinator
                                        │
                                        ▼
                              ┌──────────────────┐
                              │ Coordinator sees │
                              │ match in EHR     │
                              │ patient chart    │
                              │                  │
                              │ Trial: NCT-xxxx  │
                              │ Match score: 0.87│
                              │ Criteria met: 8/9│
                              │ Missing: ECOG    │
                              └──────────────────┘

Stage 7: Feedback

Jordan monitors the matching engine's performance and enrollment outcomes.

Key Metrics Dashboard

Trial Matching Engine -- November 2025 Performance
=====================================================

Matching Metrics:
  Daily patients screened:           1,842 avg
  Daily matches (>= 0.60):          312 avg    (17.0% match rate)
  Daily high-confidence (>= 0.75):  47 avg     (2.6% of screened)
  Real-time API latency (p50):      142 ms
  Real-time API latency (p99):      487 ms

Enrollment Funnel:
  Patients matched:    9,360 / month
  Coordinator reviewed: 1,410 / month  (15.1% review rate)
  Patient approached:    823 / month   (58.4% of reviewed)
  Consented:             412 / month   (50.1% of approached)
  Enrolled:              348 / month   (84.5% of consented)

  Overall: matched → enrolled = 3.7%
  Previous (manual): identified → enrolled = 8.3% of fewer patients

False Positive Rate (coordinator-reported):
  Week 1:  11.2%
  Week 2:   9.8%
  Week 3:   8.4%
  Week 4:   7.1%  (improving with feedback loop)

Weekly Coordinator Feedback

{
  "feedback_collection": {
    "schedule": "weekly",
    "method": "coordinator_review_form",
    "metrics_tracked": [
      "match_accuracy_rating",
      "time_saved_per_match",
      "false_positive_reason",
      "suggested_criteria_improvements"
    ],
    "sample_feedback": {
      "coordinator": "Dr. Lisa Nguyen, Site PI - Pinnacle Downtown",
      "week": "2025-11-10",
      "matches_reviewed": 34,
      "accurate_matches": 31,
      "feedback": "Engine correctly identified 3 patients for our HF trial that we would have missed. NDC matching for excluded medications is still flagging patients on generic equivalents incorrectly.",
      "action_taken": "Updated NDC-to-RxNorm mapping for generic equivalents"
    }
  }
}

Stage 8: Experimentation

Jordan runs structured experiments to improve matching accuracy and enrollment.

NLP Criteria Parsing Experiment

Jordan compares rule-based vs transformer-based parsing of eligibility criteria:

Approach	Precision	Recall	F1	Parse Time (per trial)	Notes
Rule-based (regex + patterns)	0.92	0.71	0.80	0.3s	Good for structured criteria
Clinical BERT fine-tuned	0.88	0.84	0.86	1.2s	Better on free-text criteria
Hybrid (rules first, BERT fallback)	0.91	0.83	0.87	0.6s	Best balance

Decision: Deploy hybrid approach -- rule-based for standard criteria patterns (age, lab ranges, diagnosis codes), BERT for complex free-text criteria (prior treatment history, performance status).

Matching Algorithm Comparison

Algorithm	Precision	Recall	F1	Computation	Notes
Exact match (SQL predicates)	0.95	0.62	0.75	Fast	Misses fuzzy criteria
Fuzzy match (ICD hierarchy)	0.87	0.79	0.83	Medium	Better recall, some noise
Learned scoring (RF on features)	0.89	0.76	0.82	Medium	Good but needs labeled data
Hybrid (exact + fuzzy + scoring)	0.89	0.82	0.85	Medium	Production candidate

Enrollment Rate Impact

Clinical Trial Enrollment Impact Assessment
=============================================

Period: Q2 2025 (pre-engine) vs Q4 2025 (post-engine)

                              Pre-Engine    Post-Engine    Change
Eligible patients identified:    2,340         9,360       +300%
Coordinators notified:             890         1,410        +58%
Patients approached:               534           823        +54%
Patients enrolled:                 194           348        +79%

Enrollment rate (identified → enrolled):
  Pre-engine:   8.3%
  Post-engine: 11.1%

Key improvement drivers:
  1. Automated screening:   +300% eligible patients identified
  2. Real-time alerts:      Coordinators reach patients during admission
  3. Match quality:         Higher confidence reduces wasted outreach
  4. NDC normalization:     Fixed 34% medication matching gap

Target (15% enrollment rate): On track for Q2 2026 with planned improvements
  - Add NLP-extracted ECOG scores (+5% recall expected)
  - Integrate social determinant screening (+3% enrollment conversion)
  - Multi-language consent materials (+2% for Hispanic population)

Related Walkthroughs

Data Scientist Journey -- Dr. Maya Chen builds the readmission prediction model
BI Lead Journey -- Aisha Williams creates the operations command center
Executive Leadership Journey -- Dr. Robert Kim uses AI for clinical strategy
Healthcare Overview -- Pinnacle Health datasets, KPIs, and compliance framework

Data Scientist Journey BI Lead Journey