Pruthvi Nag Gattu | Data Engineer & Distributed Systems

PySpark Bronze Ingest

Governed Bronze/Silver/Gold Lakehouse on Apache Iceberg, dbt-DuckDB, and Snowflake with strict contract enforcement.

• 7.2x Faster Queries vs SQLite, 4.8x vs Pandas
• 99.8% Leakage Reduction via Quarantine Diverter
• Strict Contract Enforcement on Analytical Marts

Code Repository Technical Paper (PDF)

PySpark Core Engine

Declarative YAML DSL to dynamic Airflow DAGs and PySpark execution jobs across 12 hospital facilities.

• Onboarding reduced from 3 weeks to 2 hours
• +34% Cluster Utilization & -45% Pipeline Latency
• 10M+ Daily Records with 99.9% Reliability

Code Repository Technical Paper (PDF)

PySpark Large-Scale Diff

Cross-engine reconciliation engine supporting 5 dialects using SQLGlot AST pushdown and PySpark distributed joins.

• 50M+ Rows Reconciled in Under 90 Seconds
• 100% Precision & Recall across 12 fault types
• -85% Reduction in Manual Reconciliation Overhead

Code Repository Technical Paper (PDF)

Autonomous Text-to-SQL agent with Schema-Guided Constrained Decoding and sandboxed progressive self-correction.

• 84.2% Execution Accuracy on CareLake-QA
• 0% Hallucinations via AST Grammar Masking
• 68% Prompt Token Reduction via Steiner Trees

Code Repository Technical Paper (PDF)

Deterministic-first cascade pipeline converting unstructured EHR notes into FHIR R4 and OMOP CDM resources.

• 91.4% Entity Extraction F1-Score
• -73% Clinician Review via Platt Calibration
• 500K+ Pages Monthly at $0.0034 / Page

Code Repository Technical Paper (PDF)

Docker Stack Host

Composable Docker Compose service profiles (MinIO, Spark 3.5, Airflow 2.10, OTel LGTM, Postgres, SQL Server, Oracle Free).

• Synthea-compatible longitudinal EHR generation
• Distributed OTel Collector & Grafana dashboards
• Local-first execution on developer workstations

Code Repository

End-to-End Healthcare Data Platform Projects