Data QA

Detailed Job Description

6️⃣ Data Quality Automation Analyst (2) (Databricks-Centric) Role Overview Design and automate data validation within Databricks pipelines to ensure accurate migration and ongoing data reliability. Key Responsibilities • Build automated data quality checks in PySpark • Develop reconciliation framework between: • Legacy SQL outputs • New Databricks outputs • Implement rule-based validation for healthcare datasets • Automate profiling and anomaly detection • Integrate quality checks into pipeline workflows • Build DQ scorecards and dashboards • Support cutover validation Required Skills • Strong SQL & PySpark • Experience with Databricks validation frameworks • Experience implementing automated DQ checks • Strong reconciliation expertise • Healthcare data validation experience preferred