Data Engineer

Project description

We are seeking a skilled and hands‑on Data Engineer with proven experience in Databricks, Apache Spark, Python, and real‑time data streaming solutions.

• Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
• Develop, optimize, and maintain PySpark applications within the Databricks environment.
• Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
• Write and maintain unit tests to ensure high code quality and reliability.
• Perform data validation and quality checks to ensure compliance with defined business rules.
• Support and manage production deployments, ensuring system stability and performance.
• Troubleshoot and resolve issues across development, testing, and production environments.
• Collaborate with Product, QA, Dev Ops, and healthcare domain teams to ensure accurate implementation.
• Maintain technical documentation for code, workflows, and deployment processes.
• Pull from and push code to Git Hub repositories, following version control and branching best practices.

• 5+ years of experience in data engineering roles.
• Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
• Strong hands‑on experience with PySpark and/or Apache Spark.
• Experience developing in Databricks.
• Experience working with CI/CD pipelines and Dev Ops best practices.
• Proficiency with CI/CD, Git Hub (pull requests, branching, code reviews).
• Experience writing unit tests and performing data validation.
• Experience supporting production systems and deployments.
• Strong analytical and troubleshooting skills.

Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.

Experience with healthcare data integration projects.

Familiarity with REST APIs and FHIR server integrations.

Experience in Agile/Scrum development environments.