Developer - Data Engineer

Job Summary
:
Duke Health is dedicated to compassionate care and innovative technology solutions in healthcare. They are seeking a Developer - Data Engineer to design, develop, and optimize databases and ETL pipelines, while collaborating with data scientists and analysts to implement effective data management solutions.

Responsibilities
:

• Individual contributor works under the direct supervision of level 3 or above and collaborates as a member of a team

• Basic understanding of local databases (Clarity; Caboodle) and underlying tables pertaining to role/domain

• Basic understanding of ETL technologies and scripting languages and terminology related to role

• Basic understanding of cloud data engineering technologies, including storage accounts, data pipelines, data lakes, structured vs. unstructured data, etc.

• Ability to design, construct and test data management systems under guidance of senior team members

• Assists in server administration tasks, ensuring system performance and availability

• Basic knowledge of SQL and relational databases

• Follows all policies and procedures set forth by DUHS/DHTS/ACE as pertains to role

• Responds to customer requests for assistance as assigned and prioritized by manager

• Participates in code review and learning best practices in data engineering

• Maintains ongoing communications (timely, clear, concise, and accurate) with manager keeping them apprised of request status

• Maintains appropriate project documentation as assigned such as GITLAB or data dictionary

• Meets all Competencies in Level 1 AND:

• Works with new hires/Level I team members to train on use of tools and technologies to accomplish job duties

• Demonstrates advanced knowledge of the assigned domain/role. Understands data, workflows, and tools related to role.

• Demonstrates advanced knowledge of, and experience with, cloud data engineering concepts, including (but not limited to) Microsoft Azure Lake houses and Delta Tables, Azure Data Factory, Azure Synapse, and medallion-style data architecture.

• Demonstrates ability to work with stakeholders to understand business needs and perform business analysis independently >75% of the time.

• Develop technical requirements from business requirements independently >75% of the time.

• Can write/modify accurate SQL queries independently > 90% of the time

• Ability to optimize SQL queries, stored procedures and scripts for performance reliability

• Can write complex SQL inclusion/exclusion criteria for cohort development

• Manages and optimizes server infrastructure for performance and reliability

• Troubleshoot and resolve complex data pipeline, server and API issues

• Ability to locate data elements independently > 75% of the time

• Knowledge/utilization of basic data design and infrastructure

• Participates in team code review

• Actively participates/contributes to database QA/submit requests for new database tables/attributes

• Actively seeks out participation in Special Projects and is able to effectively accomplish project assignments with minimal supervision

• Ability to manage multiple projects to completion in a timely manor

• Ability to develop and present topics as assigned/needed by customer and/or ACE team

• Active participation in Duke Analytic Community or other networking forums applicable to job functions

• Meets all Competencies in Level 2 AND:

• Demonstrates expertise in systems and software related to project area to accomplish job duties (e.g. SSIS, Azure Data Factory, etc..)

• Resolves escalated issues/incidents to ensure high availability and performance of ETL processes

• Architects and designs robust data solutions

• Expert knowledge and experience with Microsoft Azure Data Engineering tools, concepts, and industry best practices.

• Expert locating data elements as required. Assists team in locating data elements as needed.

• Works with team members to train on advanced skills in using tools and technologies to accomplish job duties

• Leads the development of complex data pipelines and API integration, ensuring scalability and efficiency

• Defines standards and best practices for data pipeline design, development and maintenance

• Demonstrates ability to…