Data Engineer - Azure, Fabric, ETL, SQL, PySpark

Get AI-powered advice on this job and more exclusive features.

This range is provided by Zeektek. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

$/yr - $/yr

Job Title:
Data Engineer

Role Overview

The Data Engineer plays a key role in designing, building, and maintaining data pipelines and infrastructure that support the organization’s analytics and reporting needs. This position focuses on converting raw data into actionable insights by ensuring data quality, consistency, security, and accessibility. The Data Engineer will collaborate closely with business data leads, BI engineers, and other stakeholders to deliver reliable, high-performing data solutions.

• Design and implement scalable, complex ETL (extract, transform, load) data pipelines from diverse sources such as databases, APIs, and files.
• Develop automated data ingestion, mock data generation, and validation processes.
• Optimize data pipelines for performance and efficiency.
• Maintain source control best practices (e.g., branch protection, Git-flow).
• Implement CI/CD pipelines, including build, test, and automated deployment (e.g., Jenkins, Travis CI, Azure Dev Ops).
• Demonstrate strong analytical and programming skills.

Database Design and Tuning

• Ensure referential integrity and query optimization across systems.
• Design and maintain scalable database solutions that support high-performance analytics.

Data Infrastructure Management

• Manage and maintain data warehouses, data lakes, and other storage solutions.
• Ensure data security, privacy, and compliance with applicable regulations.
• Monitor system performance and troubleshoot issues proactively.

Data Modeling

• Develop and maintain data models that accurately represent business processes and entities.
• Support data governance initiatives by implementing and maintaining policies and standards.

Data Quality Assurance

• Establish and implement data quality checks, validation rules, and error-handling procedures.
• Identify and resolve data inconsistencies or quality issues.

Tool Selection and Implementation

• Evaluate, recommend, and implement modern data engineering tools and technologies.
• Maintain and manage data engineering platforms to ensure optimal performance.

Collaboration and Agile Development

• Partner with business data leads, project managers, and stakeholders to define data requirements.
• Provide technical guidance and support for data-driven initiatives.
• Participate in agile development processes, including sprint planning, daily standups, and backlog grooming.
• Analyze business and technical requirements to develop documentation, designs, and test plans.
• Demonstrate flexibility by learning new technologies and contributing across the full stack when needed.

Qualifications

• Preferred:
  
  Experience with
  Azure Fabric and Azure Cloud environments.
• Strong experience with relational databases (e.g.,
  Oracle
  , SQL Server
  ) and No
  
  SQL databases.
• Advanced proficiency in SQL and experience with complex queries and optimization.
• Skilled in Python and Py Spark for data engineering and automation.
• Proven experience in performance tuning of data pipelines and database queries.
• Solid understanding of Dev Ops
  , microservices, and modern application design principles.
• Strong analytical, quantitative, and problem solving abilities.
• Excellent written and verbal communication skills; able to convey complex ideas through documentation and design diagrams.
• Education: High School Diploma or GED required;
  Bachelor’s degree preferred.
• Other Requirements: Valid driver’s license.

Mid-Senior level

Contract

Information Technology

Industries:
Staffing and Recruiting

Referrals increase your chances of interviewing at Zeektek by 2x

Medical insurance

Vision insurance

401(k)