Azure Data Engineer

Overview

Azure Data Engineer working with Microsoft Fabric, Synapse/ADF, SQL, ETL, PySpark.

• Data Loading Architecture:
  Design data loading patterns and lakehouse architectures for orchestration processes on an enterprise‑scale level.
• Data Preparation and Enrichment:
  Prepare and enrich data for analysis, ensuring data quality and consistency checks.
• Semantic Modeling:
  Implement semantic models for BI and self‑service analytics.
• Security Compliance:
  Secure and manage analytics assets, enforce governance policies, and monitor compliance using Microsoft Purview and Managed Identity.
• Performance Optimization:
  Monitor and optimize Fabric pipelines and workloads for cost efficiency and high availability.
• Collaboration and Stakeholder Engagement:
  Work closely with architects, analysts, and business teams to translate requirements into scalable solutions.
• Power BI Integration:
  Enable Power BI for visualization and reporting.
• Data Integration Orchestration:
  Build and manage data pipelines using Azure Data Factory (ADF) and Synapse pipelines for batch and incremental ETL workflows, implementing parameterized pipelines for scalability.
• Data Transformation Modeling:
  Develop PySpark notebooks and SQL scripts to clean, transform, and convert raw files into optimized formats (Parquet/Delta). Create views, stored procedures, and dimensional models for reporting and analytics.
• Performance Monitoring:
  Monitor Dynamic Management Views (DMVs) in Synapse to analyze system health and reduce bottlenecks.
• Workload Management:
  Implement strategies for query performance and resource governance.
• Security Governance:
  Apply role‑based authentication, encryption/decryption logic, and compliance tasks to reduce manual intervention.
• Reliability Monitoring:
  Build failure monitoring systems integrated with Logic Apps for alerting and proactive issue resolution.

• Apache Hive
• S3
• Hadoop
• Redshift
• Spark
• AWS
• Apache Pig
• No
  
  SQL
• Big Data
• Data Warehouse
• Kafka
• Scala

Employment Type:

Full Time

Experience:

years
Vacancy: 1