Data Engineer - Databricks

Data, AI, & Software Engineering Specialist

HIRING

For this specific position, the client is only considering candidates who are based in the UAE.

Job Title – Data Engineer

Salary – MAX AED 30,000 Per Month

Contract Length – 1 Year Contract with the view to extend

Start Date – ASAP

My customer is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services banking, and government.

The Data Engineer is responsible for designing, implementing, and optimising data pipelines and infrastructure to support the cutting‑edge AI systems. The Data Engineer collaborates closely with a multidisciplinary team to ensure the efficient collection, storage, processing, and analysis of large‑scale data, enabling them to unlock valuable insights and drive innovation across various domains.

• Design, develop, and maintain scalable and reliable data pipelines to ingest, transform, and load diverse datasets from various sources, including structured and unstructured data, streaming data, and real‑time feeds
• Implement robust data integration processes to seamlessly integrate data from different sources, ensuring consistency, reliability, and data quality.
• Design and optimise data storage solutions, including relational databases, No
  
  SQL databases, data lakes, and cloud storage services, to efficiently store and manage large volumes of data.
• Optimise data processing and query performance to enhance system scalability, reliability, and efficiency, leveraging techniques such as indexing, partitioning, caching, and parallel processing.
• Implement data governance frameworks to ensure data security, privacy, integrity, and compliance with regulatory requirements, including data encryption, access controls, and auditing.
• Monitor data pipelines and infrastructure components, proactively identify and address issues, and perform routine maintenance tasks to ensure system stability and reliability.
• Collaborate closely with cross‑functional teams, including data scientists, architects, and domain experts, to understand requirements, gather insights, and deliver integrated solutions.
• Create comprehensive documentation, including technical specifications, data flow diagrams, and operational procedures, to facilitate understanding, collaboration, and knowledge sharing.

• Proven demonstrable experience with Databricks
• Proven experience as a Data Engineer, with a track record of designing and implementing complex data pipelines and infrastructure solutions.
• Expertise in data modelling, ETL processes, and data warehousing concepts, with proficiency in SQL and scripting languages (e.g., Python).
• Strong hands‑on experience with big data technologies and frameworks, such as PySpark, Spark, Kafka, as well as cloud platforms (e.g., AWS, Azure).
• Familiarity with containerization and orchestration technologies, such as Docker and Kubernetes, and Dev Ops practices for CI/CD (Continuous Integration/Continuous Deployment).
• Excellent analytical, problem‑solving, and communication skills, with the ability to translate complex technical concepts into clear and actionable insights.
• Proven ability to work effectively in a fast‑paced, collaborative environment, with a passion for innovation and continuous learning.

Mid‑Senior level

Contract

Consulting

Professional Services

Thanks,

Ollie