Lead Data Engineer

Overview

Whitehall Resources currently require an experienced Lead Data Engineer to work with a key client.

Please note this role falls INSIDE IR35 and requires 3 days onsite weekly in Sheffield

• Lead data engineering initiatives and develop data engineering platforms, with hands-on involvement as needed.
• Extensive enterprise experience with Hadoop, Spark, and Splunk.
• Proficiency in object-oriented and functional scripting, particularly in Python.
• Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and No
  
  SQL).
• Experience integrating large, disparate datasets using modern tools and frameworks.
• Strong background in building and optimizing ETL/ELT data pipelines.
• Familiarity with source control and implementing CI/CD pipelines.
• Experience supporting and collaborating with BI and Analytics teams in fast-paced environments.
• Ability to pair program and work effectively with other engineers.
• Excellent analytical and problem-solving abilities.
• Knowledge of agile methodologies such as Scrum or Kanban is a plus.
• Comfortable representing the team in standups and problem-solving sessions.
• Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality.
• Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering.
• Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions.