Lead Data Engineer; Hybrid

Resp & Qualifications

PURPOSE: The Lead Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud or on‑premises infrastructure targeting big data and platform data management (Relational and No

SQL, distributed and converged) with emphasis on reliability, automation and performance. This role will focus on leading the development of solutions and helping transform the company’s platforms to deliver data‑driven, meaningful insights and value to the company.

• Lead the team to design, configure, implement, monitor, and manage all aspects of Data Integration Framework. Defines and develop the Data Integration best practices for the data management environment of optimal performance and reliability.
• Develops and maintains infrastructure systems (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data using Hadoop or equivalent Map Reduce platform.
• Provides detailed guidance and performs work related to Modeling Data Warehouse solutions in the cloud or on‑premise. Understands Dimensional Modeling, De‑normalized Data Structures, OLAP, and Data Warehousing concepts.
• Oversees the delivery of engineering data initiatives and projects. Supports long‑term data initiatives as well as ad‑hoc analysis and ELT/ETL activities. Creates data collection frameworks for structured and unstructured data. Applies data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources.
• Enforces the implementation of best practices for data auditing, scalability, reliability and application performance. Develop and apply data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources.
• Interprets data, analyzes results using statistical techniques, and provides ongoing reports. Executes quantitative analyses that translate data into actionable insights. Provides analytical and data‑driven decision‑making support for key projects. Designs, manages, and conducts quality control procedures for data sets using data from multiple systems.
• Improves data delivery engineering job knowledge by attending educational workshops; reviewing professional publications; establishing personal networks; benchmarking state‑of‑the‑art practices; participating in professional societies.

Position does not have direct reports but is expected to assist in guiding and mentoring less experienced staff. May lead a team of matrixed resources.

Bachelor’s Degree in Computer Science, Information Technology or Engineering or related field or, in lieu of a Bachelor’s degree, an additional 4 years of relevant work experience is required in addition to the required work experience.

8 years’ Experience in leading data engineering and cross‑functional team to implement scalable and fine‑tuned ETL/ELT solutions for optimal performance. Experience developing and updating ETL/ELT scripts. Hands‑on experience with application development, relational database layout, development, data modeling.

• Total of 12+ Years of IT Experience predominantly in Data Integration/Data Warehouse area
• Must have 7 years of ETL Design and Development experience using Ab Initio
• 1‑2 years of Data Integration project experience on Hadoop Platform, preferably Cloudera
• At least one project implementation of Ab Initio CDC (Change Data Capture) in a Data Integration/ETL project
• Working knowledge of HDFS, Hive, Impala and other related Hadoop technologies
• Sound understanding of SQL and ability to write well‑performing SQL queries
• Good knowledge of OLTP and OLAP data models and other data warehouse fundamentals
• Igor in high code quality, automated testing, and other engineering best practices, ability to write reusable code components
• Ability to unit test the code thoroughly and to troubleshoot issues in production environments
• Must have some working experience with Unix/Linux shell scripting
• Must be able to work independently and support other developers as needed
• Some Java development experience is nice to have
• Knowledge of…