Data Engineering Expert

Company: CGL
Department: IT

Employment Type:

Regular Full-Time
Work Model:
Hybrid
Language:
English is required, French is an asset.
Additional Information:
This/these role(s) is/are currently vacant

We are a leading Canadian financial services co-operative committed to being a catalyst for a sustainable and resilient society and our team is essential to deliver on this strategy. That’s why we prioritize our people, to ensure we provide a strong culture and development opportunities which enables our team to thrive and to live our purpose. The best part is that you will work with people that care passionately about you, our clients, and our communities.

Our Information Technology team aspires to be a leader in applying technology to power business strategies. We connect concepts with solutions to create value and efficiencies for our clients, employees, and communities. Our success is driven by our skilled and diverse team who are passionate about excellence, innovation, and agility.

The Data Engineering Expert is a critical role in the data organization, responsible for providing hands on Engineering Leadership by leading the design and build of data products that collect and store large datasets and leading the design and build of data platforms that underpin and accelerate the build of data products.

They ensure the data product is resilient and is readily accessible across various teams in a secure manner. They ensure that data products comply with the data access policies and rules that the governance team implements and leads the design and implementation of robust security measures to protect sensitive information. They are also responsible for upskilling staff members in data tools and technologies and lead the implementation of new tools.

They take end-to-end ownership of building, deploying and maintaining data products. They are a seasoned professional with deep understanding of Azure cloud infrastructure, automation principles and has a security-first mindset.

Data engineering expert will partner with the business, reporting specialists and with specialists from Advanced Analytics to drive the build of a product that satisfies the requirements of multiple stakeholders. They will also partner with networking, security, data governance, data engineering leaders and vendors like Microsoft and Databricks to drive the build of the data platform.

This role will empower Co-operators data community and the cloud community by contributing to the cloud catalog.

• Designing and implementing cloud architecture:
  Plan capacity, create infrastructure including subscriptions, tools and networking aspects such as private endpoints, subnets, (Network Security Groups) NSGs and create and maintain detailed architectural diagrams to support business requirements.
• Implement using infrastructure as a Code (IaaC) solutions (Terraform strongly preferred) to automate the provisioning and management of Azure resources such as subscriptions, resource groups, storage, networks and security.
• Continuous Integration/Continuous Deployment (CI/CD):
  Build and Maintain robust CI/CD pipelines that include code quality and security scanning, vulnerability testing to automatically deploy data engineering pipelines to various environments.
• Security and Compliance:
  Integrate security principles throughout the development and deployment life cycles, leveraging tools for vulnerability scanning, threat modeling, and compliance monitoring.
• Lead the build of Data Pipelines templates:
  Lead a team of engineers in building robust data pipeline templates to automate the future development of data pipelines according to medallion architecture including automated reconciliation, data quality, technical metadata capture and monitoring and altering mechanism.
• Performance Optimization:
  Define and implement automated process to generate alerts on failures or performance degradation of data products with a view of taking proactive steps in resolving issues before they become evident.
• Documentation and Maintenance:
  Define, lead and enforce a documentation system to capture the state of the existing implementation, and to ensure ongoing maintenance and updates.
• Process Improvement:
  Identify and implement engineering best practices and process improvements to enhance the efficiency and effectiveness of the data engineering team.
• Stakeholder Communication:
  Partner with business and technology stakeholders to identify and prioritize Customer centric data initiatives that address business challenges and support business value creation. Ensure that the data engineering team is aware of the priorities and communicate project goals, progress and outcomes.;
  Partner with security, governance and technology stakeholders to identify and implement changes to infrastructure platform code and ensure that data engineering teams are aware of the changes.
• Lead and coordinate security and platform updates to the underlying data infrastructure by…