Databricks Engineering Consultant

Position

Description:

CGI is the world's 5th largest independent IT and business consulting services firm, operating across 40 countries and delivering more than IP-based solutions to clients in Financial Services, Government, Healthcare, Manufacturing, Utilities, Oil & Gas, and more. We are recognized globally for our on-time, on-budget delivery and our role as a trusted advisor to leading organizations.
As organizations accelerate their data and analytics modernization, CGI is expanding its Data & AI consulting practice. We are seeking a Databricks Engineering Consultant with strong expertise in Databricks platform administration, Spark performance optimization, and secure, reliable operations.
Join our team and help operate and scale enterprise-grade analytics platforms that deliver reliable, high-performance data solutions.

Your future duties and responsibilities:

As a Databricks Engineering Consultant, you will be responsible for the configuration, management, and operational excellence of Databricks environments. This includes managing workspaces, clusters, compute pools, libraries, and security configurations, as well as optimizing Apache Spark performance to ensure efficient and cost-effective data processing.
You will work closely with Data Architects, Cloud Architects, and Security teams to ensure Databricks platforms are secure, performant, scalable, and compliant with enterprise operational standards.

Key Responsibilities
Databricks Platform Administration

. Configure and manage Databricks workspaces, including environments for development, testing, and production.
. Provision, configure, and optimize clusters and compute pools to balance performance, reliability, and cost.
. Manage Databricks runtime versions, cluster policies, and job configurations.
. Administer libraries and dependencies, ensuring consistent and reliable runtime environments.

Spark Performance & Optimization

. Monitor and tune Apache Spark workloads, including memory management, partitioning, caching, and execution plans.
. Identify and remediate performance bottlenecks in batch and streaming workloads.
. Collaborate with data engineering teams to optimize Spark code and pipeline execution.
. Implement best practices for cost optimization and resource utilization.

Security, Governance & Operational Controls

. Enforce workspace security policies, including access controls, permissions, and cluster policies.
. Support integration with enterprise identity providers and role-based access models.
. Implement operational standards for logging, monitoring, and alerting.
. Ensure Databricks environments comply with enterprise security, privacy, and operational governance requirements.

Cloud & Platform Integration

. Integrate Databricks with cloud-native services such as Azure Data Lake Storage, Azure Key Vault, and networking components.
. Support secure connectivity patterns and secret management.
. Collaborate with Cloud Architects on infrastructure design and optimization.

Delivery & Collaboration

. Partner with Data Architects and Data Engineers to support reliable deployment and execution of data pipelines.
. Participate in platform design discussions, providing operational and performance perspectives.
. Contribute to documentation, runbooks, and operational standards.
. Provide guidance and mentorship on Databricks operational best practices

Required qualifications to be successful in this role:

Must-Have Technical Experience

. 5–10+ years of experience in data engineering, platform engineering, or cloud data operations, including:
Databricks workspace and cluster administration Apache Spark performance tuning Compute pools, jobs, and scheduling Library and dependency management
. Strong experience with Cloud platforms (Azure preferred).
. Hands-on experience with monitoring, troubleshooting, and operational support of data platforms.
. Solid understanding of security, access controls, and operational governance.

Industry Experience

. Experience supporting data platforms in enterprise or regulated environments, such as Energy, Utilities, Financial Services, or Government, is an asset.

Consulting & Collaboration Skills

. Ability to work effectively…