Databricks SME​/Lead

1 day ago Be among the first 25 applicants

This range is provided by OP. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

$60.00/hr - $70.00/hr

We are looking for an experienced Databricks SME/Lead to guide architecture, design, and implementation across our cloud data engineering initiatives. This senior role requires deep technical expertise, exceptional attention to detail, and the ability to advise engineering teams while also coordinating with senior leadership on high‑impact architectural decisions.

• Serve as the Databricks platform SME and trusted technical advisor for engineering teams.
• Create architecture patterns to leverage Databricks features for batch, real‑time, streaming, and operational analytics.
• Drive solutions with Databricks to accelerate data engineering productivity, ensure data protection & security, and improve performance of jobs & queries.
• Review architecture diagrams, data workflows, pipelines, notebooks, and solution designs for accuracy and best practices.
• Provide corrective guidance, optimization suggestions, and technical leadership on Databricks implementations.
• Function as the bridge between engineering teams and leadership, escalating high‑impact decisions when necessary.
• Enforce enterprise standards for architecture, governance, security, and performance across Databricks solutions.
• Mentor teams on Databricks, Spark, Delta Lake, and Lakehouse architectural patterns.
• Collaborate with architects, data engineers, and platform teams on pipeline design, orchestration, cluster strategy, and data optimization.
• Contribute to technical roadmaps, reusable patterns, and long‑term platform strategy.

• 8+ years in data engineering or architecture roles, with deep Databricks platform expertise.
• Strong proficiency in PySpark, Spark SQL, Delta Lake, Lakehouse architecture, and Databricks Jobs/Workflows.
• Proven ability to review and validate architectural diagrams, solution designs, and platform components.
• Hands‑on experience designing, optimizing, and troubleshooting Databricks pipelines and production workloads.
• Strong understanding of data governance, Unity Catalog, cluster management, CI/CD for Databricks, and performance tuning.
• Excellent communication and documentation skills, with the ability to mentor teams and act as a senior escalation point.
• Highly detail‑oriented with strong analytical and technical evaluation skills.

• Experience in hybrid roles combining technical execution with architectural governance.
• Cloud expertise (Azure, AWS, or GCP) in relation to Databricks deployments.
• Relevant certifications (Databricks, Spark, AWS/Azure/GCP).

• 401(k).
• Dental insurance.
• Health insurance.
• Vision insurance.
• We are an equal‑opportunity employer and value diversity, equality, inclusion, and respect for people.
• The salary will be determined based on several factors, including, but not limited to, location, relevant education, qualifications, experience, technical skills, and business needs.

• Participate in OP monthly team meetings and participate in team‑building efforts.
• Contribute to OP technical discussions, peer reviews, etc.
• Contribute content and collaborate via the OP‑Wiki/Knowledge Base.
• Provide status reports to OP Account Management as requested.

At OP, we help you harness the power of technology for maximum impact. A technology consulting and solutions company, we offer advisory and managed services, innovative platforms, and staffing solutions across a wide range of fields, including AI, cybersecurity, enterprise architecture, and beyond. For nearly two decades, we’ve been challenging the status quo of the consulting industry, serving up fresh, ingenious thinking through a radically lean structure.

Together, this strategy delivers unprecedented performance at an unparalleled pace for faster results that propel your business forward.

Mid‑Senior level

Contract

Information Technology

Motor Vehicle Manufacturing