More jobs:
Data Engineer – Databricks + DevOps
Job in
Santa Clara, Santa Clara County, California, 95053, USA
Listed on 2026-03-01
Listing for:
novasoft
Full Time
position Listed on 2026-03-01
Job specializations:
-
IT/Tech
Data Engineer, Cloud Computing
Job Description & How to Apply Below
Location: Santa Clara, CA
Experience:8+ Years
Employment Type:Full-time / Contract
Nova Soft is seeking a highly experienced
Databricks Data Engineer with strong Dev Ops expertise
to design, implement, and optimize large-scale Lakehouse architectures on AWS.
This role requires deep architectural understanding of compute vs. serving layer separation, low-latency data/API access strategies, and multi-terabyte data processing. The ideal candidate combines hands-on engineering excellence with technical leadership — a true player-coach mindset.
You will work closely with cross-functional teams to build scalable, secure, automated, and high-performance data platforms using modern Dev Ops practices.
Key Responsibilities:- Design and implement scalable
Databricks Lakehouse architectures on AWS - Build and optimize
ETL/ELT pipelines
using PySpark, Spark, and SQL - Implement
Delta Lake best practices
(partitioning, optimization, schema evolution) - Develop and manage
CI/CD pipelines and automated deployments
using Dev Ops tools - Optimize Spark workloads for
performance, cost efficiency, and low-latency access - Implement
data governance and security
using Unity Catalog - Collaborate with cross-functional teams and provide technical leadership
- Strong hands-on experience with:
- Databricks (Delta Lake, Unity Catalog, Delta Live Pipelines, Workflows, Runtime)
- PySpark, Spark, Advanced SQL
- Lakehouse & Medallion Architecture
- AWS expertise including:
- S3, IAM, Glue / Glue Catalog
- Lambda
- Secrets Manager
- (Kinesis is a plus)
- Dev Ops expertise:
- Git-based workflows
- CI/CD pipelines
- Databricks Asset Bundles
- Terraform (preferred)
- Experience handling multi-terabyte workloads
- Strong understanding of performance tuning, partitioning, and storage optimization
:
- Structured Streaming / real-time data pipelines
- Advanced Databricks runtime configuration
- Real-time or near real-time data solutions
- Exposure to Git Lab CI/CD pipelines
- Databricks Certified Data Engineer (Associate / Professional)
- AWS Certified Data Engineer or Solutions Architect
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×