Senior Data Engineer l AWS Glue & Kafka l

Senior Data Engineer – AWS Glue & Kafka

Title: Senior Data Engineer – AWS Glue & Kafka at VE3

Location: City Of London, England, United Kingdom

We’re seeking a hands-on Senior Data Engineer with about 10 years of experience to deliver production data pipelines on AWS. You’ll design and build streaming (Kafka) and batch pipelines using Glue/EMR (PySpark), implement data contracts and quality gates, and set up CI/CD and observability. You’ve shipped real systems, coached teams, and document as you go.

• Architect and deliver lake/lakehouse data flows on AWS (S3 + Glue + Glue ETL/EMR).
• Build Kafka consumers/producers, manage schema evolution, resilience, and DLQs.
• Implement PySpark transformations, CDC merges, partitioning and optimization.
• Add quality/observability (tests, monitoring, alerting, lineage basics).
• Harden security (IAM least privilege, KMS, private networking).
• Create runbooks, diagrams, and handover materials.

• Deep AWS (Glue, RDS, S3, EMR, IAM/KMS, Cloud Watch).
• Strong Kafka (MSK/Confluent, schema registry, consumer group tuning).
• Python/PySpark in production with tests and CI/CD.
• Data modeling (bronze/silver/gold, CDC, SCD2) and data contracts.
• IaC (Terraform/CDK) and cost/performance tuning experience.
• Clear communication and stakeholder engagement.

• Work on cutting-edge technologies and impactful projects.
• Opportunities for career growth and development.
• Collaborative and inclusive work environment.
• Competitive salary and benefits package.

Executive

Full‑time

Information Technology

IT Services and IT Consulting

Referrals increase your chances of interviewing at VE3 by 2x.

Apply BELOW