Senior Data Engineer; Apache Flink

Senior Data Engineer (Apache Flink) at N Consulting Ltd

The Financial Times is one of the world’s leading business news organisations. We are hiring an experienced Apache Flink Engineer (contract) to build and help ope rationalise the stream-processing components of a near real‑time personalisation and recommender capability on AWS.

You will bring deep hands‑on expertise in production Flink on AWS, working closely with a senior FT engineer and AWS stakeholders. A key part of the role is knowledge transfer through pairing, reviews and clear documentation so the capability can be supported and extended by FT.

• Design and implement stateful stream‑processing jobs in Apache Flink (primarily Java), to process user interaction events and produce near real‑time outputs for downstream services.
• Integrate Flink with Kafka/AWS MSK, S3 and the chosen online store (Mongo
  
  DB/Mongo
  
  DB Atlas).
• Own streaming non‑functional requirements: correctness, fault tolerance, checkpointing/recovery, back pressure handling, and performance tuning.
• Define and implement production readiness: observability (metrics/logs/tracing), alerting, dashboards and runbooks.
• Support CI/CD and infrastructure‑as‑code practices to deploy and operate Flink workloads safely in AWS.
• Enable the team via knowledge transfer (pairing, design/code reviews, documentation and handover).

Essential

• Strong experience building and operating production Apache Flink applications (event‑time concepts, windows/watermarks, state, checkpointing and fault tolerance).
• Hands‑on AWS experience operating streaming/data workloads (IAM, networking/VPC basics, deployment, observability and incident troubleshooting).
• Experience integrating Flink with Kafka (including AWS MSK) and cloud services/storage (e.g., S3).
• Highly proficient in Java and Python (production experience required).
• Strong software engineering fundamentals: clean, maintainable code; pragmatic testing; effective code review.
• Comfort mentoring/pairing with engineers to up‑skill the team.

Desirable

• Experience with Amazon Managed Service for Apache Flink and/or running Flink on EKS/ECS.
• Experience with Mongo
  
  DB or Mongo
  
  DB Atlas for low‑latency, online‑serving use cases.
• Infrastructure‑as‑code (Cloud Formation) and modern observability tooling (Cloud Watch, Grafana, Open Telemetry, Splunk).