Medior​/Senior Data Engineer; Web Scrapping

About the Role

We are looking for a Medior/Senior Data Engineer to join our team in Bali. In this role, you will play a key part in building and scaling production-grade web scrapers and supporting the Lead Data Engineer in technical decisions, project planning, and team coordination. You will also mentor junior and medior engineers while ensuring best practices are followed.

• Lead the development and maintenance of scalable, production-grade web scrapers.
• Design, build, and maintain reusable scraper libraries and frameworks.
• Define coding standards, workflows, and processes with the Lead Engineer.
• Implement robust monitoring, testing, and alerting systems to ensure reliability.
• Manage and optimize cloud infrastructure (AWS/GCP/Azure) for large-scale scraping workloads.
• Optimize compute, storage, and proxy usage for performance and cost efficiency.
• Mentor and review code of junior and medior engineers.
• Collaborate with stakeholders to prioritize backlog items and integrate scraper outputs into pipelines.
• Act as second in command to the Lead Data Engineer for technical and project support.

• Strong proficiency in Python and SQL
  , with hands-on web scraping experience.
• Deep knowledge of frameworks like Scrapy, Playwright, Selenium
  , and anti-bot strategies.
• Experience designing reusable libraries, coding standards, and modular scraper frameworks.
• Proficiency with cloud infrastructure (AWS/GCP/Azure), including compute, storage, and security basics.
• Experience with Docker & Kubernetes for containerization and orchestration.
• Familiarity with CI/CD pipelines, Git workflows, and IaC tools (Terraform/Cloud Formation).
• Strong debugging, performance tuning, and large-scale scraper optimization skills.

• Workflow orchestration (Airflow, Prefect, Dagster).
• Observability/monitoring tools (Prometheus, Grafana, Cloud Watch, ELK).
• Cost optimization in cloud and proxy-heavy workloads.
• Knowledge of data governance, RBAC, and compliance.

Be part of a data-driven company that values curiosity, precision, and impact.

Collaborate with talented data engineers and analysts to solve real business challenges.

Based in beautiful Bali
, working directly on-site with a passionate and talented team.

If relocating from other parts of Indonesia, we offer one-time reimbursement for flights and relocation expenses
.