Senior Web Scraping Engineer

At Web Automation, we help businesses unlock the power of public web data  products and custom scraping solutions empower organizations to access accurate, structured, and reliable datasets. We are an international, remote-first team passionate about cutting-edge web scraping technologies and large-scale data automation.

We are looking for a Senior Web Scraping Engineer with strong expertise in Python and Node.js to lead complex data extraction projects. You will design scalable scrapers, overcome advanced anti-bot systems, and build robust pipelines that power our products.

This role is ideal for someone who thrives on solving tough technical challenges, can effectively manage multiple projects in parallel, and quickly adapts to changing priorities in a fast-paced environment.

• Build and maintain advanced scrapers using Python, Node.js, Scrapy, Puppeteer, Playwright, Selenium
• Optimize pipelines for scalability, efficiency, and minimal resource usage
• Debug, maintain, and enhance existing scraping systems under changing site structures
• Handle multiple scraping projects simultaneously while meeting deadlines and quality standards
  
  Adapt quickly to daily priority changes and shifting project requirements
• Collaborate with product and data teams to ensure accuracy and reliability
• Mentor junior engineers and contribute to best practices in large-scale scraping
• Expertise in analyzing and extracting data from Graph
  
  QL endpoints
  , including introspection queries and schema discovery
• Experience with Web Socket reverse engineering for real-time data extraction
• Proficiency in analyzing API authentication flows (OAuth, JWT tokens, API key rotation, session management)
• Experience bypassing Cloud Flare, Perimeter
  
  X, Data Dome and other anti-bot solutions
• Skills in analyzing and replicating complex request signatures and encryption mechanisms

• 5+ years in web scraping, backend engineering, or data automation
• Expert in Python (asyncio, threading) and Node.js (async/await)
• Hands-on with Scrapy, Puppeteer, Playwright, Selenium
• Strong debugging and problem-solving skills for complex scrapers
• Strong multi-tasking skills with the ability to prioritize work in a dynamic environment
• Database experience (
  SQL/No
  
  SQL, Redis
  ) and caching systems

• Experience with Flask, Django, FastAPI
• Docker, Kubernetes, and CI/CD pipelines
• R&D or security research background
• Open-source contributions in scraping/data engineering

• Competitive salary based on expertise
• 100% remote & flexible work environment
• Opportunity to work on cutting-edge web & app scraping projects
• A collaborative international team that values innovation and autonomy
• Professional growth through challenging projects & mentorship

Apply now and join us at Web Automation to shape the future of large-scale web scraping.