Software Engineer, Stream Processing Platform

Overview

Employer Industry: Technology and Social Media

Why consider this job opportunity:

• Salary up to $350,763 USD
• Eligible for equity in addition to base salary
• Opportunity for career advancement and growth within the organization
• Flexible work model with in-office collaboration required only 1-2 times per quarter
• Work on cutting-edge big data technologies at scale
• Contribute to the open-source community and influence the technical direction of the company

• Lead the strategy and technical direction of the employer’s Spark platform
• Improve and customize the internals of open-source Spark for scalability
• Build and scale batch processing frameworks to handle petabyte-scale datasets
• Collaborate with internal teams on critical business use cases relying on batch processing
• Provide thought leadership on data processing and storage efficiency at scale

• 8+ years of industry experience with a proven track record of technical excellence
• 5+ years of experience building and supporting large scalable big data infrastructure/platform
• 2+ years of experience contributing to open-source Spark
• Deep knowledge of big data technologies (e.g., Spark, Hadoop, Parquet/ORC, Flink)
• Proficiency in one or more programming languages (Java, Scala, Python) and experience with Kubernetes and AWS

• Experience leading cross-team engineering efforts
• BS+ degree in a relevant field such as Computer Science or equivalent experience

#Big Data #Software Engineering #Career Opportunity #Remote Work #Inclusive Workplace