Platform Engineer

Your Impact

The Staff Software Engineer (Platform) leads large-scale projects and is a key contributor to their group's success. They are involved in long-term strategic planning, mentor senior engineers, and are influential in technical decision-making across multiple teams or a department. They focus on implementation and system design, as well as maintaining and evolving DISCO’s distributed data platform.

• Technical Leadership and Mentorship:
  Provides technical leadership in project teams, guiding engineers, and contributing to technical decision-making. Sets technical direction for large projects or teams, driving innovation, and solving organization-wide problems.
• Performance Optimization:
  Focuses on designing systems for high performance, scalability, and reliability.
• Ownership:
  Proactively identifies areas of products and systems that currently have problems or will develop problems as well as makes plans to address those problems.
• Stakeholder Engagement:
  Works with stakeholders, including product managers, designers, developers, and executives, to align architectural designs with business needs and customer expectations.
• Software Development:
  Designs, develops, and maintains sophisticated, scalable, high-performance data software solutions, ensuring they are efficient, scalable, and meet business requirements with architectural excellence for our big data platform at petabyte scale. Responsible for the boundary identification, system design and grooming for one or more sub-systems.
• Code Quality and Optimization:
  Upholds and enhances code quality through rigorous review processes, optimization techniques, and adherence to best practices.
• Technical Problem Solving:
  Addresses intricate technical issues through innovative problem-solving strategies.
• Collaboration and Communication:
  Works effectively within the engineering team and cross-functional teams, collaborating with other engineers, product managers, and stakeholders, contributing to project planning and execution.
• Continuous Learning and Adaptation:
  Stays updated with the latest technology trends and advancements, continually enhancing skills and knowledge, bringing innovative ideas to the team.
• Technical Documentation:
  Creates and maintains comprehensive technical documentation, ensuring clarity and accessibility of information.
• Data Platform Development:
  Builds and maintains scalable and robust data platforms to support analytics, machine learning, and business intelligence.
• Data Integration and ETL:
  Develops and optimizes ETL (Extract, Transform, Load) processes and data integration pipelines. Ensures efficient data flow across various sources and systems.
• Advanced File Processing:
  Develops extensible file processing pipelines that offer flexible, self-service artifact generation. Researches and evaluates file processing software based on fidelity, reliability, and performance criteria. Models file processing outcomes within an observability framework to provide insights for engineering teams and business leaders. Develops engineering systems that facilitate rapid and automatic evaluation of file processors.
• Data Management:
  Crafts and orchestrates scaleout data pipelines that optimize serverless and containerized compute to balance cost, latency, and duration for big-data solutions. Contributes to algorithmically intensive data engines operating on streaming, large, or multi-tenant datasets. Troubleshoots systems, identify root causes, and rectify bugs.

• 10+ years of relevant experience in backend engineering, with a substantial focus on big data platforms.
• 7+ years of professional experience with each of the following:
  Service Oriented Architecture, Docker-based services, micro-services, API and Data Model Design or Implementation.
• Experience with Large-Scale Systems:
  Involvement in developing, monitoring and maintaining large-scale, high-availability software systems.
• Experience designing and developing from a Security Perspective.
• Demonstrated experience in leading or playing key roles in significant software projects, showing an ability to work effectively both independently and as part of a team.
• Demonstrated…