Senior Data Engineer Backend; Vancouver

Key Responsibilities

Databricks Platform Expertise

• Develop, manage, and optimize data pipelines on the Databricks platform.
• Debug and troubleshoot Spark applications to ensure reliability and performance.
• Implement best practices for Spark compute and optimize workloads.
• Python Development
  :
  Write clean, efficient, and reusable Python code using object-oriented programming principles.
• Design and build APIs to support data integration and application needs.
• Develop scripts and tools to automate data processing and workflows.

Mongo

DB Management

• Integrate, query, and manage data within Mongo
  
  DB.
• Ensure efficient storage and retrieval processes tailored to application requirements.
• Optimize Mongo
  
  DB performance for large-scale data handling.

Collaboration and Problem Solving

• Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
• Proactively identify and address technical challenges related to data processing and system design.

• Proven experience working with Databricks and Spark compute.
• Proficient in Python, including object-oriented programming and API development.
• Familiarity with No
  
  SQL (Mongo
  
  DB preferred), including querying, data modeling, and optimization.
• Strong problem-solving skills and ability to debug and optimize data processing tasks.
• Experience with large-scale data processing and distributed systems.

• Strong understanding of data architecture, ETL processes, and data warehousing concepts.
• Knowledge of other big data technologies like Delta Lake, Hadoop, or Kafka.
• Experience with cloud platforms (e.g., AWS, Azure, or GCP).
• Familiarity with CI/CD pipelines and version control systems like Git.

This is what you can expect of your teammates at Flagler:

• Persistence + ownership of outcomes: We wear many hats and aren’t afraid to run through walls to solve hard problems.
• Personal + professional growth: We push ourselves to learn new things and embrace challenges, even if it means that we sometimes fail.
• Don’t take things personally
  :
  We value and react quickly to constructive feedback.
• Speed is our ally
  :
  In the fast-paced world of startups, we understand the value of moving swiftly. We thrive on the adrenaline of working rapidly.
• Be Right
  :
  We are highly detailed oriented and try to be right, a lot.

Note:

This description retains the original job information and context while adhering to formatting guidelines.