Engineer, Data Engineer

Skill:
Data Engineer

• Experience with Snowflake and Snowpark.
• Data pipeline development using Python and SQL.
• 5 years of Data-related experience or equivalent combination of education and relevant experience.
• 5+ years’ experience in data engineering.

• Design, develop, and maintain end-to-end data pipelines for collecting, processing, and storing large volumes of data.
• Utilize Python and SQL for efficient and scalable data pipeline development.
• Implement best practices for data pipeline orchestration and scheduling.
• Manage and optimize Snowflake data warehouse, including schema design, performance tuning, and security configurations.
• Leverage Snowflake features such as virtual warehouses, clustering, and materialized views for optimal data storage and retrieval.
• Implement data quality checks and testing processes to ensure the accuracy and reliability of data pipelines.
• Collaborate with cross-functional teams to validate data transformations and ensure consistency in reporting.
• Create and maintain comprehensive documentation for data pipelines, data models, and system architecture.
• Provide documentation for data recipes, ensuring knowledge transfer and ease of maintenance.
• Identify and address performance bottlenecks in data pipelines and Snowflake queries.
• Work closely with data analysts and data scientists to optimize queries for efficient data retrieval.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions to meet business needs.
• Implement and adhere to data security and compliance standards.
• Work with the security team to ensure that data handling practices align with organizational policies.