More jobs:
Job Description & How to Apply Below
Job DescriptionData Pipeline Architecture and Development:
- Architect and optimize scalable data storage solutions, including data lakes, warehouses, and No
SQL databases, supporting large-scale analytics. - Design and maintain efficient data pipelines using technologies such as Apache Spark, Kafka, Fabric Data Factory, and Airflow, based on cross-functional team requirements.
- Develop robust ETL processes for reliable data ingestion, utilizing tools like SSIS, ADF, and custom Python scripts to ensure data quality and streamline workflows.
- Optimize ETL performance through techniques like partitioning and parallel processing.
- Define and implement data models and schemas for structured and semi-structured sources, ensuring consistency and efficiency while collaborating with data teams to optimize performance.
- Establish and enforce data governance policies, ensuring data quality, security, and compliance with regulations, using tools like Microsoft SQL Server.
- Implement access controls, encryption, and auditing to protect sensitive data and collaborate with IT to address vulnerabilities.
- Manage and optimize cloud and on-prem infrastructure for data processing, monitor system performance, and implement disaster recovery enhancements.
- Leverage automation for provisioning, configuration, and deployment to improve operational efficiency.
- Provide technical leadership, mentoring team members in best practices and cloud technologies, while aligning data engineering initiatives with strategic goals.
- Bachelor's degree or higher in Software Engineering, Computer Science, Engineering, or a related field.
- 3-5 years of experience in data engineering, with a proven history of designing and implementing complex data infrastructure.
- Proficient in Python, Scala, or Java, with experience in scalable, distributed systems.
- Strong knowledge of cloud computing platforms and related services like AWS Glue, Azure Data Factory, or Google Dataflow.
- Expertise in data modeling, schema design, and SQL query optimization for both relational and No
SQL databases. - Excellent communication and leadership skills, with the ability to collaborate effectively with cross-functional teams and stakeholders.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×