Senior Data Engineer

Senior Data Engineer page is loaded## Senior Data Engineer locations:
Houstonposted on:
Posted Todayjob requisition :
JR102486
*
* Job Title:

** Senior Data Engineer
** Contract Type:
****** Time Type:
*****
* Job Description:

** Main Responsibilities
** Data Engineering Leadership
*** Contribute to the definition of the target data architecture (lake/lakehouse, streaming, event-driven) and technology choices for high-volume time-series, market, and transactional data.
* Build and operate end-to-end pipelines (ingest → quality → transform → serve) with strong SLAs/SLOs, lineage, and observability.
* Establish coding, testing, CI/CD, and infrastructure-as-code standards; drive adoption of data governance, cataloging, and access controls.
* Own capacity, performance, reliability, and cost efficiency of data platforms across environments (dev/test/prod).
* Ensure data quality, lineage, and observability across all layers of the data stack.
** Strategic Collaboration & Business Alignment
*** Partner with trading desks, quantitative teams, and risk functions to translate business needs into data solutions that enhance decision-making and operational efficiency.
* Act as a senior liaison between engineering and business stakeholders, ensuring alignment on data priorities and delivery timelines.
* Prioritize a value-based backlog (e.g., faster close/settlement, improved forecast accuracy, reduced balancing penalties) and measure business impact.
* Align data models and domain ownership with business processes (bids/offers, nominations, positions, exposures,outages).
* Coordinate with Cybersecurity, Compliance, and Legal on sector-specific controls (e.g., REMIT/NERC-CIP considerations, data retention, segregation).
** Innovation & Product Development
*** Incubate and industrialize data products: curated marts, feature stores, real-time decision APIs, and event streams for forecasting and optimization.
* Introduce modern patterns (CDC, schema evolution, Delta/Iceberg, stream–batch unification) to improve freshness and resilience.
* Evaluate and integrate external data (weather, fundamentals, congestion, capacity postings), internal and external vendor systems (ETRM) safely and at scale.
* Collaborate with quantitative analysts to product ionize ML pipelines (forecasting load/renewables, anomaly detection, etc.. ) with monitoring and rollback.
** Mentorship & Technical Oversight
*** Lead incident reviews and architectural forums; provide pragmatic guidance on trade-offs (latency vs. cost, simplicity vs. flexibility).
* Develop growth paths and learning plans focused on energy domain fluency and modern data engineering practices.
** Operational Excellence
*** Implement robust monitoring/alerting, runbooks, and on-call rotations; drive MTTR down and availability up for critical data services.
* Enforce data quality contracts (SLAs/SLOs), lineage, and reconciliation for market submissions, settlements, and reporting.
* Optimize cloud spend and storage/compute footprints; plan capacity for market events and seasonal peaks.
* Ensure security and compliance by design: least-privilege access, secrets management, encryption, auditability, and disaster recovery testing.

Profile
* Master’s or Bachelor’s degree in Computer Science, Data Engineering, Applied Mathematics, or a related technical field.
* 5+ years of experience in data engineering, with at least 3 years in a senior role.
* Proven experience in the energy trading sector, ideally with exposure to Natural Gas and Power markets, balancing mechanisms, and regulatory frameworks (e.g., REMIT, EMIR).
* Azure: ADLS Gen2, Event Hubs, Synapse Analytics, Azure Databricks (Spark), Azure Functions, Azure Data
* Factory/Databricks Workflows, Key Vault, Azure Monitoring/Log Analytics;
IaC with Terraform/Bicep; CI/CD with Azure Dev Ops or Git Hub Actions.
* Snowflake (on Azure or multi-cloud):
Warehousing design, Streams & Tasks, Snowpipe/Snowpipe Streaming, Time
* Travel & Fail-safe, RBAC & row/column security, external tables over ADLS, performance tuning & cost governance.
* Programming & Engineering Practices:
Strong OOP in Python and Java/Scala; SDLC leadership, Dev Ops mindset, TDD/BDD, code reviews,…