Data Engineer

We have partnered with our client in their search for a Data Engineer in Plano, TX.

Design, build, and maintain scalable data pipelines to support real-time, batch, and streaming data processing using tools such as Python, Java, Spark, and Kafka.

Develop and optimize ETL/ELT workflows for ingesting, transforming, and integrating large datasets from multiple internal and external sources.

Create and maintain data models (logical and physical) to support analytics, reporting, and application development.

Implement robust data quality processes including validation rules, anomaly detection, monitoring, and automated data checks.

Ensure data compliance and governance by adhering to Company standards for security, data privacy, metadata management, data lineage, and audit requirements.

Collaborate with cross-functional teams including data scientists, analysts, software engineers, business stakeholders, and architects to deliver high-quality data solutions.

Optimize performance of data pipelines, distributed systems, and storage solutions to ensure reliability and scalability.

Contribute to modern data platform initiatives
, including cloud migration, big data architecture, and adoption of new data technologies.

Develop automation and CI/CD processes for deployments, testing, and monitoring of data engineering solutions using tools such as Jenkins, Git, and Terraform.

Troubleshoot and resolve pipeline issues
, ensuring stability, accuracy, and timely delivery of data across the organization.

• Experience on AWS technologies like S3, AWS Glue, EMR, and IAM roles/permissions
• Experience with one or more query languages (SQL, PL/SQL, DDL, Spark
  
  SQL, Scala)
• Experience working with relational databases such as Teradata and handling both structured and unstructured datasets
• Proficiency in SQL, Python
• Experience with No
  
  SQL and non-relational databases/data stores (object storage, document or key-value stores, graph databases, column-family databases)
• Experience with Snowflake and Databricks
• Proficiency in tools such as SQL, Python, Tableau or Power BI
• Understanding of financial industry metrics and business processes
• AWS Cloud Practitioner or Developer certifications

5+ years of experience with data modeling, warehousing, analysis & data profiling experience and ability to identify trends and anomalies in the data