Senior Data Engineering Specialist

At U.S. Bank, we’re on a journey to do our best. Helping the customers and businesses we serve to make better and smarter financial decisions and enabling the communities we support to grow and succeed. We believe it takes all of us to bring our shared ambition to life, and each person is unique in their potential. A career with U.S. Bank gives you a wide, ever-growing range of opportunities to discover what makes you thrive at every stage of your career.

Try new things, learn new skills and discover what you excel at—all from Day One.

• Guide the team to migrate from on‑prem Cloudera to an Azure cloud environment.
• Design and implement scalable data lake solutions using Snowflake and Databricks; develop and optimize data pipelines for ingestion, transformation, and storage.
• Manage data governance, quality, and security across cloud environments; implement performance tuning, automation, and CI/CD for data workflows.
• Collaborate with cross‑functional teams to support cloud migration activities.

• Install, configure, manage, and monitor Cloudera Hadoop clusters, ensuring high availability, performance, and security.
• Manage HDFS, YARN, and other ecosystem components.

• Tune Hadoop, Hive, and Spark jobs and configurations for optimal performance, efficiency, and resource utilization.
• Optimize queries, manage partitions, and leverage in‑memory capabilities.

• Diagnose and resolve issues related to Linux servers, networks, cluster health, job failures, and performance bottlenecks.
• Provide on‑call support and collaborate with other teams to ensure smooth operations.

• Implement and manage security measures within the Cloudera environment, including Kerberos, Apache Ranger, and Atlas, to ensure data governance and compliance.
• Setup and manage Hashi Corp Vault for secure key and secret management.
• Utilize Cyber Ark for privileged access management and secure administrative tasks on the cluster.

• Move Hadoop, Hive, and Spark data and applications to Azure cloud services such as Azure Synapse Analytics, Azure Databricks, or Snowflake.
• Ensure data integrity, performance tuning, and validation.
• Develop automation scripts (e.g., shell, Ansible, Python) for administrative tasks, deployments, and monitoring.
• Work with users to develop, debug, and optimize Hive, Spark, and Python programs that connect to the Cloudera environment.

• Create and maintain documentation for system configurations, operational procedures, and troubleshooting knowledge bases.

• Work closely with the Cloudera vendor to stay current with releases, perform upgrades, and address vulnerabilities.

• Advanced degree in Computer Science, Engineering, or a related field.
• Deep expertise in Data Engineering and Management technologies, synthetic data, automation, and advanced analytics.
• 10+ years of hands‑on experience in data engineering, cloud platform management, and performance optimization.
• Hands‑on experience with Hadoop, Hive, Spark, and migration of Big Data into Azure cloud services.
• Experience with Databricks and Snowflake for data integration and lake architectures.
• Working knowledge of Microsoft Azure cloud and big data migration to cloud platforms.
• Experience with Hashi Corp Vault and Cyber Ark for secrets and privileged access management.
• Proficiency in Linux, clustering, and distributed systems.
• Expertise in Hive, Spark, Hadoop ecosystem components (HDFS, YARN, Sqoop).
• Proficiency in shell, Ansible, C/C++, Java, Python, and PySpark for automating workflows, deployments, and monitoring.
• Expertise in Linux, network, Python scripting, DNS, Kerberos, LDAP/AD, MySQL, Postgre
  
  SQL, and Jupyter Hub.
• Experience in creating and maintaining documentation for system configurations, operational procedures, and troubleshooting knowledge bases.
• Strong problem‑solving skills and the ability to diagnose and resolve system failures and performance bottlenecks.
• Excellent communication and collaboration skills to work effectively with…