AWS Databricks Architect; AWS, Databricks, Apache Spark Specialist

Role description

AWS Databricks Architect (AWS, Databricks, Apache Spark Specialist)
• Data Architect II

We are seeking a highly skilled and experienced Databricks Architect with deep AWS expertise to join our team. This role requires strong proficiency in Databricks, Apache Spark, and modern cloud-based data architectures, with a particular emphasis on building large-scale data engineering solutions on AWS.

• Engage with business and technology stakeholders to define project scope, goals, and architectural direction.
• Architect and implement end-to-end Databricks solutions on AWS, leveraging services such as S3, Glue, Lambda, Kinesis, EMR, Redshift, IAM, and Step Functions.
• Design and manage lakehouse architectures optimized for scalability, performance, governance, and cost-efficiency.
• Oversee the design and development of scalable ETL/ELT pipelines using Databricks and Apache Spark.
• Lead solution design sessions, conduct code reviews, and ensure adherence to data engineering best practices.
• Create and deliver technical presentations, architecture documents, and case studies for internal and external stakeholders.
• Stay updated on Databricks platform enhancements, AWS data services, and emerging trends in data engineering, AI, and ML.
• Provide technical leadership and mentorship to engineering teams across multiple projects and client engagements.

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 10+ years of experience in data engineering, with significant hands-on expertise in Databricks and Apache Spark.
• Strong end-to-end experience with the AWS data ecosystem, including S3, Glue, Kinesis, Lambda, Redshift, Dynamo
  
  DB, IAM, and Step Functions.
• Extensive experience building and managing ETL/ELT pipelines at scale.
• Proficiency in PySpark, SQL, and Python.
• Solid understanding of data warehousing, data lakes, and lakehouse architectures.
• Strong communication skills, with the ability to simplify and present complex technical topics.
• Proven ability to create detailed architecture documents and technical presentations.
• Willingness to travel based on project or client needs.
• Skills and Competencies
• Databricks certifications (Associate Data Engineer, Professional Data Engineer, etc.).
• AWS certifications are highly preferred (e.g., AWS Solutions Architect, AWS Data Analytics Specialty).
• Exposure to Azure and GCP cloud platforms is good to have.
• Experience with CI/CD (e.g., AWS Code Pipeline, Git Hub Actions), infrastructure-as-code (Terraform or Cloud Formation), and Dev Ops practices.
• Previous technical leadership or data architecture experience in consulting or multi-client environments.

Compensation can differ depending on factors including but not limited to the specific office location, role, skill set, education, and level of experience. UST provides a reasonable range of compensation for roles that may be hired in various U.S. markets as set forth below.

Role Location: Remote

Compensation Range: $161,000-$242,000

Full-time, regular employees accrue a minimum of 10 days of paid vacation per year, 6 days of paid sick leave per year (pro-rated for new hires), 10 paid holidays, and eligibility for paid bereavement leave and jury duty. Medical, dental, and vision insurance is provided, along with Company-paid life, accidental death and disability insurance, and short- and long-term disability benefits. Regular employees may participate in a 401(k) with employer matching;

HSA/FSAs are available where allowed. Benefits offerings vary by location.

Part-time and full-time temporary employees have varying sick leave and retirement plan details as described in the original job description.

We proudly embrace the values that have shaped UST since day one. We build a culture of Humility, Humanity, and Integrity to foster a people-first, human-centric environment that prioritizes sustainable solutions and keeps our people and clients at the forefront of all decisions.

UST is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other applicable characteristics protected by law. We will consider qualified applicants with arrest or conviction records in accordance with state and local laws and fair chance ordinances.

UST reserves the right to periodically redefine roles and responsibilities based on organizational requirements and performance.

#UST

#CB

#LI-MK1

#LI-Remote