Data Engineer; AI

About Data Society Group

Role Overview

Note:

Due to the confidential nature of our federal government Clients, this role requires the ability to pass a United States federal government Public Trust background check and is exclusively open to U.S. Citizens located within the United States.

• Design, build, and maintain scalable data pipelines for structured and unstructured data ingestion, transformation, and processing.
• Architect, build, and deploy LLM-based solutions on cloud platforms, including prompt pipelines, orchestration layers, embeddings, vector databases, and evaluation workflows.
• Design and implement RAG systems end-to-end including document ingestion, chunking/embedding, indexing, retrieval, grounding, model integration.
• Architect and enforce data models, governance, cataloging and schema design to support both analytics and AI workloads.
• Build and optimize cloud-native data architectures to support compute, storage, and orchestration for high-throughput, production-grade AI workloads.
• Implement reliable and efficient ETL patterns, leveraging best practices for data quality, lineage, versioning, and cataloging.
• Instrument observability and monitoring for data pipelines, including latency, error rates, and schema drift, with alerting and automated remediation where possible.
• Implement monitoring, observability, and performance optimization for data and AI systems.
• Operate effectively within Agile workflows, contribute to sprint planning, estimations, backlog refinement and continuous improvement.
• Work closely with clients to gather requirements, provide technical guidance and present solutions and implementation plans.
• Communicate complex technical information to both technical and non-technical stakeholders.
• Work cross-functionally with UX, engineering, and PM teams to deliver client-facing solutions.
• Translate complex technical needs into clear development requirements and implementation plans.
• Stay current with emerging technologies and recommend improvements to our engineering practices, architecture patterns, and cloud ecosystem.

• Hands-on experience deploying LLM-based applications, including RAG or similar retrieval systems.
• Proven experience deploying systems on AWS or Azure (AWS preferred).
• Strong understanding of embeddings, chunking strategies, retrieval optimization, and evaluation.
• 5+ years of data and analytics engineering in cloud environments.
• Expertise in SQL, Python, and schema design with experience in data cataloging and governance tools.
• Demonstrated experience building robust and maintainable data architectures, including real-time or streaming pipelines.
• Experience working in Agile / Scrum development processes.
• Excellent communication skills and ability to work cross-functionally with non-technical teams.

Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities, and activities may change at any time with or without notice.

This position will be remote in the US though based out of the Washington, DC area with travel to client sites in DC if needed.