Data scientists, AI engineering, LLM engineer, Machine Learning Engineers

Overview

Role:
Data scientists, AI engineering, LLM engineer, Machine Learning Engineers

Location:

Charlotte, NC (Local candidate only)

Video Interview

• Implemented a chatbot internally with the bank, build interface and now users can interact.
• It's a RAG framework so instead of tuning it into the actual applications you can prompt it to give you prevectorized queries.
• Able to feed it documents even if they don't know what your team is or who you are. Should have 1000 users by the end of the year and another 2000 next year.

• LLMs & Inference:
  Experience with major LLMs, specifically Llama 3, Mistral, and possibly Quinn.
• Direct experience with VLLM (inference engine) is a strong match for handling batched requests.
• Experience with Nvidia Triton is a bonus and a key part of model serving infrastructure.
• Core Development:
  Python (mandatory);
  Python 3.12 in use, with 3.10+ acceptable.
• Web Frameworks:
  Flask or FastAPI (required for hosting the LLM via a Python endpoint).
• Java:
  Secondary preferred for creating REST services interacting with the front-end UI.
• Database & Data Management:
  Vector Databases (Redis and other vector stores) and SQL required.
• RAG
  
  Skills:
  
  Ability to interpret business-side parameters from the product team and push back if technically unfeasible; demonstrates critical thinking beyond coding.
• Infrastructure & Operations (MLOps)
• Containers & Orchestration:
  Knowledge of containers and Open Shift for CI/CD.
• CI/CD Tools:
  Experience with XLR and Datical for pipeline deployments.
• Hardware:
  Solid understanding of GPUs as a critical infrastructure component.
• Agile:
  Team uses Agile methodology.
• Scaling:
  Project growth tied to hardware availability; initial deployment capped at 1,000 users with scaling contingent on budget.

Nice to have, but not necessarily required:

• General awareness of Vector DB vs relational databases.
• Experience pushing code to controlled environments and production AI applications.
• Any model experience or quantitative modeling, or prior white papers (as observed at the client).

• Mid-Senior level

• Full-time

• Other

• IT Services and IT Consulting