Sr Lead Machine Learning Engineer at Upwork

This role is designed for a heavyweight Machine Learning Engineer or Applied Scientist who specializes in the "quality" side of AI. The ideal candidate has moved beyond simple model training and now focuses on building the complex evaluation frameworks required for AI agents and LLMs. You should have a strong background in statistical experimentation and the ability to build the data pipelines (Python/SQL) necessary to measure how humans and AI interact in real-world, open-ended scenarios. The compensation is highly competitive, topping out at over $300k, and the position is fully remote (within 21 supported U.S. states). As a "Senior Lead," you aren't just an individual contributor; you are a strategic influencer shaping the product roadmap by defining what "success" actually looks like for agentic systems. You’ll enjoy premium benefits like unlimited PTO, 401(k) matching, and an employee stock purchase plan. **You might be a good fit if you:** * Have built benchmarking pipelines specifically for LLMs or agentic systems. * Are obsessed with creating metrics for "fuzzy" outcomes like trust calibration and handoff quality. * Want to lead the technical standards for observability and experimentation at a major marketplace. * Prefer a remote-first culture that still offers the stability and scale of a Nasdaq-listed company.