Senior Machine Learning/AI Operations Engineer

As a Senior Machine Learning/AI Operations Engineer, you will develop and operate the infrastructure, tooling, and workflows that enable effective use of AI and Machine Learning at CMT. Your responsibilities will span operationalizing AI developer tools and model services, automating model workflows, and improving reliability, observability, and cost efficiency. You will partner with Engineering, Machine Learning, Compliance, Security Engineering, and SRE to translate their requirements into operational capabilities and controls. You will apply reproducible workflows, evaluation, controlled deployment, monitoring, and incident response to existing developer harnesses and coding tools, large language model (LLM) applications and agents, and machine learning training and inference. Current priorities emphasize AI usage across engineering teams, with shared operational ownership of model development and release infrastructure. The role requires hands-on expertise in scalable AI/ML systems, independent technical judgment, and the ability to turn operational needs into reusable capabilities. CMT is looking for a collaborative, customer-committed, and creative Senior Machine Learning/AI Operations Engineer who wants to join us in making roads safer by making drivers better! Responsibilities: - Develop and optimize scalable AI/ML infrastructure, ensuring a balance between cost-efficiency and speed, with the goal of empowering Engineering and Machine Learning teams - Operationalize AI developer harnesses, coding tools, and shared model services by tuning model configurations, integrating tools into engineering workflows, evaluating their effectiveness, and implementing usage controls agreed with Compliance and Security Engineering - Optimize model access and inference through routing, rate limits, retries, timeouts, and fallbacks. Monitor token usage, latency, efficiency, throughput, and costs. Attribute usage and costs by team and workload, and implement budgets, quotas, and capacity controls - Establish monitoring infrastructure for evaluating the performance of AI applications and machine learning models at scale. Trace model calls, retrieval, tool execution, and training workflows; monitor data quality, model drift, and output quality; and implement safeguards for sensitive data and tool access - Operationalize evaluation workflows for models, prompts, and AI tool configurations. Work with Engineering and Machine Learning teams to track quality metrics and detect regressions, and with Compliance, Security Engineering, and SRE to incorporate policy, security, and reliability requirements into evaluation workflows and deployment decisions - Develop and manage reproducible agent, training, and evaluation workflows, including permissions, execution limits, failure recovery, and traceability of inputs and outputs - Build robust infrastructure to expedite the delivery of new models and AI capabilities, enhancing the efficiency of experimentation, validation, and deployment. Automate versioning, experiment tracking, model registries, release pipelines, retraining, and rollback - Manage, configure, deploy, and troubleshoot AI/ML workloads on AWS and Kubernetes/EKS. Automate infrastructure with Terraform, Helm, and Argo CD, and optimize CPU/GPU capacity and autoscaling for training and inference - Partner with SRE, Engineering, and Machine Learning teams to define operational metrics and service-level objectives, investigate production issues, and conduct root-cause analysis for model services, AI tools, and training pipelines. Coordinate with Security Engineering and Compliance when incidents involve security or policy concerns, and implement improvements that prevent recurring failures - Design and document systems, including writing and reviewing code, to automate operational problems within the team’s domain. Evaluate emerging AI/ML tools and operational practices against workload requirements, reliability, security, and cost, and mentor teammates - Architect software solutions integrating developer tools, backend, and research domains, facilitating effective data collection and transmission to support AI applications and ML models. Build and optimize ingestion, retrieval, feature engineering, model training, and validation pipelines using Databricks, Spark, Ray, and AWS services Qualifications: - Bachelor’s degree or equivalent practical experience and/or certification in a related field - 4+ years of relevant working experience, with experience working as an AI / ML Operations Engineer a plus - Demonstrated experience independently designing, delivering, and operating production AI/ML infrastructure, with ownership of technical decisions, deployment, reliability, and operational improvements. - Hands-on experience integrating and operating LLM services or AI tools, including model APIs, provider quotas, context and token limits, and failure handling - Experience with model training, validation, deployment, and monitoring, including reproducibility, experiment tracking, model versioning, and evaluation-driven releases. - Experience with big data infrastructure and software, such as Databricks, SageMaker, Spark, and S3 - Strong Python programming skills and experience with SQL and shell scripting for model integrations, data processing, and operational automation. Familiarity with libraries such as Pandas, NumPy, and scikit-learn. - Hands-on experience with containerization and Kubernetes, including management, configuration, deployment, and troubleshooting - Ability to design and document systems, including writing and reviewing code, to automate operational problems across AI/ML workflows. - Good understanding of AWS services, including model services such as Amazon Bedrock or SageMaker, and experience with infrastructure as code and CI/CD using tools such as Terraform, Helm, or Argo CD. - Demonstrated experience with production monitoring, and troubleshooting model, data, and infrastructure issues - Ability to navigate code, retrieve data from data sources, and implement bug fixes; actively seeks feedback and resolves discrepancies with teammates - Ability to collaborate across Engineering, Machine Learning, Compliance, Security Engineering, and SRE teams, translate their requirements into AI/ML operational practices and controls, evaluate technical tradeoffs, drive implementation through production rollout, and mentor engineers Nice to Haves: - Experience operating AI developer harnesses, coding tools, or agent workflows, including tool integrations, permissions, and usage management - Experience with AI evaluation and tracing tools such as Langfuse or LangSmith, or ML lifecycle tools such as MLflow or Kubeflow - Experience using EC2, Lambda, DynamoDB, RDS/PostgreSQL, NGINX, S3, and Kinesis in production environments - Experience building scalable data processing systems for production environments, including retrieval, embedding, or feature pipelines - Experience with deep learning frameworks such as TensorFlow, Keras, or PyTorch, including visualization and performance analysis - Experience with GPU provisioning, model serving, or training and inference optimization - AWS certification and/or Kubernetes certification Compensation and Benefits: - Fair and competitive salary based on skills and experience including Provident Fund (PF), Corporate NPS, and a meal card program - Equity may be awarded in the form of Restricted Stock Units (RSUs) - Medical Benefits (Health insurance, Personal accident insurance, Group term life insurance), gratuity, parental leave, sick/casual leave, leave encashments, and public holidays - Employees are eligible for flexible allowances which includes Leave Travel Assistance, Telephone/Mobile Expenses, Professional development expenses, Meals Coupon, Vehicle Reimbursement - Flexible scheduling and hybrid work arrangements, as appropriate based on role and responsibilities Additional Perks: - Work on a mission with real impact: crashes prevented, injuries avoided, lives protected around the world - Join an industry leader — 65+ million drivers protected, powering 140+ programs across 25 countries - Recognized innovator in mobility AI, earning top honors including the TIME Industry Leader in AI, a Gold Edison Award, and the Artificial Intelligence Excellence Award for AI for Social Good. CMT is also Great Place to Work Certified - Be part of the team inventing the future of mobility and road safety - Move fast, own outcomes, do work that matters - High ownership, small teams, and direct access to leadership — no layers between your work and its impact - Summer Fridays provide team members with half days to recharge - Join one of our employee resource groups: Black, AAPI, LGBTQIA+, Women, Book Club, and Health & Wellness - Comprehensive wellness, education, and employee assistance programs Commitment to Diversity and Inclusion: At CMT, we believe the best ideas come from a mix of backgrounds and perspectives. We are an equal-opportunity employer committed to creating a workplace and culture where everyone feels valued, respected, and empowered to bring their unique talents and perspectives. Diversity is essential to our success, and we actively seek candidates from all backgrounds to join our growing team. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status or disability state. CMT is headquartered in Cambridge, MA. To learn more, visit www.cmtelematics.com and follow us on Instagram @cmt.ai About Cambridge Mobile Telematics: Cambridge Mobile Telematics (CMT) is the world’s largest telematics and AI company for safer mobility. Its mission is to make the world’s roads and drivers safer. The company’s AI-driven platform, DriveWell Fusion®, proactively identifies and reduces driving risk, leading to fewer crashes and injuries. To date, CMT’s technology has helped prevent over 126,000 crashes worldwide. CMT enables partners to measure risk, detect crashes, provide life-saving assistance, and streamline claims. Headquartered in Cambridge, MA, CMT operates globally with offices in Budapest, Hungary; Chennai, India; Seattle, Washington; Tokyo, Japan; and Zagreb, Croatia. Learn more at www.cmt.ai.