Skip to content

Data Scientist

KaizoAmsterdamHybrid

First seen Jul 14 · open 130 days · posted onceNo pay listed

HirQ verdict

52/100 · Fair. Based on age, reposts, pay, the company's hiring and ghost signals.

Low ghost risk
True age
130 days
Reposts
Posted once

See the evidence behind this score.

Pro shows each part of the score, the reasons, the pay percentile and the listing history.

HirQ's ghost-risk estimate and company labels are automated opinions based on what we observe on employers' public job boards: how long a listing has been open, whether it was re-listed, wording in the description, and how often roles on that board close. They are not statements about an employer's intent, and a high-risk listing can still be a real opening. Figures are as of the date shown. How we measure

At a glance

Pay
No pay listed
Level
Mid level
Type
Fulltime Fixed Term
Team
AI
Work setup
Hybrid

“- Enjoy the perks: flexible hours, open holiday policy, an office in the heart of Amsterdam with hybrid flexibility, visa sponsorship, great gear, workations, and team events.”

Kaizo hiring

Company page
SteadyHiring44· WeakSoftware & SaaS
Open roles
3
New in 30 days
0
Closed in 30 days
0
Hiring trend, 13 weeksPro
Follow-through: ••%Pro

About the role

From Kaizo's careers page, checked Nov 21

Do you want to work out how you actually measure whether an AI system is doing a good job, and then make it better? We're looking for a sharp, analytical data scientist to own the evaluation and quality loop of our AutoQA product. You can be early in your career (recent graduates with strong LLM fundamentals are welcome), we have room and a clear path for you to grow.

In a nutshell

  • Join a fast-growing SaaS company in an international environment (steep learning curve guaranteed).
  • Own a high-impact problem: making LLM-powered quality assurance measurably accurate at scale.
  • Sit at the intersection of AI and CX, working directly with enterprise customers and their real-world QA rubrics.
  • Grow into a Senior Data Scientist, AI Engineer, or ML Engineer role. We invest in progression.
  • Enjoy the perks: flexible hours, open holiday policy, an office in the heart of Amsterdam with hybrid flexibility, visa sponsorship, great gear, workations, and team events.

About Kaizo

At Kaizo, we build a performance development and quality platform for customer support teams. Our AutoQA product uses LLMs to review support conversations against each customer's own quality rubric, automatically and at scale. Behind it sits a microservices-based stream processing platform handling over 200 million events per day (Kafka, Kubernetes on Google Cloud, ElasticSearch, MongoDB, BigQuery), and an LLMOps stack built around LangSmith for experimentation, prompt management, and tracing.

The hard part isn't calling an LLM. It's knowing, with evidence, how well the system performs on every customer's unique rubric, and having a reliable, repeatable way to improve it. That's where you come in.

What you'll focus on

  • Translate customer rubrics into AutoQA instructions. Work with real customer quality criteria and turn them into precise, testable instructions that LLMs can score reliably.
  • Run experiments that move accuracy. Design and execute evaluation experiments on large, representative datasets using LangSmith and BigQuery, and track quality with our performance metrics.
  • Build the datasets that make evaluation possible. Curate raw production data into golden datasets with balanced coverage, and generate synthetic data to cover the rare cases that matter most. QA is a discipline of rare occurrences: distributions are skewed, positives are scarce, and resourcefulness beats volume.
  • Build LLM-as-a-judge pipelines to assess system quality internally and make evaluation repeatable.
  • Make results actionable. Your experiments should end in a decision: change a prompt, adjust which tools the system uses, surface context the AI is missing, or flag where new capabilities are needed. You'll work with the AI team to ship those decisions.
  • Evaluate across the full stack. Beyond scoring quality, you'll help validate retrieval (RAG/IR), tool calling, and speech pipelines (transcription and diarization quality).
  • Join customer calls with the team to understand how QA leaders define quality, and feed what you learn back into the product.

What you'll grow into

  • Shaping how customers monitor quality themselves, catch drift, and keep their AutoQA setup improving over time.
  • Smarter categorization and routing of conversations to power analytics and get the right tickets to the right evaluation.

Skills mentioned

  • LLMs
  • Quality Assurance
  • BigQuery
  • MongoDB
  • Elasticsearch
  • Kafka
  • GCP
  • Kubernetes
  • Microservices
  • Machine Learning
  • RAG
  • A/B Testing
  • Customer Support
Apply on Kaizo's siteOpens Kaizo's own application page.

Similar jobs

Same role or field, nearby or remote, ranked by HirQ score.

More at Kaizo

All roles

Keep exploring

Data Scientist

No pay listed · HirQ 52

Apply