Skip to content

Distributed Training and Inference Engineer

SciforiumSan Francisco, CA

First seen Aug 24 · open 89 days · posted once$190k–$250k/yr

HirQ verdict

61/100 · Fair. Based on age, reposts, pay, the company's hiring and ghost signals.

Low ghost risk
True age
89 days
Reposts
Posted once

See the evidence behind this score.

Pro shows each part of the score, the reasons, the pay percentile and the listing history.

HirQ's ghost-risk estimate and company labels are automated opinions based on what we observe on employers' public job boards: how long a listing has been open, whether it was re-listed, wording in the description, and how often roles on that board close. They are not statements about an employer's intent, and a high-risk listing can still be a real opening. Figures are as of the date shown. How we measure

At a glance

Pay
$190k–$250k/yr
Level
Mid level
Experience
5+ years
Type
FullTime
Team
Engineering
Work setup
On-site

Pay vs market

Employer-posted ranges for Software Engineering roles in CA in the last 12 months (6,470 postings).

$174k
25th pct/yr
$210k
Median/yr
$253k
75th pct/yr
$140k · 10th90th · $300k

This job: $190k–$250k/yr

Where this pay sits: ••th percentilePro

Sciforium hiring

Company page
SteadyHiring49· WeakAI & Machine Learning
Open roles
15
New in 30 days
4
Closed in 30 days
4
Hiring trend, 13 weeksPro
Follow-through: ••%Pro

About the role

From Sciforium's careers page, checked Nov 22

Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct sponsorship from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the full stack powering frontier AI models and real-time applications.

ABOUT THE ROLE

Sciforium is seeking a highly skilled Distributed Training and Inference Engineer to build, optimize, and maintain the critical software stack that powers our large-scale AI training and serving workloads. In this role, you will work across the entire machine learning infrastructure from low-level CUDA/ROCm runtimes to high-level frameworks like JAX and PyTorch to ensure our distributed training systems are fast, scalable, stable, and efficient.

This position is ideal for someone who loves deep systems engineering, debugging complex hardware–software interactions, and optimizing performance at every layer of the ML stack. You will play a pivotal role in enabling the training and deployment of next-generation LLMs and generative AI models.

WHAT YOU'LL DO

  • Software Stack Maintenance: Maintain, update, and optimize critical ML libraries and frameworks including JAX, PyTorch, CUDA, and ROCm across multiple environments and hardware configurations.
  • End-to-End Stack Ownership: Build, maintain, and continuously improve the entire ML software stack from ROCm/CUDA drivers to high-level JAX/PyTorch tooling.
  • Distributed System Optimization: Ensure all model implementations are efficiently sharded, partitioned, and configured for large-scale distributed training and serving.
  • System Integration: Continuously integrate and validate modules for runtime correctness, memory efficiency, and scalability across multi-node GPU/accelerator clusters.
  • Profiling & Performance Analysis: Conduct detailed profiling of compilation graphs, training workloads, and runtime execution to optimize performance and eliminate bottlenecks.
  • Debugging & Reliability: Troubleshoot complex hardware–software interaction issues, including vLLM compilation failures on ROCm, CUDA memory leaks, distributed runtime failures, and kernel-level inconsistencies.
  • Collaborate with research, infrastructure, and kernel engineering teams to improve system throughput, stability, and developer experience.

IDEAL CANDIDATE PROFILE

  • 5+ years of industry experience in ML systems, distributed training, or related fields.
  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related technical fields.
  • Strong programming experience in Python, C++, and familiarity with ML tooling and distributed systems.
  • Deep understanding of profiling tools (e.g., Nsight, ROCm Profiler, XLA profiler, TPU tools).
  • Deep expertise with partitioning configuration on the modern ML frameworks such as PyTorch and JAX.
  • Experience with multi-node distributed training systems and orchestration frameworks (DTensor, GSPMD, etc.).
  • Hands-on experience maintaining or building ML training stacks involving CUDA, ROCm, NCCL, XLA, or similar technologies.

NICE-TO-HAVE

  • Extensive experience with the XLA/JAX stack, including compilation internals and custom lowering paths.
  • Familiarity with distributed serving or large-scale inference frameworks (e.g., vLLM, TensorRT, FasterTransformer).
  • Background in GPU kernel optimization or accelerator-aware model partitioning.
  • Strong understanding of low-level C++ building blocks used in ML frameworks (e.g., XLA, CUDA kernels, custom ops).

BENEFITS INCLUDE

  • Medical, dental, and vision insurance
  • 401k plan
  • Daily lunch, snacks, and beverages
  • Flexible time off
  • Competitive salary and equity

EQUAL OPPORTUNITY

Sciforium is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Skills mentioned

  • Machine Learning
  • PyTorch
  • C++
  • Python
  • Distributed Systems
  • LLMs
  • Generative AI
  • Electrical Engineering
Apply on Sciforium's siteOpens Sciforium's own application page.

Similar jobs

Same role or field, nearby or remote, ranked by HirQ score.

More at Sciforium

All roles

Keep exploring

Distributed Training and Inference Engineer

$190k–$250k/yr · HirQ 61

Apply