Skip to content

Inference Infrastructure Engineer, Serving

Elorian Ai IncPalo AltoRemote

First seen Jul 22 · open 121 days · posted once$200k–$400k/yr

HirQ verdict

71/100 · Strong. Based on age, reposts, pay, the company's hiring and ghost signals.

Low ghost risk
True age
121 days
Reposts
Posted once

See the evidence behind this score.

Pro shows each part of the score, the reasons, the pay percentile and the listing history.

HirQ's ghost-risk estimate and company labels are automated opinions based on what we observe on employers' public job boards: how long a listing has been open, whether it was re-listed, wording in the description, and how often roles on that board close. They are not statements about an employer's intent, and a high-risk listing can still be a real opening. Figures are as of the date shown. How we measure

At a glance

Pay
$200k–$400k/yr
Level
Mid level
Experience
3+ years
Type
FullTime
Team
All Departments
Work setup
Remote

“Palo Alto”

Visa
Mentions sponsorship

“- Visa sponsorship: We sponsor visas.”

Pay vs market

Employer-posted ranges for DevOps & Cloud roles in CA in the last 12 months (1,259 postings).

$160k
25th pct/yr
$193k
Median/yr
$227k
75th pct/yr
$130k · 10th90th · $275k

This job: $200k–$400k/yr

Where this pay sits: ••th percentilePro

Elorian Ai Inc hiring

Company page
Actively hiringHiring70· StrongAI & Machine Learning
Open roles
10
New in 30 days
7
Closed in 30 days
3
Hiring trend, 13 weeksPro
Follow-through: ••%Pro

About the role

From Elorian Ai Inc's careers page, checked Nov 21

ABOUT US

We are a well-funded, early-stage AI lab focused on building the next generation of frontier multimodal AI models. Founded by former DeepMind researchers, including Andrew Dai, who was previously a leader on Gemini. Our team currently consists of 20 world-class scientists and engineers. We recently raised $55M in seed funding from Striker Ventures, Menlo Ventures, Altimeter Capital, and NVIDIA. We are tackling some of the hardest problems in artificial intelligence, and we are growing fast.

ABOUT THE ROLE

We're looking for an infrastructure engineer to design, optimize, and scale the systems that serve our large multimodal models. Your work will make inference faster, more cost-effective, and more reliable, so our teams can focus on advancing model capabilities rather than managing bottlenecks.

Our focus is on performant, efficient inference, both to power real-world applications and to accelerate research. This role owns the infrastructure that ensures every deployment and evaluation runs smoothly at scale for our visual foundation models.

WHAT YOU WILL DO

  • Build low-latency, high-throughput inference serving systems for our large multimodal models
  • Design and implement techniques that improve latency, throughput, and efficiency, including quantization, batching, speculative decoding, and KV cache management
  • Optimize our codebase and GPU fleet to fully utilize hardware FLOPs, bandwidth, and memory
  • Implement multi-GPU and multi-node model parallelism for serving (tensor or pipeline parallel)
  • Build autoscaling and load balancing for production ML services
  • Establish standards for reliability, observability, and reproducibility across the inference stack
  • Collaborate with researchers to enable high-performance inference for novel architectures

SKILLS AND QUALIFICATIONS

Minimum qualifications

  • 3+ years of experience building low-latency, high-throughput inference serving systems for large models
  • Strong knowledge of inference optimization techniques (quantization, batching, speculative decoding, KV cache management)
  • Hands-on experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang
  • Experience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline parallel)
  • Strong systems programming skills; C++/CUDA a plus alongside Python
  • Experience with autoscaling and load balancing for production ML services
  • A track record of GPU cost optimization at scale

Preferred qualifications (strong candidates may have some, not all)

  • Experience serving multimodal (vision + language) models
  • Contributions to open-source ML or systems infrastructure projects (e.g., vLLM, SGLang, TensorRT-LLM, Triton)
  • A bias for action and comfort working across stacks and teams in an early-stage environment

LOGISTICS

  • Location: This role is based on-site in Palo Alto, California.
  • Compensation: Depending on background, skills, and experience, the expected annual base salary range for this position is $200,000 - $400,000 USD, plus equity and benefits.
  • Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
  • Benefits: We offer health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.

Elorian AI is an equal opportunity employer. We are committed to building a diverse team and inclusive environment.

Skills mentioned

  • Machine Learning
  • LLMs
  • Python
  • C++
  • Observability
Apply on Elorian Ai Inc's siteOpens Elorian Ai Inc's own application page.

Similar jobs

Same role or field, nearby or remote, ranked by HirQ score.

More at Elorian Ai Inc

All roles

Keep exploring

Inference Infrastructure Engineer, Serving

$200k–$400k/yr · HirQ 71

Apply