Skip to content

Software Engineer, Inference Runtime

Lm StudioNew York CityRemote

First seen Aug 7 · open 105 days · posted once$150k–$350k/yr

HirQ verdict

55/100 · Fair. Based on age, reposts, pay, the company's hiring and ghost signals.

Medium ghost risk
True age
105 days
Reposts
Posted once

See the evidence behind this score.

Pro shows each part of the score, the reasons, the pay percentile and the listing history.

HirQ's ghost-risk estimate and company labels are automated opinions based on what we observe on employers' public job boards: how long a listing has been open, whether it was re-listed, wording in the description, and how often roles on that board close. They are not statements about an employer's intent, and a high-risk listing can still be a real opening. Figures are as of the date shown. How we measure

At a glance

Pay
$150k–$350k/yr
Level
Mid level
Type
FullTime
Team
Engineering
Work setup
Remote

“New York City”

Pay vs market

Employer-posted ranges for Software Engineering roles in NY in the last 12 months (2,034 postings).

$175k
25th pct/yr
$210k
Median/yr
$243k
75th pct/yr
$141k · 10th90th · $285k

This job: $150k–$350k/yr

Where this pay sits: ••th percentilePro

Lm Studio hiring

Company page
FrozenHiring50· FairMedia, Gaming & Entertainment
Open roles
6
New in 30 days
0
Closed in 30 days
0
Hiring trend, 13 weeksPro
Follow-through: ••%Pro

About the role

From Lm Studio's careers page, checked Nov 20

LM Studio is used by millions of people around the world to run AI on their own computers, and now with Bionic - also in the cloud. Our values prioritize putting the human in the center, and creating tools that we want to use ourselves, and recommend to our friends and family.

As a team, we work with high technical intensity and personal responsibility. We are looking for curious, self-motivated, creative, and technically excellent teammates to join us and build the future of human-AI interactions in software.

The Role

We are looking for an Inference Runtime Software Engineer to push forward LM Studio's inference stack on-device and in the cloud. You will integrate new inference engines and runtime capabilities, bring up new open-weight models and modalities, and optimize model execution for a wide range of CPU and GPU targets. You will also contribute improvements to the open-source projects we build on.

Qualifications

  • Significant experience building production ML systems, inference runtimes, or performance-sensitive infrastructure
  • Strong programming ability in Python and C++
  • Deep understanding of transformer architectures and the mechanics of model inference
  • Experience profiling CPU or GPU workloads and reasoning about compute, memory, synchronization, and data movement
  • Experience with PyTorch and inference systems such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM
  • Strong debugging instincts across model code, runtime internals, operating systems, and CPU or GPU execution
  • Takes personal responsibility for the correctness and performance of their work

Bonus Qualifications

  • Past contributions to open-source inference runtime projects such as llama.cpp, MLX, ExecuTorch, vLLM, SGLang, or TensorRT-LLM

Responsibilities

  • Maintain and push forward our inference stack on-device and in the cloud
  • Bring up new model architectures and multimodal models
  • Improve latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimes
  • Build runtime capabilities for model loading, batching, scheduling, caching, and distributed execution
  • Benchmark and diagnose correctness and performance problems across the inference stack
  • Contribute upstream to open-source projects such as llama.cpp and MLX

Benefits

  • Competitive salary and equity grants
  • Great medical, vision, dental healthcare plans
  • Catered team lunch / expensed dinners in the office
  • Flexible PTO
  • Flexible WFH
  • Sun-drenched office in SoHo in NYC

Skills mentioned

  • LLMs
  • Python
  • C++
  • Machine Learning
  • PyTorch
Apply on Lm Studio's siteOpens Lm Studio's own application page.

Similar jobs

Same role or field, nearby or remote, ranked by HirQ score.

More at Lm Studio

All roles

Keep exploring

Software Engineer, Inference Runtime

$150k–$350k/yr · HirQ 55

Apply