Remote Jobs RockRemote Jobs Rock

Sr. Staff Engineer, Lustre

🕒 12 days ago
LustrefsC ProgrammingLinux Systems DebuggingLinux Kernel Internals
Apply Now

📜 Description

  • Design, develop, and debug LustreFS features, fixes, and enhancements across subsystems such as llite, MDS/MDT, OSS/OST, LDLM, and LNet.
  • Investigate customer and scale-related defects, identify root causes, and deliver maintainable fixes.
  • Tune performance and improve reliability through failure analysis and workload characterization.
  • Collaborate with QE and support to reproduce issues, improve diagnostics, and test high-risk failure scenarios.
  • Review designs and code, document subsystem behavior and operational practices, and use AI-assisted tools to accelerate triage and debugging.

🛠️ Requirements

  • 10+ years of experience in systems software, distributed systems, storage, Linux kernel or filesystem engineering.
  • Strong experience in LustreFS development, support or performance engineering with depth in at least one major subsystem.
  • Strong C programming and Linux systems debugging skills.
  • Working knowledge of Linux kernel internals, filesystem semantics, networking and performance analysis.
  • Experience with LNet and/or high-performance transports such as RDMA, InfiniBand, RoCE or TCP-based storage networking.
  • Ability to debug and resolve issues spanning multiple layers including client, server, network and backend storage.
  • Strong collaboration skills and the ability to work across functions in a fast-moving engineering environment.
  • Experience in HPC, AI infrastructure or large-scale parallel storage environments.
  • Exposure to metadata-heavy and throughput-heavy workload characterization and tuning.
  • Familiarity with ZFS, ldiskfs, NVMe-backed storage and related observability / performance tooling.
Full job description

We are seeking a Staff Engineer with 10+ years of experience in distributed storage and Linux-based systems engineering. This is a hands-on senior technical role focused on design, debugging, performance, and operational excellence across LustreFS and adjacent stack components. The ideal candidate brings strong expertise in one or more Lustre subsystems, can independently drive complex investigations, and collaborates effectively across engineering, QE, support and release teams. Engineers who are comfortable using AI to accelerate triage, debugging, code comprehension and new feature design will be especially valuable.

 

Key Responsibilities

  • Design, develop and debug LustreFS features, fixes and enhancements across relevant subsystems such as llite, MDS/MDT, OSS/OST, LDLM and LNet.

  • Investigate customer and scale-related defects, drive root-cause analysis and implement high-quality fixes with strong attention to correctness and maintainability.

  • Contribute to performance tuning, failure analysis and reliability improvements for large-scale Lustre deployments.

  • Participate actively in code reviews, design reviews and subsystem discussions, bringing rigor to testing and operational readiness.

  • Work closely with QE and support to reproduce issues, improve diagnostic data quality and increase coverage for high-risk failure scenarios.

  • Help document subsystem behavior, debugging approaches, known failure patterns and operational best practices.

  • Use AI-assisted tools where appropriate to speed up issue triage, summarize logs, improve code understanding and capture reusable lessons learned.

 

Required Qualifications

  • 10+ years of experience in systems software, distributed systems, storage, Linux kernel or filesystem engineering.

  • Strong experience in LustreFS development, support or performance engineering with depth in at least one major subsystem.

  • Strong C programming and Linux systems debugging skills.

  • Working knowledge of Linux kernel internals, filesystem semantics, networking and performance analysis.

  • Experience with LNet and/or high-performance transports such as RDMA, InfiniBand, RoCE or TCP-based storage networking.

  • Ability to debug and resolve issues spanning multiple layers including client, server, network and backend storage.

  • Strong collaboration skills and the ability to work across functions in a fast-moving engineering environment.

 

Preferred Skills

  • Experience in HPC, AI infrastructure or large-scale parallel storage environments.

  • Exposure to metadata-heavy and throughput-heavy workload characterization and tuning.

  • Familiarity with ZFS, ldiskfs, NVMe-backed storage and related observability / performance tooling.

  • Experience creating test plans, reproducer frameworks, runbooks or diagnostic automation.

  • Comfort using AI tools to accelerate debugging, code reviews, triage, documentation and early-stage design ideation.

  • Experience mentoring junior engineers or leading focused technical efforts within a subsystem.

 

What You Will Work On

  • Hands-on development and debugging of LustreFS defects, performance issues and subsystem enhancements.

  • Customer-facing and scale-related issue investigation across llite, metadata, object storage, LNet and transport layers.

  • Collaborative design and implementation of reliability, observability and serviceability improvements.

  • Reviewing and validating fixes through targeted tests, failure injection, log analysis and performance characterization.

  • Using AI-assisted workflows to accelerate triage, debug loops, code understanding and documentation quality.

  • Contributing to team redundancy by strengthening documentation, code review quality and subsystem knowledge sharing.

     

Why This Role Matters

This role is central to building durable engineering redundancy in LustreFS: expanding deep subsystem ownership, reducing concentration risk, and accelerating next-generation delivery through strong engineering fundamentals and AI-enabled execution.

 

Atlan

Principal Software Engineer - Agent & App Platform

🕒 yesterday
Atlan👥 201 - 500 employees🏢 Computer Software

Sets technical direction for Atlan’s agent registry and gateway platform, building reliable, multi-tenant infrastructure and shared primitives for registering, executing, distributing, and governing apps and AI agents.

GoPythonDistributed SystemsMulti-tenant Architecture
Vanta

Staff Software Engineer, AI and Data Risk

🕒 yesterday
Vanta👥 10,000+ employees🏢 Software Development🤝 B2B

Leads cross-team technical initiatives and builds scalable product functionality and infrastructure for privacy, AI governance, and risk management, while shaping engineering practices and mentoring teams.

Software EngineeringTechnical LeadershipTypeScriptReact
ServiceTitan

Principal Software Engineer, Platform (DevEx)

🕒 2 days ago
ServiceTitan👥 1001 - 5000 employees🏢 Software

Join ServiceTitan as a Principal Software Engineer to lead the development of an AI-enabled Developer Platform, setting technical direction and influencing engineering practices across teams.

AI-enabled Development ToolsCoding Agent RuntimesAgent OrchestrationLLM Reasoning Loops
ServiceTitan

Staff Software Engineer, Platform (DevEx)

🕒 2 days ago
ServiceTitan👥 1001 - 5000 employees🏢 Software

ServiceTitan is seeking a Staff Software Engineer to lead the technical direction of its Developer Platform, enhancing software delivery for enterprise-scale SaaS.

Software EngineeringTechnical LeadershipCI/CD PipelinesBuild Systems

Trusted by Remote Workers