Staff Machine Learning Engineer

Locations Supported 🌍

• London, UK

Relocation available: No

Work pattern: Hybrid: our teams meets in the office ~1-2 days a week

About the Opportunity

Every transaction we process requires a real-time decision. Declining a legitimate transaction leaves a customer stuck at the point of purchase, while approving a fraudulent one carries a direct cost.

This role owns the decisioning system and underlying platform. From the serving path and feature infrastructure to the underlying models and the machinery required to make safe, live updates. You will continuously improve the platform and our day to day workflows, rather than treating these as secondary projects.

As a Staff Machine Learning Engineer, you will hold a hands-on technical position. You will be part of a team that builds, ships, and maintains the entire machine learning lifecycle.

Our main focus is fraud detection and prevention, an adversarial domain where opponents constantly adapt and feedback arrives in the form of financial impact. Alongside, this we build broader capabilities to enable machine learning across Moonpay.

Lead through ambiguity

• Turn vague problems into well-defined solutions and bring people with you.

• Set the technical bar through rigorous reviews, clear standards, and lasting engineering habits.

Build and scale the platform

• Develop feature infrastructure across batch, near-real-time, and in-request paths, managing specific freshness budgets for each.

• Maintain alignment between training and serving to ensure models behave in production exactly as they did offline.

• Integrate feedback loops to capture every decision and its outcome, including blocked transactions where results are counterfactual.

• Scale the platform as volume and model complexity grow, ensuring operational load remains manageable.

Decide in real time

• Own the services that score transactions in-flight, inside a hard latency budget

• Design the degraded paths: what we answer when the model can't, and who agreed that policy

Ship safely, continuously

• Mature the replay, shadow and staged-rollout tooling until changing a live model is routine and reversible

• Own models across their lifecycle, from training through to retirement, and catch decay long before losses confirm it

About You

Must-have experience and skills

• Real-time serving. You have built and operated high-availability services that execute within strict latency budgets on critical paths, and you’ve designed robust fallback mechanisms

• Systems thinking. You view the architecture holistically: identifying failure points, managing graceful degradation, and ensuring the system remains responsive even when dependencies fail. You build the feedback loops that allow a system to learn from its own decisions.

• Engineering craft. You write code other people are happy to inherit — tested, typed, and correct when events arrive twice, late, or out of order. Adding the next feature to something you built is fast and painless.

• Pipelines in production. You have owned feature or data pipelines end-to-end, including troubleshooting cases where offline and production metrics diverged and resolving the underlying discrepancies.

• Ambiguity and influence. You've taken a problem nobody had scoped and turned it into work that shipped, and raised the level of the engineers around you while doing it.

Nice-to-have experience

• Decision explainability. You've built systems where the reason for a decision mattered as much as the decision: audit trails, per-layer attribution, llm-driven analyses, or defending a model's behaviour to a non-technical audience.

• Anomaly detection. You have developed systems to detect novel attack patterns and emerging abuse without existing labels, identifying suspicious behavior relative to historical baselines.

• Familiarity with our stack: GCP, BigQuery, Bigtable, Memorystore, Vertex AI, Kubernetes.