Lead Data Consultant - AWS / Databricks

About Scale Factory

We are an AI enablement consultancy. Delivering quality-led AI-first foundations, embedding capability and excellence into the foundational fabric of the companies we work with. We give leaders the confidence to move faster, take smarter risks and pursue transformation.

Formerly Ten10, we have deep roots in Quality which means we don't just build for the future, we ensure it actually works. We combine this engineering discipline with cutting-edge AI expertise to deliver robust, scalable foundations that turn ambitious transformation into reliable reality.

Role Purpose

We see data as the foundations for AI and are growing our data practice to help businesses embrace the future and make their data fit for the AI age. We are seeking an AI literate (and willing to learn / gain certifications in this important new area) Data Leader. You will act as a senior technical lead and hands-on data architect on client engagements.

You'll work directly with client engineering teams, data leads, and technical stakeholders to design, build, and deploy production-grade data platforms and AI applications. Rooted in the codebase, you will lead delivery teams by example, ensuring high engineering standards, robust software practices, and architectural alignment. Off-project, you will contribute directly to shaping our technical practice by building reference implementations, accelerators, and reusable delivery assets.

Technical Architecture & Delivery

• Design end-to-end target data platforms (ingestion, storage, transformation, serving, and governance) in alignment with overall technical direction.

• Make and document key architectural decisions and technical trade-offs (e.g., lakehouse vs. warehouse, AI context stores).

• Translate business and functional requirements into scalable, resilient data pipeline architectures and detailed technical roadmaps.

• Embed non-functional requirements, data security, privacy, lineage, quality controls, and FinOps guardrails directly into the engineering design.

Hands-on Engineering & Technical Leadership

• Serve as the Tech Lead on client engagements where required, guiding delivery teams and unblocking technical hurdles.

• Write, review, and maintain production-grade code.

• Establish and champion practice engineering standards, including unit/integration testing, CI/CD pipelines, automated deployments, environments, and observability.

• Mentor, coach, and support mid-level and senior engineers to elevate the overall technical output of the team.

Client Engagement & Technical Advisory

• Facilitate technical discovery and architecture alignment sessions with client engineering teams and technical leads.

• Act as a trusted technical advisor on trade-offs, performance optimization, refactoring, and risk mitigation.

• Communicate architectural concepts, decision records, and technical trade-offs clearly to both technical and non-technical stakeholders.

Practice Building & Knowledge Sharing

• Build reusable delivery assets, reference architectures, code accelerators, and technical playbooks.

• Assist with interviewing and technical assessments for prospective data engineering candidates.

• Collaborate across cloud infrastructure and testing practice groups to ensure seamless technical cross-pollination.

Essential Experience

• 6+ years in data architecture and technical delivery, with hands-on experience leading data transformation projects.

• Modern Data Platform Delivery: Proven experience building and deploying scalable data platforms end-to-end on AWS and/or Databricks.

• Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools.

• Consulting / Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability to project delivery outcomes.

• Data Modeling: Solid depth in dimensional modeling, NoSQL schemas, and lakehouse/medallion architecture patterns.

• Data Governance & Operations: Practical experience implementing data catalogues, lineage, access controls, data quality checks, and monitoring.

• Cost Optimization: Experience implementing cost controls, platform monitoring, and optimizing data compute/storage expenses.

• Clearance: SC clearance, or eligibility for it.

Desirable

• Certifications: Databricks and/or AWS certifications at Associate or Professional level.

• Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming).

• Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models.

• AI/ML Infrastructure: Exposure to AI/ML data foundations, including feature stores, MLflow, model serving, or retrieval pipelines (RAG).

• Regulated Sectors: Experience working within regulated environments (Financial Services, Healthcare, Public Sector).

How You Work

• Pragmatic & Objective: Recommends practical, elegant solutions over unnecessary complexity.

• Engineering-Focused: Leads with working code, rigorous standards, and sound engineering practices.

• Clear Communicator: Produces clear technical documentation, architecture diagrams, and decision records.

• Collaborative Coach: Enjoys mentoring team members and sharing knowledge to raise collective engineering capability.

• Adaptable: Comfortable navigating technical ambiguity within fast-paced or evolving environments.

What’s in It for You?

At Scale Factory, we believe in recognising and rewarding great work. Here’s what you can expect:

• 25 Days of Annual Leave: Plus 1 extra day every year for the first three years, with a holiday buying scheme.

• Pension Plan: Employer-matched contributions up to 5%.

• Health Benefits: Comprehensive coverage for medical, dental, optical, and alternative therapies.

• Fitness Support: Subsidised gym memberships and a bike-to-work scheme.

• Commuter Benefits: Season ticket loans to make travel easier.

• Free Annual Rail Cards

• Electric vehicle salary sacrifice scheme

• Flexible Work Model: Hybrid working that fits your lifestyle.

• Social Events: From annual kick-offs and Christmas parties to team socials and sporting events, we love celebrating success together.

• Training & Development: Tailored learning opportunities and full support for certifications to keep your skills sharp.

Let’s Build the Future Together