Senior Data Engineer
$140,000 - $150,000 per year
Apply (opens in a new tab)Senior Data Engineer
Posted today
SENIORITY
Lead
SALARY
$140,000 - $150,000 per year
About the role
- Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Apache Airflow, processing structured and semi-structured data across the Medallion architecture (Bronze → Silver → Gold).
- Implement incremental load patterns, change data capture (CDC), and event-driven ingestion to ensure data freshness across the platform.
- Build and optimise Snowflake data warehouse objects — tables, views, dynamic tables, streams, tasks, and stored procedures — for performance and cost efficiency.
- Develop modular, tested dbt models aligned to each Medallion layer, enforcing consistent naming conventions, documentation, and lineage across all transformations. Data Quality & Observability
- Embed automated data validation at every Medallion layer using Elementary (dbt's observability layer), ensuring anomaly detection, freshness checks, and schema drift alerts are in place before data reaches consumers.
- Define and enforce data contracts between producers and consumers — row count checks, null rate thresholds, referential integrity, and value domain validation.
- Build and maintain data quality dashboards to give engineering and business stakeholders real-time confidence in platform health. Azure Cloud Infrastructure
- Manage and optimise Azure Data Lake Storage Gen2 (ADLS) — folder structures, lifecycle policies, access tiers, and partition strategies.
- Build and maintain Azure Functions and Azure Logic Apps for lightweight event-driven processing, orchestration triggers, and operational automation.
- Manage secrets, credentials, and environment-specific configuration securely using Azure Key Vault — no hardcoded credentials in pipelines or code.
- Contribute to infrastructure-as-code practices for provisioning Azure data services (Terraform or Bicep preferred). Collaboration & Delivery
- Translate ambiguous business requirements into well-defined data models and pipeline designs, working with analysts and stakeholders to validate assumptions before build.
- Participate in code reviews, enforce standards, and mentor junior engineers on data engineering best practices.
- Support CI/CD adoption for pipeline and dbt model deployment across Dev / Test / Prod environments.
- dbt +Elementary: dbt + Elementary– Writing, testing, and documenting production dbt models– Elementary integration for data observability and anomaly detection– dbt incremental strategies, snapshots, and semantic layer▸Azure Cloud: Azure Cloud– Azure Data Factory — pipeline authoring, triggers, parameterisation, linked services– ADLS Gen2 — zone/folder design, lifecycle management, Parquet/Delta partitioning– Azure Key Vault — secret management, managed identities– Azure Functions / Logic Apps — event-driven triggers and lightweight automation▸Airflow: Airflow– DAG authoring, task dependencies, XCom, sensors, and connection management– Airflow deployment and monitoring in cloud-hosted environments▸Python: Python– Data pipeline scripting, PySpark basics, REST API integration– Unit testing pipeline logic and transformation functions
- Data Quality & Medallion Architecture: Medallion Architecture:– Hands-on experience implementing Bronze / Silver / Gold Medallion architecture– Data validation checks at each layer — not just at the final Gold layer– Schema evolution handling and SCD Type 2 dimension management
- 4+ years of professional data engineering experience with at least 2 years on Azure cloud data platforms. Nice-to-Have
- Exposure to Snowflake Cortex, dbt Semantic Layer, or Boomi Data Hub for AI-assisted data enrichment within pipeline layers.
- Experience integrating LLM-based quality checks or AI-assisted anomaly detection into data workflows.
- Familiarity with Microsoft Fabric and OneLake as a complementary or future-state platform.
- Knowledge of data mesh or data product thinking and how it maps to Medallion layer ownership.
- Experience with Terraform or Bicep for Azure infrastructure provisioning.
- dbt +Elementary: dbt + Elementary– Writing, testing, and documenting production dbt models– Elementary integration for data observability and anomaly detection– dbt incremental strategies, snapshots, and semantic layer▸Azure Cloud: Azure Cloud– Azure Data Factory — pipeline authoring, triggers, parameterisation, linked services– ADLS Gen2 — zone/folder design, lifecycle management, Parquet/Delta partitioning– Azure Key Vault — secret management, managed identities– Azure Functions / Logic Apps — event-driven triggers and lightweight automation▸Airflow: Airflow– DAG authoring, task dependencies, XCom, sensors, and connection management– Airflow deployment and monitoring in cloud-hosted environments▸Python: Python– Data pipeline scripting, PySpark basics, REST API integration– Unit testing pipeline logic and transformation functions
- Data Quality & Medallion Architecture: Medallion Architecture:– Hands-on experience implementing Bronze / Silver / Gold Medallion architecture– Data validation checks at each layer — not just at the final Gold layer– Schema evolution handling and SCD Type 2 dimension management
- 4+ years of professional data engineering experience with at least 2 years on Azure cloud data platforms. Nice-to-Have
- Exposure to Snowflake Cortex, dbt Semantic Layer, or Boomi Data Hub for AI-assisted data enrichment within pipeline layers.
- Experience integrating LLM-based quality checks or AI-assisted anomaly detection into data workflows.
- Familiarity with Microsoft Fabric and OneLake as a complementary or future-state platform.
- Knowledge of data mesh or data product thinking and how it maps to Medallion layer ownership.
- Experience with Terraform or Bicep for Azure infrastructure provisioning.
- Comprehensive benefits plan including medical, dental, vision, disability, and life insurance
- Healthcare HSA
- HSA with employer contribution
- 10 paid holidays
- 10 days of paid annual leave
- 9 days of paid sick time
- Paid parental leave
- 401(k) with employer match Other Perks
- Education assistance program
- Dependent Care HSA match
- Adoption assistance
- Fertility care support
- Backup care for family and pets
- A growing network of employee resource groups
- Employee referral program
- Employee Assistance Program We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
