Skip to main content
David Careers

Azure Databricks Engineer

Posted yesterday

zeus solutionsHouston (TX)
Data Warehousing SpecialistsCustom Computer Programming Services

SENIORITY

Lead

Apply (opens in a new tab)

About the role

Location: Onsite - Houston Employment Type: Full-TimeAbout the RoleWe are seeking a highly skilled, hands-on Databricks Lead with 8+ years of experience in data engineering, including deep expertise in Azure Databricks, PySpark, and Structured Streaming. This role is ideal for a senior engineer with a programming-first mindset, a strong understanding of distributed systems, and the ability to build high-performance, cost-optimized data solutions. This is not a traditional ETL role. It requires extensive knowledge of Spark internals, declarative pipeline development, and real-time data processing. The successful candidate will also provide technical leadership and collaborate closely with cross-functional teams to deliver robust, scalable data solutions.
Key Responsibilities:
  • Lead the design and development of scalable, high-performance data pipelines, streaming tables, and Delta Live Tables (DLT) using Azure Databricks and PySparkDrive Spark performance tuning and implement cost-optimization strategies within the Databricks environment
  • Build and manage real-time and batch workflows using Structured Streaming
  • Leverage Delta Live Tables (DLT) and Lakehouse Declarative Pipelines (LDP) to build scalable, reliable, and maintainable data pipelines
  • Collaborate with data architects, analysts, and business stakeholders to deliver robust data solutions
  • Provide technical leadership through mentorship, code reviews, and architectural guidance
  • Ensure system reliability and performance through proactive monitoring and engineering best practices
Must-Have: Qualifications 8+ years of hands-on experience in data engineering, big data development, or a related field Extensive hands-on experience with Azure Databricks and PySparkStrong programming background and in-depth knowledge of distributed data processing beyond traditional ETL tools Proven expertise in:
  • Spark performance tuning and optimization
  • Cost management within Databricks
  • Structured Streaming for real-time data processing
  • Lakehouse Declarative Pipelines (LDP) and Delta Live Tables (DLT)
  • Familiarity with the Azure data ecosystem, including ADLS, Azure Data Factory, and Synapse
  • Excellent communication skills and the ability to collaborate effectively with cross-functional teams
  • Willingness and ability to work onsite in Houston
  • Nice-to-Have Qualifications
  • Industry experience in energy, utilities, or heavy industry
  • Knowledge of data governance, security, and monitoring

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this