Senior Data Engineer (Databricks)
Senior Data Engineer (Databricks)
Posted today
SENIORITY
Senior
About the role
- ImpactDesign, develop, and optimize robust data pipelines in Databricks to process and transform large-scale data sets
- Collaborate with data scientists, engineers, and cloud architects to build reliable, secure data systems
- Automate infrastructure provisioning using Terraform and manage resources on Azure (or AWS/GCP)
- Monitor, troubleshoot, and support production-grade data pipelines and jobs
- Develop CI/CD workflows to streamline deployment and version control of data processing code
- Ensure security, performance, and reliability across all stages of data workflow
- Utilize and manage Docker containers where needed to deploy data services
- Participate in Agile team ceremonies and help shape iterative data delivery plans
- Support continuous improvement in data architecture, automation, and governance practices
- Other duties as reasonably required
- Essential Experience 5+ years of experience in data engineering or backend data infrastructure roles 2+ years of hands-on experience with Databricks, Spark, or distributed data processing systems in a production environment
- Deep proficiency with PySpark and Python for data engineering tasks
- Experience with data modeling
- Solid experience with Airflow, dbt, or other orchestration and transformation tools
- Proven skills in data governance, lineage, or security best practices
- Experience working with Azure (preferred), AWS, or Google CloudStrong working knowledge of CI/CD tools and version control workflows
- Comfortable working with relational (e.g., PostgreSQL, MySQL) and NoSQL databases
- Proven experience in monitoring, debugging, and scaling cloud-based data systems
- Strong communication skills and ability to work independently in a remote team environment
- Kudos If You Have: Experience with Kubernetes and Helm for deploying data workloads (not required)
- Solid experience with Terraform or other Infrastructure as Code tools
- Familiarity with containerization using Docker
- Knowledge of data lakehouse architectures and Delta Lake
- Experience working on Federal or government projects
- Existing security clearance (preferred but not required)
- Databricks Certification
- 401(k)401(k) matching
- Dental insurance
- Health insurance
- Paid time off
- Retirement plan
- Vision insurance
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
