Manager-Data Platforms
buchanan ingersoll rooneyPittsburgh (PA)
Manager-Data Platforms
Posted today
buchanan ingersoll rooneyPittsburgh (PA)
SENIORITY
Manager
About the role
Overview:
In this role you will lead the design, build, and optimization of enterprise data platforms on Databricks within Azure. You’ll guide a data engineering team and partner with data scientists and stakeholders to deliver scalable, high-performance pipelines and analytics solutions. The position focuses on data governance, quality, and robust architecture to enable data-driven decision-making across the firm. You will operate at cloud scale, balancing performance with cost, and mentor teams to adopt best practices. This is a senior, impact-oriented role in a data-centric transformation. Compensation / Benefits Hybrid work model Medical, Dental, Vision 401K Program Paid Time Off Holidays including floating holiday Wellness program
Responsibilities:
Lead and mentor a team of data engineers, conducting code reviews and ensuring development standards Design, develop and deploy scalable data pipelines on Azure Databricks for data ingestion, processing, and loading into the Azure Databricks Data Warehouse Collaborate with data scientists, analysts, and stakeholders to translate requirements into technical specifications Optimize pipeline performance, cost, and scalability in Azure cloud Define data ingestion, processing, storage, and governance best practices; implement data quality checks Operationalize machine learning models with data scientists and analysts Architect end-to-end Lakehouse with Delta Lake and medallion (Bronze/Silver/Gold); model schemas Oversee batch and streaming ETL/ELT pipelines using PySpark, Scala, SQL; integrate Kafka and ADF Implement Unity Catalog for governance, RBAC, privacy, and data lineage Tune Spark jobs and Databricks clusters for throughput and cost efficiency; apply partitioning and optimization techniques Integrate Databricks with ADF, ADLS Gen2, and Azure DevOps for CI/CD pipelines
Key requirements:
5-7+ years of hands-on data engineering or architecture experience 2-4 years of experience managing a team of data engineers, data scientists or analysts Bachelor in Computer Science or related field Microsoft Certified: Azure Data Engineer Associate (DP-203) and/or Azure Solutions Architect Expert; Databricks Certified Data Engineer Professional (preferred) Strong programming in Python (PySpark), SQL, Scala; PowerShell Hands-on experience with Azure Databricks and related Azure data services (ADLS Gen2, Azure SQL, ADF, Synapse) Knowledge of relational and NoSQL databases; data modeling and schema design Experience with big data tools like Spark, Kafka; familiarity with Airflow or similar orchestration tools Understanding of security, RBAC, Unity Catalog, data governance and data lineage Cloud infrastructure experience including VNETs, Key Vault, and identity management leadership and mentoring cross-functional collaboration problem-solving and critical thinking Azure Databricks Azure Data Factory Azure Data Lake Storage Gen2
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.