Skip to main content
David Careers

Databricks Architect (Lead Data Platform Architect)

Posted 2 days ago

ilink digitalNew York (NY)
Database ArchitectsComputer Systems Design Services

SENIORITY

Manager

Apply (opens in a new tab)

About the role

Databricks Architect (Lead Data Platform Architect) New York, United States | Posted on 06/25/2026We are looking for a highly skilled Databricks Architect to design, build, and scale enterprise-grade Lakehouse data platforms. This role will drive architecture strategy, platform standardization, and enterprise data modernization initiatives, leveraging Databricks and cloud ecosystems. The ideal candidate brings deep expertise in Spark, Delta Lake, and cloud-native architecture, along with strong leadership in driving large-scale data transformations.
Key Responsibilities: Define and implement end-to-end Databricks Lakehouse architecture. Design scalable systems for:
  • Batch & real-time data processing
  • Structured & unstructured workloads
  • Establish medallion architecture (Bronze, Silver, Gold layers) as a standard.
  • Lead deployment and optimization of:
  • Azure Databricks / AWS Databricks / GCP Databricks
  • Define standards for:
  • Workspace design & cluster strategy
  • Job orchestration
  • Data storage (Delta Lake)
  • Drive adoption of:
  • Unity CatalogMLflowDatabricks SQL & PhotonBatch (ADF, Airflow)
  • Data modeling (star schema, data vault, dimensional models)
  • Guide engineering teams on best practices in Spark/PySpark optimization.
  • Performance & Cost Optimization
  • Query performance
  • Implement cost governance strategies (auto-scaling, job clusters, spot instances).Data Governance & Security
  • Fine-grained access control (RBAC/ABAC)
  • Ensure compliance with data privacy and regulatory standards.
  • Integrate Databricks with:
  • Data sources (ERP, CRM, APIs, IoT)BI tools (Power BI, Tableau)ML pipelines and AI platforms
  • Collaborate with cloud architects for:
  • Networking, security, and storage strategies.
  • Provide architectural guidance to data engineers, scientists, and TPMs.
  • Conduct design reviews and enforce architecture governance.
  • Mentor teams on emerging patterns:
  • Data MeshDataOps / MLOpsGenAI workloads on Databricks
Skills & Qualifications: Mandatory Skills 12+ years of experience in data engineering, architecture, or platform design.

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this