Skip to main content
David Careers

Databricks Integration Lead

Posted 9 days ago

new york technology partnersIndianapolis (IN)
Data Warehousing SpecialistsComputer Systems Design Services

SENIORITY

Manager

Apply (opens in a new tab)

About the role

Role Overview: We need a senior, hands-on Databricks Integration Lead to own the integration architecture for a large migration from Veeva CRM to Vault CRM. The scope covers 5 Veeva CRM orgs and 46 unique integration points that must beanalyzed, rationalized, and optimized before connecting to the new Vault CRM platform. The lead sets technicaldirection, makes architecture decisions, and guides a skilled Databricks engineering team (including offshoreresources). Deep, current, hands-on Databricks expertise is the top screening criterion. This is not anoversight-only role.
Key Responsibilities:
  • Own end-to-end integration architecture across Databricks, 5 Veeva CRM orgs, Vault CRM, and connected systems
  • Catalog and analyze all 46 integration points; consolidate, retire, or standardize rather than lift-and-shift
  • Define standards: medallion architecture, Delta Lake, Unity Catalog governance, workflow orchestration, CI/CDDesign coexistence patterns while Veeva CRM and Vault CRM run in parallel
  • Define Veeva-to-Vault data mapping (accounts, HCP/HCO, calls, samples, consents, territories)
  • Review designs, notebooks, and pipelines; act as technical escalation point for the team
  • Establish validation/reconciliation frameworks and support GxP, privacy, and audit requirements
  • Own technical cutover plan, rollback strategy, and monitoring/operational support model
Required Qualifications:
  • 10+ years in data integration/architecture; 5+ years deep, hands-on Databricks in enterprise production
  • Expert in PySpark/Spark SQL, Delta Lake, Workflows/Jobs, Delta Live Tables, Auto Loader, Unity Catalog, performance and cost tuning
  • Delivered large multi-source integration programs (dozens of interfaces)
  • Hands-on Veeva CRM data model and integration experience; Vault CRM knowledge strongly preferred
  • Life sciences / pharma commercial experience (HCP/HCO data, call activity, samples, consent, MDM)
  • Strong REST/SOAP API, bulk load, and file/event-based integration skills
  • Proven technical leadership of engineering teams, including distributed/offshore teams
  • Familiarity with GxP validation and regulated-environment delivery
Preferred Qualifications:
  • Databricks certifications (Data Engineer Professional, Solution Architect, or equivalent)
  • Prior Veeva CRM to Vault CRM or other CRM-to-CRM migration experience
  • Experience with Veeva Network, OpenData, or multi-org integration rationalization
  • DevOps for Databricks (Terraform, Azure DevOps, GitHub Actions); AWS, Azure, or GCP

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this