Senior Data Orchestration Engineer
horizontal talentMinneapolis (MN)
Senior Data Orchestration Engineer
Posted 4 days ago
horizontal talentMinneapolis (MN)
SENIORITY
Lead
About the role
Senior Data Orchestration Engineer (Apache Airflow)
Role Overview:
We are seeking an experienced Data Orchestration Engineer to lead the migration from AutoSys to Apache Airflow and establish a modern, scalable enterprise orchestration platform. This role will focus on migrating legacy workloads, automating data pipelines, improving reliability, and standardizing orchestration across our enterprise technology ecosystem.
Key Responsibilities:
Lead the migration of enterprise workloads from AutoSys to Apache Airflow, including job discovery, dependency mapping, conversion, testing, and cutover.
Design, develop, and maintain Apache Airflow DAGs for data pipelines and enterprise workflows.
Assess existing AutoSys scheduling patterns and identify opportunities to consolidate, simplify, or eliminate legacy jobs.
Orchestrate workloads across Databricks, Microsoft Fabric, cloud services, APIs, and enterprise applications.
Establish standards for scheduling, dependencies, retries, error handling, monitoring, and operational recovery.
Implement CI/CD, version control, and infrastructure-as-code practices for orchestration.
Partner with data engineering, application development, and platform teams to ensure migration continuity and minimize operational disruption.
Required Qualifications:
5+ years of experience in data engineering, platform engineering, or enterprise workload orchestration.
Hands-on experience with AutoSys and Apache Airflow, including migration of legacy enterprise scheduling workloads to modern orchestration platforms.
Strong Apache Airflow expertise, including DAG development, operators, sensors, scheduling, and production operations.
Proficiency in Python, SQL, and REST APIs.
Experience with cloud platforms (Azure, AWS preferred), Databricks, and modern data architectures.
Understanding of CI/CD, containerization, observability, and production support.
Success Measures
Successful migration and retirement of AutoSys workloads with minimal business disruption.
Reduction in legacy job complexity and manual intervention.
Improved reliability, observability, and recovery of enterprise workflows.
Standardized, reusable orchestration patterns that accelerate future development.
Senior Data Orchestration Engineer (Apache Airflow)
Role Overview:
We are seeking an experienced Data Orchestration Engineer to lead the migration from AutoSys to Apache Airflow and establish a modern, scalable enterprise orchestration platform. This role will focus on migrating legacy workloads, automating data pipelines, improving reliability, and standardizing orchestration across our enterprise technology ecosystem.
Key Responsibilities:
Lead the migration of enterprise workloads from AutoSys to Apache Airflow, including job discovery, dependency mapping, conversion, testing, and cutover.
Design, develop, and maintain Apache Airflow DAGs for data pipelines and enterprise workflows.
Assess existing AutoSys scheduling patterns and identify opportunities to consolidate, simplify, or eliminate legacy jobs.
Orchestrate workloads across Databricks, Microsoft Fabric, cloud services, APIs, and enterprise applications.
Establish standards for scheduling, dependencies, retries, error handling, monitoring, and operational recovery.
Implement CI/CD, version control, and infrastructure-as-code practices for orchestration.
Partner with data engineering, application development, and platform teams to ensure migration continuity and minimize operational disruption.
Required Qualifications:
5+ years of experience in data engineering, platform engineering, or enterprise workload orchestration.
Hands-on experience with AutoSys and Apache Airflow, including migration of legacy enterprise scheduling workloads to modern orchestration platforms.
Strong Apache Airflow expertise, including DAG development, operators, sensors, scheduling, and production operations.
Proficiency in Python, SQL, and REST APIs.
Experience with cloud platforms (Azure, AWS preferred), Databricks, and modern data architectures.
Understanding of CI/CD, containerization, observability, and production support.
Success Measures
Successful migration and retirement of AutoSys workloads with minimal business disruption.
Reduction in legacy job complexity and manual intervention.
Improved reliability, observability, and recovery of enterprise workflows.
Standardized, reusable orchestration patterns that accelerate future development.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.