Data Automation Engineer
robert halfWashington (DC)
Data Automation Engineer
Posted today
robert halfWashington (DC)
SENIORITY
Senior
About the role
Overview:
In this role you design and implement scalable data automation workflows that integrate AWS and Azure platforms. You will build and maintain ETL/ELT pipelines across diverse data sources, and develop batch and near real-time data processing with Spark and streaming technologies. You will prototype Generative AI solutions, enforce secure data practices, and implement CI/CD and automation using modern DevOps tools. You engage with cross-functional teams to deliver end-to-end data solutions that enable analytics and AI initiatives at scale.
Responsibilities:
Design and implement scalable data automation workflows using AWS services with integration to Azure platforms Develop and maintain ETL/ELT processes to ingest, transform, and move data across DynamoDB, SQL Server, Azure SQL, and other enterprise data sources Build and support batch and near real-time data pipelines using Apache Spark and streaming technologies such as Kafka or Flume Develop scalable data processing solutions utilizing Amazon EMR, AWS Fargate, Kubernetes, and related cloud technologies Integrate REST APIs, external services, and Amazon Connect customer interaction data into analytics platforms Evaluate and prototype Generative AI solutions using Amazon Bedrock, Azure OpenAI, Hugging Face, LangChain, and related technologies Implement source control, CI/CD, and automation practices using GitHub, Azure DevOps, Jenkins, Docker, and Agile methodologies Ensure data solutions meet security and compliance requirements, including IAM, encryption, RBAC, VPC isolation, and firewall policies
Key requirements: AWS services Azure platforms ETL/ELT DynamoDB SQL Server Azure SQL
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.