Skip to main content

Job Description - AWS Data Engineer (Python, AWS Glue, PySpark & Attunity)

Posted today

purple driveMalvern (PA)

SENIORITY

Senior

Apply (opens in a new tab)

About the role

Purple DriveAWS Data Engineer – Python, AWS Glue, PySpark & Attunity We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS Glue, PySpark, Attunity (Qlik Replicate), and AWS Cloud to design, develop, and maintain scalable enterprise data integration solutions. The ideal candidate will have hands-on experience building ETL pipelines, developing cloud-native data lakes, implementing real-time and batch data replication, and automating infrastructure using AWS services. The role requires collaboration with business, analytics, and application teams to deliver secure, scalable, and high-performing data engineering solutions while following DevOps and CI/CD best practices.5–8+ Years of experience in Data Engineering 3+ Years of experience with AWS Data Services Experience with enterprise ETL/ELT and cloud migration projects Data Engineering & ETL Development Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, PySpark, and Python. Build and optimize data transformation workflows for processing large-scale datasets. Develop reusable data ingestion and transformation frameworks. Implement batch and real-time data processing pipelines. Data Integration Use Attunity (Qlik Replicate) for real-time and batch data replication. Integrate data from on-premises systems to AWS Cloud. Support heterogeneous source systems and cloud-based data platforms. AWS Cloud & Data LakeDesign and manage enterprise data lakes using Amazon S3.Ensure data quality, security, governance, and lifecycle management. Implement serverless data processing using AWS Lambda. Integrate Glue, Lambda, S3, and other AWS services for end-to-end workflows. Infrastructure & Automation Configure and manage AWS IAM roles, policies, and permissions. Automate infrastructure provisioning using AWS Cloud Formation. Support Infrastructure-as-Code (IaC) initiatives. Participate in CI/CD pipeline implementation and deployment automation. Performance & Support Monitor and optimize ETL jobs, Glue workflows, and cloud resources. Troubleshoot production issues and perform root cause analysis. Improve pipeline performance, reliability, and scalability. Maintain technical documentation and operational runbooks. Collaboration Work with Business Analysts, Data Architects, Analytics teams, and Application Developers. Gather business requirements and translate them into technical solutions. Participate in Agile ceremonies, code reviews, and design discussions. Mandatory Technical SkillsAWS CloudAmazon S3AWS GlueAWS LambdaAWS IAMAWS Cloud FormationData Engineering PythonPySparkETL / ELTData Pipelines Data Integration Data Transformation Data Replication Attunity (Qlik Replicate) Real-time Replication Batch Processing DatabasesSQLRelational Databases Data Warehousing (preferred) DevOpsGitCI/CDInfrastructure as Code (IaC) Cloud FormationMonitoring & Operations Performance Tuning TroubleshootingRoot Cause Analysis Technical Documentation Preferred SkillsAWS Step Functions Amazon Redshift Amazon AthenaAWS Lake Formation Apache Spark AirflowTerraformDockerKubernetesData Quality Frameworks Soft Skills Excellent communication skills Stakeholder management Strong analytical and problem-solving skills Leadership and ownership mindset Team collaboration Mentoring and coaching Planning and execution Risk management Customer-focused approach

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this