Data Engineer
iris softwareMorris Twp (NJ)
About the role
Greetings! We are seeking a senior Python / PySpark Data Engineer to join a high-impact data engineering initiative for a leading global financial institution in Whippany, NJ. In this role, you will design, engineer, and optimize distributed data pipelines and analytics engines supporting risk calculations, regulatory compliance reporting, transaction processing, and enterprise data lake modernization.
Job Title: Python / PySpark Data Engineer
Domain: Banking & Financial Services
Location: Whippany, NJ (Hybrid: 3 Days Onsite, 2 Days Remote)
Contract Duration: 12+ Months (with potential extension)
Work Authorization: Open to all authorized candidates (W2 only)
Interview Process: In-Person Interview onsite at the Whippany, NJ campus.
Key Responsibilities:
Design, develop, and maintain high-throughput batch and real-time distributed data pipelines using Python and PySpark.
Ingest, transform, and clean multi-terabyte financial datasets across structured, semi-structured, and unstructured sources.
Optimize PySpark jobs by analyzing DAG execution plans, tuning memory management, addressing data skew, and refining partition strategies.
Implement robust data quality checks, reconciliation routines, and audit logging to ensure strict adherence to banking data governance standards.
Collaborate with upstream transaction systems and downstream consumption layers (BI dashboards, risk analytics models, and regulatory platforms).
Build automated CI/CD deployment pipelines, automated testing frameworks, and monitoring alerts for production data jobs.
Partner with engineering leads in Agile standups, code reviews, and architectural design sessions during onsite collaboration days.
Required Technical Skills & Qualifications:
- Professional Experience: 6 to 9+ years of overall software and data engineering experience, with a solid footprint in Banking, Capital Markets, or Financial Services.
- Python & PySpark: Advanced proficiency in Python (OOP, collections, multiprocessing) and hands-on expertise building scalable pipelines with PySpark (DataFrames, Datasets, Spark SQL).
- Performance Tuning: Proven ability to debug and optimize Spark applications (broadcast joins, caching, shuffle partition tuning, Catalyst optimizer analysis).
- Database & Querying: Strong SQL writing skills with experience working with relational databases (PostgreSQL, Oracle) as well as modern analytical data warehouses (Snowflake, Databricks Delta Lake, or Redshift).
- Cloud & Orchestration: Practical familiarity running workloads in public cloud environments (AWS/Azure) utilizing services like S3, EMR/Glue, and workflow orchestrators (Apache Airflow, Step Functions).
- Financial Domain Knowledge: Understanding of banking data domains such as Trade Lifecycle, PnL, Market/Credit Risk, Fixed Income, or Regulatory Reporting (e.g., CCAR, Basel).
- Best Regards,
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.