Skip to main content

Sr Data Engineer II

Posted yesterday

american expressPalo Alto (CA)

SENIORITY

Lead

Apply (opens in a new tab)

About the role

Overview: As a Senior Data Engineer, you help design and scale AI-ready data platforms that empower real-time, intelligent payment capabilities. You will join a cross-functional team to transform high-volume transaction data into reliable, secure products and experiences. You’ll shape architectures that support ML, Generative AI, and AI-driven insights at scale. This role offers impact through building resilient systems that underlie Amex’s global payment network and AI initiatives.
Responsibilities: Design and build scalable, low-latency data platforms for mission-critical payment workloads Develop real-time streaming pipelines capable of handling high-volume data Create AI-ready data architectures enabling ML, Generative AI, and real-time intelligence use cases Experiment with AI capabilities (LLMs, RAG, embeddings, vector search, AI agents) where appropriate Design microservices and event-driven architectures using modern engineering practices Advance cloud-native tech adoption (containerization, CI/CD, automation, observability) Establish engineering practices for performance, security, data quality, and reliability Apply Responsible AI and data governance principles including privacy and explainability Evaluate emerging AI and data technologies and translate to practical payment solutions Provide technical leadership, mentor engineers, and contribute to engineering standards and reviews Partner with product, architecture, data science, and engineering teams to translate opportunities into scalable solutions
Key requirements: Experience designing and operating large-scale distributed data platforms Strong experience with NoSQL technologies (Cassandra, Elasticsearch, Couchbase, Redis) Experience with large-scale distributed data processing (Apache Spark) Proficiency in Python, Java, Scala, or similar languages Experience designing data pipelines, data models, APIs, and event-driven architectures Solid understanding of distributed systems including scalability, security, and performance Proven ability to deliver production-grade solutions Strong communication and collaboration Technical leadership and mentorship Problem-solving and strategic thinking Apache Spark NoSQL (Cassandra, Elasticsearch, Couchbase, Redis) Python, Java, or Scala

Before you apply

Applying takes about a minute. These four things decide how fast it moves after that.

Your profile is current

It's what we read first. Occupations, seniority and locations matter more than a long history.

Two examples you can talk through

Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.

A number in mind

What you're on now and what would make you move. We negotiate better when we know both.

Your notice period

Employers plan around it, and it's the question that stalls offers most often.

Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.

More like this