Overview:
As a Senior Data Engineer, you help design and scale AI-ready data platforms that empower real-time, intelligent payment capabilities. You will join a cross-functional team to transform high-volume transaction data into reliable, secure products and experiences. You’ll shape architectures that support ML, Generative AI, and AI-driven insights at scale. This role offers impact through building resilient systems that underlie Amex’s global payment network and AI initiatives.
Responsibilities:
Design and build scalable, low-latency data platforms for mission-critical payment workloads Develop real-time streaming pipelines capable of handling high-volume data Create AI-ready data architectures enabling ML, Generative AI, and real-time intelligence use cases Experiment with AI capabilities (LLMs, RAG, embeddings, vector search, AI agents) where appropriate Design microservices and event-driven architectures using modern engineering practices Advance cloud-native tech adoption (containerization, CI/CD, automation, observability) Establish engineering practices for performance, security, data quality, and reliability Apply Responsible AI and data governance principles including privacy and explainability Evaluate emerging AI and data technologies and translate to practical payment solutions Provide technical leadership, mentor engineers, and contribute to engineering standards and reviews Partner with product, architecture, data science, and engineering teams to translate opportunities into scalable solutions
Key requirements:
Experience designing and operating large-scale distributed data platforms Strong experience with NoSQL technologies (Cassandra, Elasticsearch, Couchbase, Redis) Experience with large-scale distributed data processing (Apache Spark) Proficiency in Python, Java, Scala, or similar languages Experience designing data pipelines, data models, APIs, and event-driven architectures Solid understanding of distributed systems including scalability, security, and performance Proven ability to deliver production-grade solutions Strong communication and collaboration Technical leadership and mentorship Problem-solving and strategic thinking Apache Spark NoSQL (Cassandra, Elasticsearch, Couchbase, Redis) Python, Java, or Scala