Data Architect
ejamericaHouston (TX)
About the role
Job Description: Job Description Senior Data Engineer – AI, Automation & Databricks
Location: Houston, TX
Work Arrangement: Houston-based / Onsite as required
Experience: 8+ years preferred
Employment Type: Contract / Full-Time, based on project requirement
Job Summary:
We are looking for a highly experienced and innovative Senior Data Engineer to assess, modernize, and optimize enterprise Data Engineering processes while driving the adoption of automation, Artificial Intelligence, and Agentic AI.The ideal candidate will combine strong hands-on Data Engineering expertise with a proactive problem-solving mindset. This individual will evaluate existing engineering processes, identify opportunities for AI and automation, develop Proofs of Concept (POCs), and help transition successful ideas into production-ready solutions. The role requires deep experience with Databricks, AWS Redshift, modern Data Engineering architectures, AI-enabled engineering practices, and AI solutions across the Software Development Life Cycle (SDLC). The successful candidate should be comfortable working independently, challenging existing approaches, proposing innovative solutions, and collaborating with both technical and business stakeholders.
Key Responsibilities:
- Assess existing Data Engineering platforms, workflows, pipelines, architecture, and operational processes and identify opportunities for optimization.
- Design and implement improvements to enhance data pipeline performance, scalability, reliability, maintainability, and operational efficiency.
- Identify opportunities to introduce automation and AI capabilities across Data Engineering processes.
- Develop Proofs of Concept (POCs) for AI- and automation-driven ideas and evaluate their technical and business feasibility.
- Drive successful POCs through implementation and production adoption in collaboration with engineering, architecture, security, and business teams.
- Design and implement solutions leveraging Agentic AI, Generative AI, and AI-assisted Data Engineering.
- Identify opportunities to apply AI throughout the Software Development Life Cycle, including:
- Requirements analysis
- Solution design
- Code generation and optimization
- Data pipeline development
- Automated testing
- Data quality validation
- DocumentationMonitoring and observability
- Incident analysis and troubleshooting
- Build and optimize scalable Data Engineering solutions using Databricks.
- Design, develop, and optimize enterprise data workloads using AWS Redshift.
- Develop and optimize ETL/ELT pipelines supporting large-scale structured and unstructured data environments.
- Evaluate existing engineering practices and proactively recommend improvements rather than relying solely on predefined requirements.
- Identify repetitive or manual engineering activities that can be automated through scripts, workflows, AI agents, or orchestration platforms.
- Partner with Data Engineers, Data Architects, Data Scientists, AI/ML teams, application teams, and business stakeholders.
- Translate complex technical opportunities into clear business value and build stakeholder alignment around proposed solutions.
- Establish reusable engineering patterns, frameworks, accelerators, and best practices.
- Conduct technical evaluations of emerging AI and Data Engineering technologies and recommend appropriate adoption strategies.
- Support troubleshooting and root-cause analysis of complex Data Engineering and platform-related issues.
- Document technical designs, architecture decisions, POCs, implementation approaches, and operational procedures.
- Required Technical SkillsStrong hands-on experience in Data Engineering within large-scale enterprise environments.
- Advanced experience with Databricks.
- Strong experience with AWS Redshift.
- Strong understanding of:
- Data lakes and lakehouse architectures
- Data warehousesETL/ELT architecture
- Distributed data processing
- Data pipeline orchestration
- Data quality and governance
- Performance optimization
- Hands-on experience with Apache Spark / PySpark.
- Strong programming experience with Python.
- Strong SQL development and query optimization skills.
- Experience working with AWS Data and Analytics services.
- Experience designing and supporting scalable cloud-based data platforms.
- Understanding of CI/CD and DevOps principles within Data Engineering environments.
AI & Automation Experience:
Candidates should have practical experience applying AI beyond traditional analytics or machine learning use cases.
Preferred experience includes:
- Agentic AI / AI Agents
- Generative AI applications within Data EngineeringLLM-based engineering solutionsAI-assisted code generation and review
- Automated pipeline troubleshooting
- Intelligent Data Quality validationAI-assisted data transformation and mappingAI-driven monitoring and anomaly detection
- Automated documentation and metadata generationAI-enabled SDLC processes
- Prompt engineering and LLM integration
- Development of AI/automation POCsIntegration of AI solutions into production engineering workflows
Experience with AI development frameworks such as LangChain, LangGraph, Semantic Kernel, Amazon Bedrock, OpenAI/Azure OpenAI, or similar technologies would be advantageous.
Required Professional SkillsStrong analytical and problem-solving capabilities.
Demonstrated ability to independently identify technical and process improvement opportunities.
Strong innovation mindset with a track record of proposing and implementing new ideas.
Ability to operate effectively in ambiguous environments without requiring detailed direction.
Strong communication and stakeholder-management skills.
Ability to collaborate across technical teams, business stakeholders, leadership, and architecture groups.
Ability to explain complex technical concepts to non-technical stakeholders.
Strong ownership mentality with the ability to take ideas from concept → POC → validation → implementation.
Ability to work effectively as part of a collaborative engineering team.
Preferred Qualifications:
8+ years of overall Data Engineering, Data Platform, or related engineering experience.
Significant hands-on experience with enterprise Data Engineering architectures.
Experience leading Data Engineering modernization or optimization initiatives.
Demonstrated experience building AI- or automation-enabled engineering solutions.
Experience working within large, complex enterprise environments.
Experience influencing architecture or engineering standards across multiple teams.
Exposure to airline, transportation, travel, or other large-scale transactional environments would be advantageous.
Ideal Candidate Profile:
The ideal candidate is not simply someone who can build pipelines from predefined requirements. We are looking for an engineer who can evaluate the existing environment, identify what can be improved, and proactively develop solutions.
This person should be able to:
Understand the current Data Engineering landscape quickly.
Identify inefficiencies and improvement opportunities.
Determine where AI or automation can create measurable value.
Build working POCs rather than limiting recommendations to conceptual presentations.
Demonstrate the value of proposed solutions to technical and business stakeholders.
Convert successful concepts into scalable production implementations.
Challenge existing engineering practices constructively.
Work collaboratively and fit effectively within a multidisciplinary team.
Core Skills:
Databricks | AWS Redshift | Data Engineering | Python | PySpark | SQL | AWS | ETL/ELT | Data Lakehouse | Agentic AI | Generative AI | LLMs | AI Automation | AI across SDLC | POC Development | Data Platform Optimization | Stakeholder Management | Innovation
Company Description:
We provide Management Consulting, Information Technology Consulting, and Sourcing Advisory Services. Our management consulting services focus on our client's most critical issues and opportunities: strategy, marketing, organization, operations, technology, and mergers & acquisitions, across all industries and geographies. We bring deep, functional expertise, but are known for our holistic perspective: we capture value across boundaries and between the silos of any organization. We have proven a multiplier effect by optimizing the sum of the parts, not just the individual pieces. In today's dynamic business environment there is a strong need for the organizations to become globally competitive. Enterprise Resource Planning (ERP) has become an important software tool for managing business applications across various industries. Customized ERP solutions that matches the business will definitely maximize the return on investment ROI of Organization. Implementing right ERP helps companies to manage their data.
Company Description:
We provide Management Consulting, Information Technology Consulting, and Sourcing Advisory Services. Our management consulting services focus on our client's most critical issues and opportunities: strategy, marketing, organization, operations, technology, and mergers & acquisitions, across all industries and geographies. We bring deep, functional expertise, but are known for our holistic perspective: we capture value across boundaries and between the silos of any organization. We have proven a multiplier effect by optimizing the sum of the parts, not just the individual pieces. In today's dynamic business environment there is a strong need for the organizations to become globally competitive. Enterprise Resource Planning (ERP) has become an important software tool for managing business applications across various industries. Customized ERP solutions that matches the business will definitely maximize the return on investment ROI of Organization. Implementing right ERP helps companies to manage their data.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.