Data Engineer
pashtek salesforce partner data aiDenver (CO)
About the role
Location: Remote (United States)
Employment Type: contract
About the Role:
As a Data Engineer, you’ll build and operate the pipelines, tables, and services that power our hybrid data platform across on-premises and AWS. You’ll implement lakehouse patterns, productionize batch and streaming workloads, and partner with security, platform, analytics, and application teams to deliver governed, high-performance data products at scale.
What You’ll Do:
- Build data pipelines: Develop reliable ELT/ETL jobs in Spark/SQL/dbt/Airflow/Glue to ingest from on-prem and cloud sources into S3-backed lakes and warehouses.
- Implement lakehouse tables: Create and maintain Iceberg (or Delta/Hudi) tables using the appropriate catalog (AWS Glue, Hive Metastore, Polaris/REST) with ACID, time travel, and schema evolution.
- Operate compute engines: Run and tune Spark (Databricks/EMR), Trino/Starburst, Dremio, and Snowflake workloads; leverage pushdown and query acceleration where applicable.
- Model data for analytics: Deliver dimensional models, semantic layers, and domain-oriented data products; document contracts and SLAs with consumers.
- Governance & security: Apply data cataloging, lineage, PII classification, Lake Formation permissions, IAM roles, and row/column-level security; contribute to audit readiness.
- Performance & cost tuning: Optimize partitioning, clustering/Z-order, predicate pushdown, file sizing/compaction, caching, and workload isolation; monitor and right-size clusters.
- Migrations: Execute migration workstreams from legacy/on-prem EDW (Informatica/SSIS/SAP BW, SQL Server/Oracle, Hadoop) to lakehouse patterns (Spark/dbt/ELT), including dual-run cutovers and reconciliation.
- Streaming & CDC: Build real-time and near-real-time pipelines using Kafka/MSK/Kinesis and Spark Structured Streaming; implement CDC with Debezium, DMS, or Fivetran.
- Quality & observability: Add unit/integration tests, expectations/rules, data contracts, lineage, alerting, and SLO dashboards; participate in on-call rotations.
- Platform guardrails: Contribute to standards for naming, zones, schemas, S3 layout, encryption, backup/DR, and multi-region replication; write clear runbooks and docs.
- DevOps for data: Use Terraform/Cloud
- Formation and CI/CD (GitHub Actions/GitLab/Azure DevOps) to version, test, and deploy data assets.
Required Experience:
- 5+ years in data engineering (or equivalent), delivering production pipelines and tables on at least two large-scale platforms.
- Hands-on with AWS data services: S3, Glue/EMR, Lake Formation, IAM, and at least one warehouse (Snowflake or Redshift).Deep experience with Apache Spark and at least one of: Databricks, EMR Spark, Snowflake Snowpark, Dremio, or Starburst/Trino.
- Production experience with open table formats: Apache Iceberg (preferred), Delta Lake, or Apache Hudi; strong grasp of metadata/manifest, compaction, and schema evolution.
- Comfortable with on-prem stacks: Hadoop/Hive, Spark on Kubernetes, SQL Server/SSIS and/or Oracle/Exadata; Netezza/Teradata a plus.
- Proven data modeling (3NF, dimensional, Data Vault), ELT/ETL design, and SQL performance tuning.
- Security/governance: RBAC/ABAC, row/column-level security, masking/tokenization, KMS/key management.
- IaC & CI/CD for data workloads.
- Excellent communicator who can collaborate with platform engineers, analysts, and stakeholders to meet SLAs and roadmap goals.
- Nice to Havedbt for ELT, Airflow orchestration, Great Expectations/Deequ for quality, Open
- Lineage/Marquez for lineage.
- Streaming experience with Kafka/MSK, Kinesis, or Flink.
- Catalogs/semantic: AWS Glue Data Catalog, Unity Catalog, Amundsen/DataHub, Atlan/Collibra.
- BI/serving: DuckDB, Athena, QuickSight, Tableau/Power BI/Looker.
- Compliance: SOC 2, HIPAA/PHI, GDPR, PCI; SSO/OIDC with Okta.
- Multi-tenant platforms or federated governance exposure.
- Location & Work StyleRemote with core hours in PST/CST;
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
More like this