Databricks Architect

Location: United States (Remote)

Job Type: Full-time

Experience: 8+ years

Industry: Analytics Services

Roles and Responsibilities: Architecture & Design Design and implement modern data platforms using Databricks Lakehouse architecture. Define data architecture, data models, and governance frameworks for enterprise analytics platforms. Architect scalable batch and real-time data pipelines. Data Engineering Develop and optimize ETL/ELT pipelines using Apache Spark, Python, and SQL.Implement data ingestion frameworks using tools like Azure Data Factory, Apache Kafka, or Apache Airflow. Work with structured, semi-structured, and unstructured data. Platform & Performance Optimization Optimize Spark workloads, cluster configurations, and query performance. Implement data quality, monitoring, and cost optimization strategies. Ensure data platform scalability, reliability, and security. Advanced Analytics & ML Enablement Enable ML pipelines and collaboration with data science teams using MLflow. Build feature engineering pipelines and support AI/ML use cases. Required Technical and Professional Expertise: Strong experience with Databricks platform Expertise in Apache Spark, Spark SQL, and distributed computing Programming experience in Python, Scala, or SQLExperience with data lakes and lakehouse architectures Strong experience with cloud data platforms such as: Microsoft Azure Google Cloud Platform About Polestar As a data analytics and enterprise planning powerhouse, Polestar Solutions helps its customers bring out the most sophisticated insights from their data in a value-oriented manner. From analytics foundation to analytics innovation initiatives, we offer a comprehensive range of services that help businesses succeed with data. We have a geographic presence in the United States (Dallas, Manhattan, New York, Delaware), UK(London) & India (Delhi-NCR, Mumbai, Bangalore & Kolkata) and have 600+ people strong world-class team. We are growing at a rapid pace and plan to double our growth each year. This provides immense growth and learning opportunities to those who are choosing to work with Polestar. Our expertise and deep passion for what we do has brought us many accolades. The list includes: Recognized as “Great Place to Work” - 2024 Recognized as the Top 50 Companies for Data Scientists in 2023 by AIM.Financial Times awarded Polestar as High-Growth Companies across Asia-Pacific for a 5th time in a row in 2023.Featured on the Economic Times India's Growth Champions in FY2023.Polestar Analytics Selected as a 2022 Red Herring Global Top Data Science Providers in India 2022: Penetration and Maturity (PeMa) QuadrantFT ranking of 500 of the Asia-Pacific region’s high-growth companies. India’s most promising data science companies in 2022.Featured on Forrester's Now Tech: Customer Analytics Service Providers Report Q2, 2021.Recognized as Anaplan's India RSI Partner of the Year FY21.Elite Qlik Partner and a member of the ‘Qlik Partner Advisory Council’ & Microsoft Gold Partners for Data & Cloud Platforms Culture at Polestar. We have one of the most progressive people’s practices which are all aimed at enabling fast-paced growth for those who deserve it.