Research Scientist (Pyspark/Databricks)

Data Scientist Required Qualifications:Experience in Data Science, Machine Learning, Applied AI, or a related field Strong Python programming experience Hands-on experience with PySpark and distributed data processing Experience working within Databricks environments Advanced SQL skills Experience building, maintaining, or optimizing large-scale data pipelines Understanding of machine learning concepts, model development, and experimentation workflows Experience working with datasets ranging from millions of records Ability to navigate and enhance existing codebases Strong communication and mentoring abilities Job Description:Responsibilities:Enhance and maintain existing Python and PySpark-based data processing solutions Extract, transform, and prepare large datasets for machine learning and analytics initiatives Design and optimize data pipelines supporting research and experimentation efforts Work with structured and unstructured data, including text-heavy datasets and NLP use cases Collaborate with data scientists, researchers, and engineering teams to develop scalable solutions Troubleshoot data quality, pipeline, and performance issues across large-scale environments Conduct exploratory analysis and support model development activities Evaluate new approaches, tools, and technologies that improve research outcomes Provide technical mentorship, code reviews, and guidance to junior team members Translate business and research requirements into practical data solutions