Lead Data Engineer

Overview

• Lead the development of a scalable market data platform within a dynamic investment technology team.
• Collaborate with stakeholders to define architectural principles and implement robust data engineering solutions.
• Mentor and guide a team of engineers, fostering a culture of technical excellence.
• Design and optimize data models and database architectures for performance and scalability.
• Utilize AWS services to build cloud-native solutions tailored to financial data needs.
• Engage in hands-on development of ETL pipelines and data normalization workflows.
• Evaluate and select tools and technologies to enhance the data engineering ecosystem.
• Contribute to the strategic vision for data architecture and platform scalability.

Key Responsibilities & Duties

• Assess current data architecture and define target-state designs for scalability and resilience.
• Develop and implement ETL pipelines for ingesting and processing vendor data.
• Design workflows for data enrichment, symbology mapping, and distribution.
• Optimize database designs for analytical and operational use cases.
• Implement caching strategies to support low-latency data access.
• Collaborate with DevOps to manage cloud infrastructure using AWS services.
• Lead and mentor a team of engineers, promoting best practices and standards.
• Conduct code reviews and ensure adherence to architectural principles.

Job Requirements

• Bachelor of Science (BS) degree in a relevant field.
• 5-7 years of experience in data engineering or architecture roles.
• Proficiency in Python, SQL, and workflow orchestration tools like Airflow.
• Expertise in AWS services including S3, Lambda, and Redshift.
• Strong knowledge of financial data and market data vendors.
• Experience with database design, data modeling, and caching technologies.
• Ability to lead and mentor engineering teams effectively.
• Familiarity with real-time data streaming architectures is preferred.