Data Engineer

Our Partner offers future-proof quant technologies to global investors. Cloud-hosted and Python-based, the platform integrates a next-gen backtest engine and analytics with curated datasets covering equities, rates, FX, commodities and volatility. Our Partner eliminates the expensive upfront costs of infrastructure build-out, giving clients an edge in alpha generation from day one. 

Founded in 2019 we are a rapidly growing IT and services firm with offices in the UK and the USA. Investors with combined AUM $5 trillions are using Our Partner. We're launching our operations in Bangalore, and we're looking for people who are highly talented, competent, and committed to our clients' needs.

As a Data Engineer, you will be responsible for building, managing and optimizing data pipelines and then moving these data pipelines effectively into production. You will be working closely with the Quant developers to make the data easy to use within the platform.

Role and Responsibilities

• Maintain, support and expand existing data pipelines
• Implement standardised data ingress and egress pipelines..
• Onboard new, disparate data sets, sourced from many and varied data vendors, covering all asset types and frequencies from daily to real-time.
• Liaise and work closely with our team of quant developers to make the data easy to use within the platform.
• Create thorough operationalised documentation and support for all new pieces of work and retrospectively existing pieces of work.
• Take part in a full test of all new pieces of work that you develop, including Unit Test Scripts, hand-off.
• Contribute to our cloud-based analytics platform.

Key Requirements for this Role

• A minimum of 2+ years experience in data management disciplines including data integration, modeling, optimization and data quality.
• Degree in Computer Science, IT or similar field (Statistics, Mathematics); a Master is a plus
• Skilled Data Engineer with strong proficiency in Python
• Working experience with Python libraries like Pandas and NumPy.
• Able to effectively use of AWS managed services (Step function, EMR, Lambda, Kinesis, and Athena) and design/develop data engineering solutions
• Experience working with Kafka for building real-time data streaming pipelines is 
• Experience in Data warehousing tools like Snowflake/Pyspark/Databricks
• Strong working experience in popular database programming languages like SQL.
• Good experience of Lambda and serverless compute
• Good experience of AWS Step Functions with exposure to API Gateway
• Experience building large-scale data pipelines using S3 and different storage types.
• Experience with non-relational database such as DynamoDB is a plus
• Financial services experience specifically capital markets or funds industry is preferred
• Strong communication skills and ability to present results to management teams to allow them to interpret the results and make effective decisions
• Capability to solve complex technical issues, comprehend risks prior to the circumstance.