Data Engineer with Capital Markets Experience

Job Requirement – Data Engineer with Capital Markets Experience
Job ID: 438590
Location: New York, NY
Experience: 8–12 Years
Skill Area: Data Concepts & Data Modelling

Mandatory Skills & Requirements
- Strong Capital Markets and post-trade data experience.
- Understanding of post-trade data flows including allocations, confirmations, settlement, risk, finance, and regulatory reporting.
- Knowledge of securities data modeling and financial data structures.
- Experience with reference data, security master data, and transactional trade data.
- Strong SQL development skills with experience designing and optimizing relational data models and database schemas.
- Strong Python experience, including Pandas and PySpark.
- Understanding of Data Warehouse, Data Lake, and Data Lakehouse architectures.
- Experience working with large-scale transactional and analytical data platforms.
- Experience with PostgreSQL, SQL Server, Snowflake, Redshift, Databricks, or similar technologies, preferably on AWS.
- Experience with batch and streaming data processing using Kafka, Spark, Flink, or similar platforms.
- Experience building ETL/ELT data pipelines using Python and SQL.
- Strong understanding of data quality, lineage, validation, and reconciliation.
- Experience with messaging technologies such as Kafka, RabbitMQ, IBM MQ, or similar.
- Understanding of data security, privacy, and regulatory compliance requirements.

Key Responsibilities
- Develop and maintain scalable batch and streaming data pipelines.
- Design and optimize relational data models and database schemas.
- Build ETL/ELT workflows using Python and SQL.
- Work with post-trade data across allocations, confirmations, settlement, risk, finance, and regulatory reporting.
- Develop data solutions for transactional and analytical workloads.
- Implement data quality, validation, lineage, and reconciliation processes.
- Work with large-scale data platforms, data warehouses, data lakes, and lakehouse environments.
- Support streaming and event-driven data processing using Kafka, Spark, Flink, or similar technologies.
- Collaborate with business and technology stakeholders to understand Capital Markets data requirements.
- Ensure data solutions meet security, privacy, and regulatory requirements.

Preferred Skills & Qualifications
- Time-series data management and financial market data.
- AWS cloud data platforms.
- Databricks, Snowflake, Redshift, or similar cloud data technologies.
- Datadog, Prometheus, Grafana, or similar monitoring/observability tools.
- Exposure to Data Science, Machine Learning, and AI/GenAI data preparation concepts.