Sr. Data Engineer with Gen AI

Job Title: Sr. Data Engineer with Gen AI

Location: Remote

Job Description:-

As a Data Engineer, you’ll provide specialized Data Engineering support to build, optimize, and maintain cloud‑based data pipelines, platforms, and data stores that enable analytics across large, diverse datasets. This role includes designing scalable ETL/ELT workflows, structuring data storage architectures, and ensuring reliable data ingestion from multiple structured and unstructured sources. You’ll support mission‑driven analytics efforts by transforming raw data into analysis‑ready formats, performing backend SQL/NoSQL testing, and validating data quality throughout the pipeline lifecycle. In this position, you will collaborate with analysts, product teams, and developers to understand evolving data requirements, define technical specifications, and implement end‑to‑end solutions supporting reporting, modeling, and visualization needs.

Key Responsibilities:

·         Assist in designing, developing, and maintaining scalable ETL/ELT pipelines to ingest, transform, and deliver structured and unstructured data from multiple cloud‑based sources.

·         Assist in building and optimizing data storage solutions (data warehouses, data lakes, or lakehouses) that support large‑scale analytics, reporting, and business intelligence needs.

·         Assist in performing backend SQL and NoSQL testing, including data validation, post‑deployment checks, and troubleshooting of ingestion or schema issues.

·         Collaborate cross‑functionally with analysts, engineers, and stakeholders to translate requirements into technical data models, workflows, and pipeline specifications.

·         Assist in developing scripts and automation to convert diverse data types into usable formats, while scaling, monitoring, and operating cloud‑based data platforms.

Must Have:   

·         12+years of IT experience required

·         2+ years of experience in Data Warehouse development

·         Experience with data analysis of aggregated data from multiple sources in cloud-based environments

·         Experience developing and maintaining scalable data stores that supply big data in forms needed for business analysis 

·         Experience with SQL and Python for retrieving, parsing, and processing structured and unstructured data 

·         Experience developing scalable ETL or ELT workflows for reporting and analytics using cloud-based technologies

·         Experience using Cloudera or Hadoop technologies to ingest, transform, and process data

·         Experience leveraging generative AI coding tools to assist in development

·         Ability to perform backend SQL and NoSQL database testing, including post-deployment and data validation testing

·         Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements

·         Bachelor's degree 

Nice If You Have:   

·         Experience creating solutions within a collaborative and cross-functional team environment 

·         Experience with AWS Glue, AWS Athena, and Postgres

·         Experience with distributed data and computing tools such as Spark, Databricks, Hive, AWS EMR, or Kafka 

·         Experience working on real-time data and streaming applications  

·         Experience with NoSQL implementation using MongoDB or Cassandra 

·         Experience with data warehousing, including AWS Redshift, MySQL, or Snowflake 

·         Experience with UNIX and Linux, including basic commands and Shell scripting 

·         Experience with Agile engineering practices  

Educational Qualifications:

• Required - Bachelor’s degree in Computer Science, Information Technology, Computer Engineering or closely related or equivalent.
• Preferred - Master’s degree in Management Information Systems (MIS), Computer Science, Big Data or Analytics or equivalent.

Travel:

·   Open to travel based up on the nature of the engagement.

Thanks & Regards