Big Data / AWS Data Engineer

Job Summary We are seeking a Big Data / AWS Data Engineer to design, develop, and support enterprise-scale data platforms that enable analytics, customer data management, and data governance initiatives. This role is responsible for building scalable data pipelines, developing ETL/ELT solutions, optimizing cloud data platforms, ensuring data privacy compliance, and providing technical leadership across enterprise data initiatives. The ideal candidate will have extensive experience with AWS data services, Spark, Redshift, distributed data processing, and enterprise data architecture. Key Responsibilities Design, develop, and maintain scalable data pipelines using Spark (Scala/Python), AWS Glue, AWS Lambda, and Shell scripting. Build, optimize, and support ETL/ELT workflows across AWS data platforms, including Amazon S3, Hive, Apache Iceberg, and Amazon Redshift. Develop and maintain Redshift SQL/PLSQL code, stored procedures, and enterprise data processing frameworks. Optimize database performance through query tuning, workload optimization, and efficient data-sharing implementations. Design and support ingestion frameworks for structured and semi-structured data formats, including JSON, CSV, XML, ORC, Parquet, and Avro. Design, implement, and support enterprise Customer 360 solutions that provide a unified view of customer data across multiple systems. Integrate data from multiple enterprise applications and business systems to support reporting, analytics, and customer engagement initiatives. Collaborate with business stakeholders and technical teams to define scalable data architecture and engineering solutions. Support Data Subject Requests (DSRs), including customer data deletion and privacy compliance processes. Partner with Data Protection Office (DPO) teams to ensure compliance with privacy regulations, governance policies, and enterprise data standards. Maintain and support enterprise privacy compliance solutions, including the MicroStrategy Privacy Compliance module. Contribute to enterprise data governance initiatives that improve data quality, consistency, metadata management, and compliance. Analyze and troubleshoot complex data quality, reporting, and platform issues using Redshift SQL, MySQL, Python/PySpark, Shell scripting, and Excel. Investigate data discrepancies, reporting anomalies, and customer data issues while performing root cause analysis and implementing corrective actions. Support analytics related to customer segmentation, loyalty programs, and customer engagement initiatives. Provide production support for enterprise data platforms and resolve critical production issues. Participate in enterprise initiatives related to Customer 360, Privacy, Data Governance, and data modernization programs. Conduct code reviews, design reviews, and provide technical guidance to engineering teams. Research, design, and implement next-generation data engineering capabilities and cloud-native data solutions. Serve as a subject matter expert for enterprise customer data platforms and cloud data engineering technologies. Required Qualifications Bachelor's degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related field, or equivalent professional experience. 12+ years of experience in Data Warehouse, Big Data, or Enterprise Data Engineering environments. 5+ years of experience in a Lead Data Engineer, Technical Lead, or Subject Matter Expert (SME) role. Strong experience designing and developing enterprise data pipelines using Spark (Scala/Python). Extensive experience with AWS services, including Amazon S3, EC2, Redshift, AWS Glue, Lambda, Kafka, Airflow, Hive, and Apache Iceberg. Strong experience developing ETL/ELT frameworks and enterprise data integration solutions. Experience developing and optimizing Redshift SQL/PLSQL, MySQL, stored procedures, and database performance. Strong understanding of data architecture, dimensional modeling, distributed data processing, and cloud-native data platforms. Experience designing and supporting enterprise Customer 360 or customer data platforms. Knowledge of data governance, privacy compliance, and enterprise data security principles. Experience working with structured and semi-structured data formats, including JSON, CSV, XML, ORC, Parquet, and Avro. Strong analytical, troubleshooting, and root cause analysis skills. Excellent verbal and written communication skills with the ability to collaborate across technical and business teams. Experience managing enterprise-scale data platforms and delivering complex data engineering initiatives. Preferred Qualifications Experience in large enterprise or global delivery environments. Experience supporting cloud modernization and enterprise data transformation initiatives. AWS, Azure, Google Cloud Platform, or Data Engineering certifications. Experience working in regulated industries such as Hospitality, Finance, Healthcare, Telecommunications, or similar sectors. Experience leading enterprise data governance and customer data management initiatives. Strong leadership, stakeholder management, and mentoring experience. Experience working in Agile software development and data engineering environments. Education: Bachelors Degree Certification: AWS , Azure , Google Cloud Platform , Data Engineering