Start searching
Back to Jobs
Recently postedEngineering

Senior AWS Data & Solution Engineer

Acunor InfotechNew York, NY🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeOn Site
LocationNew York, NY, United States
Generative AIAgileData Analysis

Job description

Location: New York City, NY(Hybrid)

Experience: 10+ Years

Certification Requirement: AWS Solutions Architect, AWS Data Engineer, or AWS Data Analytics Specialty Certification

Job Summary

Acunor is seeking a Senior AWS Data & Solution Engineer with strong hands-on expertise in Python, PySpark, and AWS data engineering. The ideal candidate will design and build enterprise data lakes, develop batch and event-driven data pipelines, implement secure REST APIs, and support production applications.

This role combines hands-on development, solution design, production support, and technical leadership.

Key Responsibilities

  • Design, develop, and unit test Python and PySpark applications using Apache Spark.

  • Build data pipelines integrating Oracle, MongoDB, DynamoDB, Amazon S3, and HDFS.

  • Develop PySpark ETL pipelines using AWS Glue and Amazon EMR.

  • Build Python-based AWS Lambda functions using boto3, awswrangler, pandas, JSON, and requests.

  • Develop event-driven integrations using Apache Kafka, Amazon MSK, and Confluent libraries.

  • Implement Apache Iceberg tables with schema evolution, CDC merges, and current-state snapshots.

  • Develop REST APIs using FastAPI or Flask with Swagger/OpenAPI specifications.

  • Secure API endpoints using OAuth2 and JWT authentication.

  • Orchestrate workflows using AWS Step Functions and Amazon EventBridge.

  • Configure AWS Glue crawlers and Data Catalog integrations.

  • Optimize Spark applications, SQL queries, partitioning, compression, and file formats.

  • Deploy applications using Docker, Kubernetes, and Amazon EKS.

  • Build agentic AI workflows and use Copilot/GPT for development.

  • Monitor and troubleshoot applications using Amazon CloudWatch and Datadog.

  • Participate in code reviews, production releases, deployments, and incident resolution.

  • Create technical documentation, deployment guides, and operational runbooks.

  • Coordinate technical designs across stakeholders and onsite/offshore teams.

  • Mentor engineers and contribute to Agile delivery and engineering standards.

Required Qualifications

  • Bachelor's degree or equivalent in Computer Science or a related discipline.

  • 10+ years of relevant professional experience.

  • Valid AWS certification in Solutions Architecture, Data Engineering, or Data Analytics Specialty.

  • Strong hands-on Python, PySpark, Apache Spark, and SQL skills.

  • Experience with AWS S3, Glue ETL, Glue Data Catalog, Lambda, EventBridge, Step Functions, Athena, and EMR.

  • Hands-on Kafka integration, including event publishing and consumption.

  • Experience with Apache Iceberg, CDC processing, schema evolution, and snapshots.

  • Experience with pandas, requests, boto3, and awswrangler.

  • Experience processing Avro, Parquet, ORC, JSON, and XML data formats.

  • Strong FastAPI or Flask development experience, including Swagger/OpenAPI.

  • Experience implementing OAuth2/JWT authentication for REST APIs.

  • Experience with Docker, Kubernetes, and Amazon EKS.

  • Hands-on experience building agentic AI workflows and using Copilot/GPT for coding.

  • Production troubleshooting experience using CloudWatch and Datadog.

Similar jobs

Apply