Job opportunity Financial Services Client/AWS Data Engineer/Newark, NJ (hybrid 3 days)

Client: Financial Services

Job title: AWS Data Engineer

Location: Newark, NJ (hybrid 3 days)

Duration: 24 Months

Skills: AWS, AWS- Glue, S3, Athena, Redshift, Databricks

General Roles & Responsibilities:

• Technical Leadership: Demonstrate leadership, and ability to guide business and technology teams in adoption of best practices and standards
• Design & Development: Design, develop, and maintain robust, scalable, and high-performance data estate
• Architecture: Architect and design robust data solutions that meet business requirements & include scalability, performance, and security.
• Quality: Ensure the quality of deliverables through rigorous reviews, and adherence to standards.
• Agile Methodologies: Actively participate in agile processes, including planning, stand-ups, retrospectives, and backlog refinement.

Technical Responsibilities:

• Optimize data pipelines for performance and efficiency.
• Work with Databricks clusters and configuration management tools.
• Use appropriate tools in the cloud data lake development and deployment. Developing/implementing cloud infrastructure to support current and future business needs.
• Provide technical expertise and ownership in the diagnosis and resolution of issues.

Required Technical Skills:

• Experience & Proficiency with Databricks platform - Delta Lake storage, Spark (PySpark, Spark SQL). Must be well versed with Databricks Lakehouse, Unity Catalog concept and its implementation in enterprise environments.
• Experience & Proficiency with AWS Data Services S3, Glue, Athena, Redshift etc.| Airflow scheduling
• Proficiency in SQL and experience with relational databases.
• Proficiency in at least one programming language (e.g., Python, Java) for data processing and scripting.
• Experience with DevOps practices - AWS DevOps for CI/CD, Terraform/CDK for infrastructure as code