Quick Overview
Seniority
Mid Senior
Work mode
On Site
Location
Malvern, PA, United States
Posted
20 hours ago
DynamoDBSQLAWSETLAgileCloudFormationData PipelineGitJavaKafkaPythonRedshift
Job Description
Job Title : Senior AWS Data Engineer
Location : Malvern, PA
Ex-Vanguard and local Candidates preferred
JD :
Role Name - Senior Data Engineer
EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python, AWS- S3, AWS Lambdas, AWS Glue, SQL, DynamoDB , Kafka/ Kinesis , pySpark
ROLE_DESCRIPTION -
Overview:
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
• Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
• Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
• Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
• Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
• Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
• Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
• Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
• Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
• Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
• Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
• Expertise in building and optimizing ETL pipelines in a cloud environment.
• In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
• Strong problem-solving and troubleshooting skills.
• Familiarity with infrastructure as code tools like AWS CloudFormation.
• Experience with version control (e.g., Git) and CI/CD pipelines.
• Solid understanding of data security, data privacy, and compliance frameworks.
• Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Digital : PySpark~Core Java
Experience Required: 10 & Above
EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python, AWS- S3, AWS Lambdas, AWS Glue, SQL, DynamoDB , Kafka/ Kinesis , pySpark
ROLE_DESCRIPTION -
Overview:
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
• Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
• Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
• Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
• Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
• Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
• Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
• Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
• Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
• Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
• Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
• Expertise in building and optimizing ETL pipelines in a cloud environment.
• In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
• Strong problem-solving and troubleshooting skills.
• Familiarity with infrastructure as code tools like AWS CloudFormation.
• Experience with version control (e.g., Git) and CI/CD pipelines.
• Solid understanding of data security, data privacy, and compliance frameworks.
• Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Digital : PySpark~Core Java
Experience Required: 10 & Above
#NOTE : Are you intrusted please share your updated resume to
Similar jobs
- FT
MS Fabric Data Engineer
NewFlexon Technologies Inc.
Tampa, FL🇺🇸Hybrid20 hours agoSQLETLData PipelineTechnology - CS
Data Engineer - Remote - EST - W2 / Own corporation - 35/hr on w2 - NO agency candidates
NewCyber Sphere LLC
United States🇺🇸Remote20 hours agoSQLPower BIdbtTechnology - AS
Data engineer with Atacamma
NewAlgo Soft Solutions LLC
United States🇺🇸Hybrid20 hours agoScrumAgileTechnology - AK
lead data engineer with snowflake cortrex + Aws Bed rock
NewAkshaya Inc
United States🇺🇸Hybrid20 hours agoShellAWSEncryption+4Technology - FY
Lead Data Engineer
NewFynbosys Inc
Cary, NC🇺🇸On-site20 hours agoNeo4jSQLScala+13Technology - PT
Senior Data Scientist / Data Engineer Generative AI Engine Optimization
NewProjas Technologies, LLC
United States🇺🇸Hybrid20 hours agoSQLAWSETL+13Technology - KT
Senior AI/ML Data Engineer
NewKforce Technology Staffing
Frisco, TX🇺🇸Hybrid20 hours agoNeo4jSQLAWS+9Technology - ST
Data Engineer with MS Fabric
NewSaim Technologies
United States🇺🇸Hybrid20 hours agoSQLData PipelineTechnology - KT
Remote Senior Data Engineer
NewKforce Technology Staffing
Atlanta, GA🇺🇸Remote20 hours agoSQLAWSETL+8Technology - TE
Lead AWS Data Engineer
NewTekcel8
United States🇺🇸Remote20 hours agoAWSETLApache+2Technology - CS
Journeyman Data Manager in Arlington, Virginia with Security Clearance
CIS Secure
Arlington, VA🇺🇸On-site2 weeks agoSQLMicrosoft OfficePower BI+1Technology - CS
Lead Data Engineer
NewCynet Systems
Fremont, CA🇺🇸Hybrid20 hours agoSQLETLSOC 2+2Technology