Data Engineer

Data Engineer

Location: El Segundo, CA
Work Arrangement: Hybrid – 3 Days Onsite / 2 Days Remote

Position Summary

We are looking for a skilled Data Engineer with strong hands-on experience in Python, Apache Airflow, ETL/ELT, and SQL. The ideal candidate will be responsible for designing, developing, maintaining, and optimizing data pipelines and workflows that support business and analytical needs.

The candidate should have strong experience building reliable ETL pipelines, developing Airflow DAGs, writing complex SQL queries, and using Python for data processing and automation.

Key Responsibilities

·       Design, develop, and maintain scalable ETL/ELT data pipelines.

·       Develop and maintain Apache Airflow DAGs for workflow orchestration, scheduling, dependencies, retries, and monitoring.

·       Develop Python scripts and applications for data extraction, transformation, validation, and automation.

·       Write complex and optimized SQL queries for data extraction, transformation, and manipulation.

·       Integrate data from multiple sources and ensure accurate and reliable data movement.

·       Monitor, troubleshoot, and optimize Airflow workflows and data pipelines.

·       Implement data validation and quality checks to ensure data accuracy and consistency.

·       Identify and resolve pipeline failures, data issues, and performance problems.

·       Optimize ETL processes for reliability, scalability, and performance.

·       Work closely with data analysts, business teams, and other technical teams to understand data requirements.

·       Maintain technical documentation for data pipelines, workflows, and processes.

·       Follow development best practices including version control, code reviews, testing, and documentation.

Required Skills

·       Strong hands-on experience with Apache Airflow.

·       Strong Python programming and scripting skills.

·       Strong SQL skills, including complex queries, joins, CTEs, subqueries, and data transformations.

·       Hands-on experience developing ETL/ELT pipelines.

·       Strong understanding of data pipelines, data integration, and data processing.

·       Experience troubleshooting and monitoring production data pipelines.

·       Strong analytical and problem-solving skills.

·       Good communication and collaboration skills.

Preferred Skills

·       Experience with cloud-based data environments such as AWS, Azure, or Google Cloud Platform.

·       Experience with data warehouses or relational databases.

·       Experience with Git/version control and CI/CD practices.

·       Experience with data quality, validation, and pipeline monitoring.

·       Familiarity with Linux/Unix environments.

Qualifications

·       Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

·       5+ years of experience in Data Engineering.

·       Strong hands-on experience with Python, Apache Airflow, ETL/ELT, and SQL.

About Us:
AgreeYa is a global systems integrator delivering competitive advantage for its customers through software, solutions, and services. Established in 1999, AgreeYa is headquartered in Folsom, California, with a global footprint and a team of more than 1,850 staff across offices. AgreeYa works with 500+ organizations ranging from Fortune 100 firms to small and large businesses across industries such as Telecom, Banking, Financial Services & Insurance, Healthcare, Utility & Energy, Technology, Public sector, Pharma & Biotech, and others. Please visit us at Agreeya for more information.
 

AgreeYa is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion, gender identity, sexual orientation, national origin, disability, veteran status or other protected characteristics. Visit our website to learn about our Career & Culture.