Why This Role Stands Out
This role offers an exciting opportunity to build sophisticated data solutions within a collaborative environment, allowing you to significantly impact a custom application's core functionalities. You'll thrive here if you possess advanced Python skills, a strong understanding of data pipelines and cloud technologies, and a passion for tackling complex data challenges. Apply to join a dynamic team and elevate your data engineering expertise.
Quick Overview
Job Description
The Data & Software Engineer works with a small team to build complex data flows for a custom application. Successful candidate will have advanced Python programming skills, familiarity with Java, an understanding of data security, privacy, governance and compliance principles and a demonstrated history of building production data pipelines and ETL workflows at scale. Candidate must have experience:
- Building end-to-end data pipelines leveraging Python
- Using orchestration tools to deploy data pipelines, including configuring and updating Spark Jobs
- Containerizing and deploying applications in cloud environments like AWS.
- Working with MySQL and PostgreSQL including performance tuning, schema design, and query optimization for complex, analytical workloads.
- Leveraging industry standard tools for code control (Git, IaaC control, etc.)
- Working with data catalogs, tracking data lineage and handling a variety of data formats, including Geospatial.
- Using Bash scripting for automation and data processing tasks
- Integrating Al/ML services and models
Responsibilities:
- Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
- Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
- Leverage a background in large-scale data migration or platform modernization efforts
- Contribute to data engineering documentation, best practices, and design patterns.
Minimum of 5 years' experience with:
- Apache Spark & PySpark
- Advanced Python skills (including Pandas & NumPy)
- Docker, Podman
- AWS S3, Lambda & Step functions
- Apache Iceberg, Airflow, etc.
- SQL (with Trino)
- NoSQL, DynamoDB
- Unity Catalog OSS, Apache Polaris
- Apache Superset
- Terraform or CloudFormation
- OpenLineage
- H3, PostGIS
Eligibility requirements apply.
- Employer-Paid Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (401k, IRA) with a generous matching program
- Life Insurance (Basic, Voluntary & AD&D)
- Paid Time Off (Vacation, Sick & Public Holidays)
- Short Term & Long Term Disability
- Training & Development
- Employee Assistance Program
Similar jobs
- SE
Senior Systems Engineer with Security Clearance
Serco Inc.
Albuquerque, NM🇺🇸On-site1 week agoActive DirectoryAnsibleBash+4Technology - PS
Software Engineer (Back-end, API, Python) with Security Clearance
Power3 Solutions
Fort Meade, MD🇺🇸Hybrid4 weeks agoPythonTechnology - AE
Tool Development and Integration Engineer with Security Clearance
Aerospace Corporation
Colorado Springs, CO🇺🇸On-site4 weeks agoFortranMATLABC+++3Technology - SE
Senior Systems Engineer with Security Clearance
Serco Inc.
Orlando, FL🇺🇸Hybrid5 weeks agoEmbedded SystemsBambooConfluence+3Technology - BS
Software Engineer Level 5 with Security Clearance
Base-2 Solutions, LLC
Norfolk, VA🇺🇸$10k/yrHybrid5 weeks agoFastAPISQLPostgreSQL+1Technology - SE
Lead Systems Engineer - San Diego, CA with Security Clearance
Serco Inc.
San Diego, CA🇺🇸Hybrid5 weeks agoHTTPSMicrosoft OfficeTechnology