Quick overview
Job description
We are looking for a Data Engineer with strong hands‑on experience designing, developing, and managing large‑scale data workflows across structured and unstructured datasets. This role focuses heavily on building reliable RAG (Retrieval-Augmented Generation) pipelines, orchestrating ETL/ELT processes, and deploying scalable data systems in AWS.
Responsibilities
- Design, build, and maintain RAG pipelines, including document ingestion, indexing, embedding workflows, and model retrieval flows.
- Develop and manage structured and unstructured data pipelines supporting analytics, ML, and application workloads. Build and optimize ETL/ELT pipelines in AWS using services such as S3, Lambda, Step Functions, EMR, Glue, ECS/EKS, and IAM best practices. Implement and operate NiFi flows for high‑throughput, low‑latency data ingestion and transformation.
- Develop, orchestrate, and schedule workflows using Prefect, ensuring reliability, observability, and proper error handling. Implement indexing, search, and retrieval patterns using ElasticSearch, including schema design, cluster management, and query optimization.
- Collaborate closely with architecture, ML, and application teams to support scalable data solutions.
- Ensure data quality, lineage, governance, and security across all pipelines.
- Monitor system performance and troubleshoot issues across distributed data systems.
Qualifications
- Solid understanding of ETL/ELT processes and data modeling best practices.
- Hands‑on experience implementing workflows in Prefect (Prefect 2.0 preferred). In‑depth knowledge of ElasticSearch indexing, cluster management, and search optimization.
- Proficiency in Python and familiarity with common data libraries (Pandas, PySpark, requests, etc.).
Preferred Qualifications
- Strong experience with Apache NiFi for data flow management and real‑time ingestion.
- Experience building or maintaining RAG pipelines (e.g., vector databases, embeddings, document chunking strategies, retrieval optimization).
- Proven ability to manage structured and unstructured data pipelines at scale.
- Experience with AWS cloud services for data engineering.
- Strong version control and CI/CD experience.
- Experience with vector databases (OpenSearch, Pinecone, Weaviate, etc.)
- Familiarity with containerized workflows (Docker, Kubernetes)
- Experience supporting LLM or generative AI production systems
- Background in distributed systems, streaming platforms, or data mesh architectures
Clearance
- An active TS/SCI federal security clearance is required
As required by local law, Accenture Federal Services provides reasonable ranges of compensation for hired roles based on labor costs in the states of California, Colorado, Connecticut, Hawaii, Illinois, Maine, Maryland, Massachusetts, Minnesota, New Jersey, New York, Ohio, Vermont, Virginia, Washington, and the District of Columbia. The base pay range for this position in these locations is shown below. Compensation for roles at Accenture Federal Services varies depending on a wide array of factors, including but not limited to office location, role, skill set, and level of experience. Accenture Federal Services offers a wide variety of benefits. You can find more information on benefits here. We accept applications on an on-going basis and there is no fixed deadline to apply.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- SL
Founding Engineer, Platform & DataRecently posted Auto Apply
Sylvan Labs · New York · On-site · Posted 21 Nov 2026 🇺🇸
SnowflakeBigQueryDatabricks+5$200k - $350k/yr - CE

Staff Data EngineerRecently posted Auto Apply
Consumer Edge · United States · Hybrid · Posted 21 Nov 2026 🇺🇸
SQLSnowflakeAirflow+6Technology$175k - $210k/yr - HI
Deployment Strategist - DatabricksRecently posted Auto Apply
Hiflylabs · Jersey City, NJ · On-site · Posted 20 Nov 2026 🇺🇸
DatabricksAgileData AnalysisTechnologySalary not listed - AI

Senior Data Software EngineerRecently posted Auto Apply
Airbnb · United States · Hybrid · Posted 20 Nov 2026 🇺🇸
SQLScalaETL+3Technology$196k - $227k/yr - MO

Senior Data Engineer Auto Apply
Momsmeals · Ankeny · Remote 🇺🇸
Stakeholder ManagementSQLEncryption+9Technology$120k - $130k/yr - GA

Senior Databricks Apps Full-Stack Engineer Auto Apply
Growth Acceleration Partners · Austin · Hybrid · Posted 3 Nov 2026 🇺🇸
FastAPISQLApache+7TechnologySalary not listed - GA

Staff Data EngineerRecently posted Auto Apply
Gamechanger · GameChanger Remote - US · Remote · Posted 18 Nov 2026 🇺🇸
SQLAWSMLOps+8Technology$193k - $210k/yr - GH

Staff Data EngineerRecently posted Auto Apply
Garner Health · New York City · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLAWSLooker+13Technology$256k - $287k/yr - LA
Data Engineer IIRecently posted Auto Apply
Lantern · Dallas · Hybrid · Posted 19 Nov 2026 🇺🇸
Talent AcquisitionGCPSQL+7TechnologySalary not listed - CL

Founding Data EngineerRecently posted Auto Apply
Clera · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
SQLBigQueryERP+5Technology$170k - $200k/yr - MI
Lead Forward Deployed Engineer, Data Auto Apply
Midstream · United States · Remote · Posted 12 Nov 2026 🇺🇸
OnboardingWorkdayOracle+9EngineeringSalary not listed - MA

Senior Data EngineerRecently posted Auto Apply
Machinify · United States · Remote · Posted 18 Nov 2026 🇺🇸
SQLApache SparkAWS+3Technology$180k - $220k/yr