Automation Engineer- Ansible/Puppet/Chef

Automation Engineer

We are seeking someone with experience with automation and orchestration tools such as Ansible, Puppet, and Chef. Experience in UNIX, Linux, and/or Windows Operating System (OS), including file management, scripting, editing, and security. Experience with a variety of programming languages, software, or tools to develop regular or ad-hoc queries from large datasets. Familiar with network communication protocols, and other related protocols. Ability to implement solutions integrating Rest application program interface (API) based web services.

Responsibilities

• Evaluate and recommend data ingestion and pipeline tools (open source and vendor solutions) to support scalable data workflows.
• Design and implement data pipelines to ingest, process, and transform structured and unstructured data from diverse sources.
• Develop and manage data orchestration workflows using open-source or COTS tools to automate and monitor complex data pipelines.
• Design, develop, and maintain APIs or leverage existing APIs to enable seamless data integration across COTS, custom-built applications, and data platforms.
• Work with both on-premises infrastructure (Oracle/Microsoft databases, MongoDB) and cloud platforms (AWS, Azure, Google Cloud Platform, or others) to support current and future data architectures.
• Integrate data pipelines with various applications, ensuring seamless data flow and interoperability.
• Collaborate with database administrators, application owners, solution architects, and other stakeholders to understand data requirements and deliver robust solutions.
• Implement and manage data virtualization solutions, with a preference for experience in Denodo.
• Ensure data quality, security, and compliance throughout the data lifecycle.
• Document processes, architectures, and best practices for data engineering workflows

Qualifications

• BS in relevant discipline plus minimum 3 years or more years of directly related experience that demonstrates the knowledge, skills, and ability to perform the duties of the job.
• Must be able to obtain and maintain a DOE Security Clearance
• Work is performed onsite and therefore candidate must reside in the Albuquerque area.

Required Skills:

• Knowledge in automation technologies such as Ansible, Puppet, or Chef.
• Strong programming skills in Python and SQL.
• Experience designing, developing, and consuming RESTful APIs and other API technologies to facilitate data integration and interoperability.
• Experience with ETL/ELT tools and frameworks (e.g., dbt, PySpark).
• Proven experience (3+ years) in data engineering, data pipeline development, and tool evaluation.
• Excellent problem-solving skills and ability to work collaboratively in a cross-functional team.

Desired Skills:

• Knowledge of data virtualization concepts and experience with Denodo preferred.
• Hands-on experience with data orchestration tools such as Apache Airflow, Prefect, Luigi, or similar platforms for workflow automation and scheduling.
• Familiarity with big data technologies (Hadoop, Spark, Kafka) is a plus.
• Hands-on experience with cloud platforms (AWS, Azure, Google Cloud Platform) and hybrid data architectures.
• Experience integrating data pipelines with COTS and custom applications.
• Strong understanding of data modeling, metadata management, and data governance.