Start searching
Back to Jobs
Recently postedTechnology

Data Engineer Lead / Data Warehouse Architect

COOLSOFTLansing, MI🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationLansing, MI, United States
DockerSQLGitHTMLApache SparkAWSETLTableauApacheDatabricksGoogle CloudJenkinsKafkaPythonRedshiftTerraformdbtAzureKubernetesMachine LearningPower BIAgile

Job description

Job Summary
The role will lead the design and delivery of enterprise Data Lake, Lakehouse, Data/ML products and distributed data architectures using Agile/DevOps practices. The position requires strong hands-on experience with cloud data platforms, data modeling, data pipelines, Medallion Architecture and ML/data engineering technologies.
Required Experience:
  • 4+ years hands-on experience building enterprise Data Lakes on AWS, Azure or Google Cloud.
  • Experience ingesting relational, semi-structured and unstructured data.
  • 4+ years transforming Data Lake datasets into data products.
  • Experience maintaining traceability from data-product attributes/features back to critical data elements.
  • 2+ years enforcing data-sharing policies and access controls for data products.
  • 4+ years designing and implementing data, analytics and AI solutions for large enterprises.
  • Strong Python / PySpark / SQL experience.
  • Experience extracting data from relational databases, flat files, CSV, HTML and JSON sources.
  • Experience implementing Lakehouse Medallion Architecture.
  • Experience working with Parquet data formats.
  • Experience designing Star Schema-based operational data stores.
  • Power BI or Tableau experience.
Key Technologies:
  • Databricks
  • SageMaker
  • Apache Spark / PySpark
  • Apache Kafka
  • Kubernetes
  • SQL / NoSQL
  • Graph Databases
  • Python / R
  • Terraform
  • Docker
  • YAML
  • DAGs
  • GitHub
  • Jenkins
  • dbt
  • AWS / Azure / Google Cloud
  • Parquet
  • Power BI / Tableau
  • AWS Glue
  • Redshift Spectrum
  • DataHub
  • Machine Learning
Key Responsibilities:
  • Design, develop and maintain enterprise data pipelines.
  • Architect and optimize data infrastructure.
  • Design storage solutions and orchestration frameworks.
  • Develop ETL/ELT processes.
  • Transform and prepare data for analytics and operational systems.
  • Establish data quality, reliability and security standards.
  • Implement validation and governance practices.
  • Lead and mentor data engineering teams.
  • Review technical designs and code.
  • Collaborate with data scientists, analysts, product teams and business stakeholders.
  • Support the State of Michigan technology environment.
  • Work with the University of Michigan proof-of-concept team to promote data engineering best practices.
  • Provide technical training and guidance to data engineers.
Preferred Skills:
  • GitHub
  • Jenkins
  • Terraform
  • YAML
  • Docker
  • DAGs
  • Databricks
  • SageMaker
  • Data governance/security tools
  • Data-sharing policy enforcement
  • Data engineering standards and best practices
  • Lakehouse/Medallion Architecture
  • Data/ML pipeline development
Important:The ideal candidate should be a hands-on senior Data Engineer/Architect, not just a data analyst. Strong experience with Cloud + Data Lake/Lakehouse + Python/PySpark/SQL + Medallion Architecture + Data Products + Data Governance is critical.Please send me your updated resume

Similar jobs

Apply