Data Architect (PySpark)

Job Title - Data Architect (PySpark)
Location - Jersey City, NJ, Plano, TX

Type: Fulltime

Job Description:

Required Skill and Experience

• Experience in technology consulting, enterprise and solutions architecture and architectural frameworks
  Experience in Cloudera Hadoop ecosystem, i.e. Hadoop, Hive, Scala, SPARK, Sqoop, Flume, Kafka, Python, NoSQL database technologies
  Understanding and knowledge of Hadoop architecture and various components such as high availability architecture, experience in workload management, scalability and distributed platform architecture
  Experience with design and implementation of ETL/ELT framework for complex warehouses/marts. Knowledge of large data sets and experience with performance tuning and troubleshooting
  Background in all aspects of software engineering with strong skills in parallel data processing, data flows, REST APIs, JSON, XML, and micro service architecture

Preferred Skill and Experience

• Experience in working in any hyperscalers, AWS, Azure, Google Cloud Platform, Databricks and Snowflake
  Experience in data virtualization/data federation for creating data product (e.g. in Starburst) in heterogenous environment (Hadoop, RDBMS, NoSQL etc.)
  Hands-on development mentality, with a willingness to troubleshoot and solve complex problems