Hadoop Support with DevOps

Job Title: Hadoop Support with DevOps

Location: Plano, TX / Charlotte, NC (Onsite)

Job Type: Fulltime/Permanent

Experience: 10+ years

 

Must Have Technical/Functional Skills:

Primary Skill:  Hadoop

Secondary: Linux, DevOps

 

Key Responsibilities:

• L1, L2 & L3 level support in Hadoop Platform
• Monitor and resolve HDFS and Ozone related issues.
• Investigate cluster performance, storage utilization, and capacity concerns.
• Manage quota requests, storage allocation, and capacity planning.
• Support platform maintenance and perform integration activities.
• Coordinate with application teams during incident and problem management.
• Create and maintain operational documentation, runbooks, and knowledge articles.

 

Required Qualifications:

• Experience on L1, L2 & L3 level support in Hadoop Platform.
• Experience in Yarn, Spark and Impala job debugging and troubleshooting.
• Experience in addressing issues around Name Node, HDFS Space, File/Folder Permission & backups.
• Understanding and experience in query performance tuning and resource utilization tuning.
• Technical knowledge on relational databases, data warehousing and SQL skills.
• Hadoop, Kafka, Spark, Impala, Hive, HBase, Ozone etc.
• Strong knowledge of Hadoop Architecture, HDFS, Hadoop Cluster and Hadoop Administrator’s role.
• Intimate knowledge of fully integrated AD/Kerberos authentication.
• Experience setting up optimum cluster configurations.
• Debugging knowledge of YARN.
• Hands-on with analyzing various Hadoop log files, compression, encoding, & file formats.
• Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper and Postgres.
• Strong technical knowledge: Unix/Linux; Database (Sybase/SQL/Oracle), Java, Python, Perl, Shell scripting, Infrastructure.
• Experience in Monitoring & Alerting, and Job Scheduling Systems.
• Being comfortable with frequent, incremental code testing and deployment.
• Strong grasp of automation / DevOps tools - Ansible, Jenkins, SVN, Bitbucket.

Tools Involved:

• ETL tools: Hadoop Stack
• Job Scheduling tools: Autosys
• BI tools: Tableau; Tableau dashboard building connecting Hadoop-Hive Tables is equally preferred.