Hadoop Admin w2

Job Description

Hadoop Admin

Long term

Chicago, IL

Mandatory

Skills:

• Knowledgeable in SQL server DB
• Aggressive self-starter, can operate without Infrastructure support
• IOT experience

Desired

Skills:

• 1. a. Config DB backup and recovery processes
• b. File system (Blob & HDFS) management and monitoring
• c. Manage and review Hadoop log files; house keeping
• d. Experience with at least one popular Hadoop flavor – Cloudera, Horton Works, MapR
• e. Installation and Config of Hadoop projects such as Hive, Pig, HBase, Spark, etc.
• f. Performance optimization (MR job and other)
• g. Setup performance monitoring & tuning

Cloud

• 1. a. Solid experience managing & administering cloud platform - Azure, AWS, Big Table, etc.
• b. deploying Big Data solutions in Cloud
• c. Cluster install & config
• d. Capacity estimation and configure for elasticity
• e. Script automation for allocation, deal location, etc.

Establish Big Data Security

• 1. a. Data at Rest & Transit
• b. Access from backend and front-end applications
• c. Row level security & Role based security
• d. integration - LDAP, AD, Okta

Ingestion

• 1. a. Sqoop, Kakfa, Storm – highly desirable
• b. ADF, Informatica Cloud, etc. are definite plus
• c. Setup processes for historical and incremental data ingestion

Front-end access – Configure connectivity and interfaces enabling Cognos, Tableau, Power

BI, R, and SAS applications to access Hadoop Big data.

Scheduler & Runtime environment – Configure YARN and Oozie for optimal resource utilization and end-to-end workflow process.

Deploy and configure Neo4J (Graph DB), Document DB and emerging big data repositories

Big data modeling

Experience with at least one popular Hadoop flavor – Cloudera, Horton Works, MapR

Installation and Configration of Hadoop projects