Hadoop Administrator

Job Description Responsibilities Cluster Administration

• Manage, monitor, and optimize MapR-based Hadoop clusters in a production environment.
• Oversee daily operations, including cluster health checks, performance tuning, capacity planning, and system upgrades.
• Perform installation, configuration, patching, and version upgrades for MapR distributions and ecosystem tools.

• Configure, support, and optimize multi-tenant setups, ensuring isolation, resource fairness, and performance SLAs.
• Manage YARN queues, ACLs, quotas, and security policies per tenant requirements.
• Collaborate with data engineering and analytics teams to onboard new tenants and align resource needs.

• Diagnose and resolve cluster performance issues, bottlenecks, and failures across MapR FS, YARN, Spark, Hive, Kafka, and related services.
• Manage node failures, job failures, and cluster rebalancing while maintaining uptime.
• Automate alerting, monitoring, and reporting using tools like MapR Control System (MCS), Grafana, Nagios, Prometheus, or similar.

• Implement and manage security features such as Kerberos, Ranger/Sentry policies, SSL encryption, and user/role management.
• Ensure compliance with internal and external standards for data governance and security.

• Maintain reliable backup and DR strategies using MapR snapshots, mirrors, and failover configurations.
• Ensure 24/7 availability of production environments and participate in on-call rotation.

• Develop automation for operational tasks using Python, Shell scripting, or Ansible.
• Streamline cluster provisioning, monitoring, and deployment pipelines.

• 4+ years of hands‑on experience as a Hadoop/MapR Administrator.
• Strong knowledge of MapR‑FS, MapR‑DB, MapR Streams, YARN, Hive, Spark, and Kafka.
• Experience managing production clusters with high availability requirements.
• Proficiency in Linux administration (RHEL/CentOS/Ubuntu).
• Expertise with multi‑tenant Hadoop environments, resource governance, and user management.
• Strong skills in shell scripting, automation, and performance tuning.
• Experience with monitoring tools (Grafana, Nagios, Prometheus, MapR MCS).
• Understanding of Hadoop security (Kerberos, SSL, ACLs, AD integration).

• Experience migrating from MapR to other Hadoop distributions (Cloudera, Hortonworks) – a plus.
• Knowledge of Cloud (AWS/Azure/GCP) big data services.
• Experience with CI/CD pipelines, Dev Ops tools, or containerized big data workloads.