Senior Big Data Engineer

Responsibilities

• Participate in the engineering and administration of big data systems.
• Apache Storm/Java development for both data transformation and augmentation.
• Employ best practices to ensure high availability and scalability.
• Install, configure, and maintain big data technologies and systems.
• Maintain documentation and troubleshooting playbooks.
• Respond to and resolve access and performance issues.
• Ability to work non-peak hours when needed.
• Assist in capacity planning.
• Proactively monitor performance.
• Collaborate with engineering teams to optimize data collection.
• Maintain change control and testing processes for all modifications and deployments.
• Conduct research and make recommendations on big data products, services, and standards.

• Experience with operational analytics and log management using ELK or Splunk.
• Understanding and experience with SQL and MPP databases such as Greenplum.
• Experience with Salt Stack or comparable deployment automation tools.
• Working knowledge of Zenoss, New Relic, Pager Duty or other monitoring technologies.
• Experience with Apache Drill, Apache HBase, Hive, Phoenix, or other Hadoop with SQL technologies.
• Experience working with batch-processing and tools in Hadoop technology stack (Map Reduce, Yarm, Pig, Hive, HDFS).

Basic Requirements

• BS in Computer Science, Information Systems, or related technical degree.
• 5 or more years’ experience with data transformation pipelines preferably Storm.
• 5 or more years’ experience in messaging frameworks preferably Kafka.
• 5 or more years’ experience with Java programming.

All your information will be kept confidential according to EEO guidelines.