System Monitoring & Observability Engineer; Prometheus​/Grafana

Position: System Monitoring & Observability Engineer (Prometheus / Grafana)
SRT Marine Systems plc (SRT) is a market leader in the domain of international marine surveillance technology and systems. We are a respected, established, and an ambitious multi-national company headquartered in the UK with a global customer base.

The company has a worldwide impact in the marine sector by leading the next generation of maritime domain awareness technologies "MDA", products, and systems that significantly enhance security, safety, environmental protection, and sustainability. Our customers are global and range from the largest national coast guards to individual vessel owners.

SRT is an exciting company where high-quality results are rewarded. We are ambitious and constantly seek to innovate in order to deliver better products and services to our customers. We strive to make SRT a rewarding and challenging place to work, where talented, hard-working individuals have the opportunity to make a real impact across the marine industry.

Role overview of our System Monitoring & Observability Engineer (Prometheus / Grafana)

You as a System Monitoring & Observability Engineer (Prometheus / Grafana) here at SRT, you will be part of a small team tasked with implementing an end-user observability visualisation. Currently, we have observability dashboards in place for our engineers, utilising Prometheus for metrics collection and Grafana for visualisation. This initiative aims to deliver a more user-friendly solution tailored for our end-users.

Our clients are located across various countries worldwide, each with differing WAN capabilities, and our system is geographically distributed on-premises across multiple sites. We are fortunate to have a team of highly experienced engineers, including UX designers, who can provide support and guidance. Our lead observability engineer will oversee and assist with your work throughout the project in the role of System Monitoring & Observability Engineer (Prometheus / Grafana).

Key Responsibilities - System Monitoring & Observability Engineer (Prometheus / Grafana) - (not exhaustive)
 
Monitoring & Metrics Collection  
Design, configure, and maintain Prometheus-based monitoring solutions
Develop and manage metric exporters for application and system-level data
Optimise Prometheus scraping configurations and retention policies   
Alerting & Incident Response  
Define and maintain alert rules based on SLIs/SLOs and performance baselines
Ensure alerts are actionable, with minimal false positives
Participate (not necessarily lead) in on-call rotations and incident postmortems   
Observability Dashboards  
Design and maintain Grafana dashboards for real-time operational insights
Collaborate with engineering and product teams to create tailored visualisations
Provide self-service dashboard capabilities for end users   
System Performance & Reliability  
Monitor infrastructure (servers, containers, databases, services) for uptime, latency, and throughput
Identify bottlenecks and recommend improvements    

Required Skills & Experience - System Monitoring & Observability Engineer (Prometheus / Grafana)
 
Proven experience with Prometheus (including PromQL) and Grafana in production environments
Strong knowledge of Linux-based systems
Experience writing and optimising PromQL queries for alerts and dashboards
Familiarity with exporters (, , custom exporters)
Understanding of alert manager configuration and routing
Proficiency with Grafana dashboard creation and templating
Strong troubleshooting skills for infrastructure and application issues
Familiarity with containers (Docker)
Scripting skills (Bash, Python, or Go) for automation  

Just some of the benefits we offer
 
Highly Competitive Salary
Matched company pension contributions up to 5%
25 days annual leave rising to 28 days with service

Career development opportunities
Company "Get to know you" days  

SRT Marine Systems plc are an equal opportunity employer. We are committed to creating an inclusive working environment for all employees and actively encourage applications from all sectors of the community