Jobs / VERDA
Senior Observability Engineer
VERDA · London, ENG, United Kingdom
London, ENG, United KingdomFull timeRemote
Remuneration
Cash and equity compensation along with local benefits
Location
London, ENG, United Kingdom
Visa sponsorship
Not specified
Job summary
The Senior Observability Engineer designs, develops, and operates a shared observability platform spanning bare-metal servers, virtual machines, Linux, Kubernetes, GPU clusters, networks, and cloud-native applications. The role improves telemetry, incident detection and response, and AI-assisted monitoring using Grafana and tools from the VictoriaMetrics and Prometheus ecosystems, while establishing OpenTelemetry standards and collaborating across engineering teams.
Benefits
Cash and equity compensationLocal benefits
Qualifications
- Hands-on experience with Grafana and production telemetry systems, including VictoriaMetrics and VictoriaLogs or comparable backends.
- Strong Linux fundamentals, production Kubernetes experience, and experience monitoring cloud-native applications and distributed systems.
- Practical experience with instrumentation, metrics, traces, structured logs, trace context propagation, and Collector pipelines.
- Experience with declarative deployments, CI/CD, and configuration management tools such as Argo CD, GitLab CI/CD, Ansible, and SaltStack.
- Experience investigating production incidents, maintaining alerts, and sharing operational responsibilities.
- Ability to explain technical decisions, guide teams, and support adoption of shared observability practices.
Responsibilities
- Design, scale, and operate shared metrics, logs, and tracing services.
- Manage declarative Kubernetes deployments and automate host and virtual machine configuration.
- Improve monitoring across bare-metal hardware, Linux, physical and overlay networks, and Kubernetes.
- Build telemetry pipelines and dashboards for GPU clusters, AI training runs, and inference services.
- Define shared OpenTelemetry standards for instrumentation, context propagation, and Collector pipelines.
- Build dashboards and alerts, improve PagerDuty notification workflows, and participate in on-call rotations.
- Document observability standards and support their adoption across teams.
- Evaluate and develop AI-assisted monitoring and incident investigation approaches.
Skills
AnsibleArgo CDGitHubGitLabGitLab CIGoGrafanaKubernetesLinuxNode.jsOpenTelemetryPagerDutyPrometheusPythonSaltStackVictoriaMetricsGitHub Actions
Languages
GoNode.jsPython
Work schedule
On-call rotations
Industry
AI cloud infrastructure
Relocation
No