Senior HPC and Platform Engineer

Job summary

Own the execution platform and delivery pipeline for scientific geoscience workloads, making them run consistently across workstations, on-premises HPC, managed services, and cloud environments, including bring-your-own-cloud deployments. Build distributed execution, secure infrastructure, performance and storage optimizations, CI/CD, release engineering, and observability across technologies such as Slurm, PBS, Dask, Ray, Databricks, and Azure, AWS, and GCP batch services.

Qualifications

• At least five years of professional engineering experience in distributed computing, HPC, or cloud platform engineering.
• Production experience with at least one distributed compute framework.
• Experience with infrastructure as code and working proficiency in one or two major cloud providers.
• Experience owning CI/CD and release pipelines end to end.
• Sound security practices for cloud and containerized deployments, including identity management, secrets handling, and least privilege.
• Ability to work independently in a small, remote, multi-timezone team and communicate asynchronously.
• Fluent written and spoken English.
• Desired: experience with Slurm or PBS in a shared HPC environment.
• Desired: experience with managed cloud batch or Spark services such as Databricks, Azure Batch, or AWS Batch.

Responsibilities

• Design and build distributed execution with task dependencies, elastic scaling, checkpointing, retries, structured errors, and partial-output preservation.
• Run workloads through a consistent interface across local multiprocessing, Dask, Ray, Slurm, PBS, Spark services, and cloud batch services.
• Develop structured job submission, status, progress, and cost reporting, with safeguards against invalid or excessive jobs.
• Build job admission, entitlement, and metering capabilities.
• Maintain Terraform infrastructure for repeatable deployment into bring-your-own-cloud and externally administered environments.
• Containerize compute components as lean, independently versioned images.
• Implement deployment security, including identity and access management, secrets handling, network isolation, encryption, and image and dependency scanning.
• Optimize compute-bound workloads and diagnose blob and object storage performance issues.
• Establish and maintain measured performance baselines.
• Own CI/CD, build and release pipelines, nightly regression testing, native installers, and containerized cloud delivery.
• Provide observability for job lifecycle, resource use, and costs across compute backends.
• Support deployed systems and incorporate operational findings into platform design.

Skills

Certifications

Degrees

Languages

Industry

Relocation