Skip to main content
T

Artificial Intelligence Engineer

Tiger Advisory
3 hours ago
Full-time
On-site
New York City, New York, United States
AI Infrastructure Engineer

Location:

Remote — must be available during Eastern Time business hours Engagement:

6–12-month contract Financial Services

We are seeking a hands-on

AI Infrastructure Engineer

to help stand up and operate a new enterprise GPU compute environment. GPU fabric and high-speed networking Broader network operations and datacenter connectivity GPU platform, orchestration, and workload enablement The ideal candidate can operate across the full stack—from physical switch ports and fabric telemetry through GPU drivers, workload schedulers, containers, and production AI workloads. This is not a pure network engineering or pure platform engineering position.

GPU Fabric and High-Speed Networking Manage and operate an

InfiniBand XDR / Quantum-3400 rail-aligned fabric topology

supporting an 18-node, 144-GPU NVIDIA HGX cluster. Administer

NVIDIA UFM and NetQ

for subnet management, telemetry, monitoring, and firmware management. Run

NCCL and HPL validation and performance regression testing

to confirm cluster health and throughput following infrastructure changes. Diagnose performance and connectivity issues across multi-node GPU workloads. Network Operations Manage and support large-scale IP networking technologies and infrastructure. Monitor network health across on-premises and cloud environments. Work with peering and datacenter interconnect technologies, including: Private Network Interconnects (PNIs) Improve change-management processes and day-to-day network operations. Document technical standards, operating procedures, and infrastructure best practices. GPU Platform and Workload Enablement Manage the lifecycle of

NVIDIA AI Enterprise and the NVIDIA GPU Operator

. Maintain driver, firmware, CUDA, and platform compatibility across the cluster. Administer

Run

, including quotas, project structures, resource allocation, and tenant onboarding. Manage container images, runtime dependencies, and model-serving components across production and pre-production environments. Support engineering and data science teams as adoption of the GPU platform expands.

Hands-on experience supporting

NVIDIA InfiniBand

, preferably the Quantum series, within an HPC or GPU cluster environment. Familiarity with

NVIDIA HGX architecture

and rail-aligned GPU networking topologies. Experience using

NCCL, HPL, or comparable tools

to validate GPU cluster health, connectivity, and performance. Working knowledge of the

NVIDIA GPU Operator

, GPU driver and firmware lifecycle management, and compatibility planning. Experience deploying and supporting container-based workloads in an HPC or AI environment. Experience with at least one workload orchestration platform, such as

Slurm, Run, or an equivalent scheduler

. Strong knowledge of large-scale IP networking and datacenter peering/interconnect technologies. Experience working within structured change-management and production operations processes. Ability to troubleshoot across physical infrastructure, network fabric, operating systems, containers, orchestration platforms, and application workloads.

Experience with new GPU cluster bring-up or greenfield AI infrastructure deployments. Exposure to MLOps pipelines, production AI workloads, and model-serving infrastructure. Familiarity with

DWDM and wave circuit technologies

in a datacenter interconnect environment. The strongest candidate will combine deep GPU-cluster networking knowledge with practical platform engineering experience. They should be comfortable moving between InfiniBand troubleshooting, Ethernet and datacenter operations, GPU software lifecycle management, workload scheduling, and production AI enablement.