Location: Toronto / Sunnyvale
We're looking for a deeply technical, hands-on engineering leader for our Inference Service Platform. You will lead a high performing team to tackle a critical challenge: scaling LLM inference on Cerebras’ advanced compute clusters and delivering a world-class, on-prem solution for enterprise customers. In this role, you’ll set the technical vision while staying close to the code, architecting highly reliable, low latency distributed systems. If you have proven expertise in distributed systems and scaling modern model-serving frameworks, we want to hear from you.
Responsibilities
- Provide hands-on technical leadership, owning the technical vision and roadmap for the Cerebras Inference Platform, from internal scaling to on-prem customer solutions.
- Lead the end-to-end development of distributed inference systems, including request routing, autoscaling, and resource orchestration on Cerebras' unique hardware.
- Drive a culture of operational excellence, guaranteeing platform reliability (>99.9% uptime), performance, and efficiency.
- Lead, mentor, and grow a high-caliber team of engineers, fostering a culture of technical excellence and rapid execution.
- Productize the platform into an enterprise-ready, on-prem solution, collaborating closely with product, ops, and customer teams to ensure successful deployments.
Skills & Qualifications
- Technical Leadership: 6+ years in high-scale software engineering, with 3+ years leading distributed systems or ML infra teams; strong coding and review skills.
- Inference Expertise: Proven track record scaling LLM inference: optimizing latency (<100ms P99), throughput, batching, memory/IO efficiency and resources utilization.
- ML Systems Knowledge: Expertise in distributed inference/training for modern LLMs; understanding of AI/ML ecosystems, including public clouds (AWS/GCP/Azure).
- Frameworks & Tools: Hands-on with model-serving frameworks (e.g. vLLM, TensorRT-LLM, Triton or similar) and ML stacks (PyTorch, Hugging Face, SageMaker).
- Infrastructure: Deep experience with orchestration (Kubernetes/EKS, Slurm), large clusters, and low-latency networking.
- Operations & Monitoring: Strong background in monitoring and reliability engineering (Prometheus/Grafana, incident response, post-mortems).
- Leadership & Collaboration: Demonstrated ability to recruit and retain high-performing teams, mentor engineers, and partner cross-functionally to deliver customer-facing products.
Preferred Skills
- Experience with on-prem/private cloud deployments.
- Background in edge or streaming inference, multi-region systems, or security/privacy in AI.
- Customer-facing experience with enterprise deployments.
