Lead Principal Software Engineer, Core Infrastructure
$146,300 - $306,400 per year
Apply (opens in a new tab)Lead Principal Software Engineer, Core Infrastructure
Posted today
SENIORITY
Lead
SALARY
$146,300 - $306,400 per year
About the role
- Mentor the team in the architecture and design of highly scalable, interdependent distributed systems such as DNS and L7 Proxy, ensuring horizontal and vertical scalability and overall performance, including leveraging distributed state management tools.
- Lead the identification of performance and scalability bottlenecks and recommend solutions to optimize code and/or systems for large-scale data processing and high-throughput requirements to improve performance for hyper-scale systems.
- Lead collaboration with stakeholders to define system scalability requirements, ensuring the defined requirements meet customer expectations.
- Leverage deep expertise to design high-impact, interdependent systems to scale with elasticity (e.g., effectively scaling both up and down).Drive innovation in the use of data plane platforms.
- Evaluate whether systems are meeting nonfunctional scalability requirements, and proactively anticipate growing business needs within the business unit.
- System Reliability Performance:
- Define key performance indicators (KPIs) and telemetry to identify risks, gaps, or cyclical dependencies in running, interdependent systems.
- Drive the creation and customization of highly complex dashboards, telemetry systems, and alerting mechanisms, proactively ensuring system health and reliability.
- System Reliability Design:
- Design and oversee the implementation of fault-tolerant, interdependent systems capable of withstanding in-service updates by implementing sophisticated redundancy, replication, and automatic failover capabilities.
- Lead the design and implementation of systems that effectively handle service disruptions (e.g., network partitions) by prioritizing consistency, availability, or partition tolerance.
- Guide the optimization of advanced mechanisms to handle network unreliability, including load-shedding, throttling, and rate-limiting.
- Design interdependent systems that are durable and adhere to service level objectives (SLOs), driving standards for availability and durability of other computing services within the organization
- Maintain expertise in industry standards for verifying correctness and apply existing techniques to interdependent systems.
- Formally verify complex features (e.g., via TLA+) to ensure system design correctness for various interdependent systems.
- Develop advanced strategies for data replication and synchronization, ensuring robust data integrity and availability
- Operational Troubleshooting & Incident Management:
- Advise on efforts to diagnose, debug, and resolve complex issues in active, interdependent systems to support ongoing operation.
- Develop and implement comprehensive strategies to prevent interruptions, ensuring no maintenance windows are required for customers and users when resolving issues.
- Maintain expertise in dependencies, dependents, and owned systems to drive effective troubleshooting and performance.
- Set standards for operational readiness and standard operating procedures within the department, and hold third-party partners accountable for meeting those standards.
- Oversee operational support rotations, providing expert guidance in incident response and leading root cause investigations to prevent future occurrences.
- Compliance & Security:
- Architect advanced security measures to protect data and applications in multi-tenant environments, and lead initiatives to enhance data and application protection.
- Guide the execution of comprehensive remediation plans to address identified security vulnerabilities.
- Ensure cloud infrastructure is in compliance with industry standards and regulations, and guide documentation efforts across projects.
- Automation & Change Management:
- Develop enterprise-level automation tools and strategies (e.g., Infrastructure as Code (IaC)) and oversee their implementation.
- Drive alignment of change management plans and organizational initiatives for patching, updating, and rolling back applications, and design interdependent systems to allow for automation of these processes.
Before you apply
Applying takes about a minute. These four things decide how fast it moves after that.
Your profile is current
It's what we read first. Occupations, seniority and locations matter more than a long history.
Two examples you can talk through
Not a portfolio — just two pieces of work where you can explain the decisions and what you'd change.
A number in mind
What you're on now and what would make you move. We negotiate better when we know both.
Your notice period
Employers plan around it, and it's the question that stalls offers most often.
Once you apply, someone reads it and calls you before anything reaches the employer — usually within two working days.
