The Principal Observability Engineer is responsible for defining, leading, and advancing enterprise observability strategy, architecture, and implementation across applications, platforms, infrastructure, and operational services. This role requires deep hands-on expertise with Dynatrace SaaS and recent Dynatrace platform capabilities and innovations, as well as strong experience with OpenTelemetry, cloud-native observability, AIOps, and agentic operations.
This individual will serve as a principal-level technical leader, applying strategic thinking, systems architecture expertise, and sound decision-making to establish modern observability standards and scalable telemetry practices. Working in a fast-paced, highly collaborative technology environment, this role partners closely with engineering, site reliability, platform, security, infrastructure, operations, and architecture teams to guide the organization beyond traditional monitoring toward more intelligent and adaptive operational models.
Essential Duties & Responsibilities:
Define and evolve our enterprise observability vision, standards, principles, and roadmap, using strategic thinking and sound judgment to align technical direction with business needs.
Lead the implementation, optimization, and adoption of Dynatrace SaaS and its latest platform capabilities across the enterprise.
Establish scalable telemetry architecture and OpenTelemetry standards that improve consistency, interoperability, and long-term flexibility.
Design observability solutions for Kubernetes, containers, microservices, distributed applications, and public cloud environments.
Apply AIOps and agentic operations capabilities to strengthen detection, event correlation, diagnosis, automation, operational response, and continuous improvement.
Develop and maintain service health models, SLOs, dashboards, alerting strategies, telemetry governance, and observability best practices that support operational excellence and service reliability.
Collaborate across engineering, site reliability, platform, security, infrastructure, operations, and architecture functions to expand observability adoption and maturity.
Guide the transition from legacy monitoring practices to modern, adaptive, and outcome-focused observability models, including initiatives involving production systems and enterprise platform transformation.
Mentor engineers, influence technical direction, solve complex problems, and help shape enterprise architecture and engineering standards.
Required Qualifications :
Master’s degree from an accredited college or university in Computer Science, Information Systems, Engineering, or a related technical field.
8+ years of experience in observability, monitoring, site reliability engineering, platform engineering, infrastructure engineering, or related technical disciplines.
Deep hands-on experience with Dynatrace SaaS and recent Dynatrace platform capabilities and innovations.
Strong experience with OpenTelemetry and telemetry instrumentation, collection, and architecture practices.
Experience designing and implementing observability solutions for cloud-native infrastructure, distributed systems, and modern application environments.
Hands-on experience implementing AIOps and/or agentic operations capabilities.
Strong understanding of metrics, logs, traces, event correlation, service health, alerting models, and operational intelligence.
Demonstrated ability to apply lessons from traditional monitoring approaches to modern observability strategy and technical design.
Proven ability to lead technical initiatives, influence architectural decisions, and drive adoption across multiple teams and stakeholders.
Strong verbal and written communication skills, with the ability to collaborate across functions and influence technical and non-technical stakeholders.
Preferred Qualifications:
Experience leading enterprise-scale observability transformation initiatives.
Experience with AWS, Azure, and/or Google Cloud Platform.
Strong knowledge of site reliability engineering principles, including SLIs, SLOs, incident response, and operational resilience.
Experience with automation, orchestration, and remediation workflows.
Familiarity with additional observability platforms, frameworks, or ecosystems beyond Dynatrace.
Experience with scripting or programming languages such as Python, Go, Bash, or JavaScript.
Similar Jobs
What you need to know about the NYC Tech Scene
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
