System Engineer - ( L3 End-to-End support expert)

The System Engineer – L3 End-to-End Support Expert is a senior technical role responsible for supporting complex enterprise production environments across applications, Linux, Kubernetes, cloud, infrastructure, networking, integrations and monitoring.

The role combines expert-level L3 production Support, proactive system reliability, cloud/infrastructure operations and deployment capabilities for enterprise-scale AMI and utility solutions.

The engineer operates as part of our global 24/7/365 Follow-the-Sun Support organization and takes end-to-end technical ownership of complex incidents and customer escalations.

ZONOS and customer-specific knowledge is not required at recruitment and will be acquired through structured transition, knowledge transfer and hands-on experience.

Role Summary:

Level 3 End-to-End Support provides expert-level technical Support for complex and critical production issues across the complete solution stack.

The role is responsible for:

· Complex end-to-end troubleshooting and root-cause analysis

· Production reliability and availability

· Application and backend Support

· Linux and operating-system troubleshooting

· Kubernetes

· Cloud and infrastructure

· Networking

· Monitoring and observability

· Automation

· HA and disaster recovery

· Security and performance

· Technical customer communication

· SLA-driven incident management

The objective is not only to restore service but to identify the root cause, implement sustainable solutions, prevent recurrence and continuously improve service reliability and supportability.

Your mission is to ensure that our customers operate stable, reliable and supportable production environments.

You will take ownership of complex technical issues from initial investigation through resolution and RCA while coordinating with other technical specialists where required.

You will also contribute to the continuous evolution of our Support organization by improving monitoring, automation, documentation, operational readiness and knowledge sharing.

Your work will directly contribute to:

· Higher production stability

· Faster incident resolution

· Improved SLA performance

· Better customer experience

· Earlier identification of operational risks

· Reduced manual operational effort

· Better Support documentation

· Increased automation

· Stronger 24/7 Support capabilities

· Continuous improvement of our products and services

L3 End-to-End Support.
· Provide expert-level L3 end-to-end Support for complex application, backend, infrastructure and cloud-related issues.

· Independently analyze, troubleshoot and resolve customer-reported production issues.

· Take technical ownership of incidents from investigation through resolution.

· Perform deep root-cause analysis (RCA) across the complete solution stack.

· Support major incidents, outages and critical customer escalations.

· Ensure issues are handled according to defined SLAs.

· Identify SLA risks early and actively escalate where required.

· Coordinate with Development, DBA, Device Integration, System Engineering and other specialists where deeper expertise is required.

· Participate in customer calls and technical incident sessions.

· Clearly communicate customer impact, findings, recovery status, risks and next actions.

Reliability, Availability & Performance

· Ensure the reliability, availability, performance and operational health of production systems.

· Monitor system health and proactively identify potential risks.

· Analyze performance bottlenecks and resource constraints.

· Implement preventive measures and performance optimizations.

· Contribute to scalability and resilience improvements.

· Support capacity planning and performance analysis/testing.

· Support operational cost optimization where infrastructure is managed by Cuculus.

· Identify recurring operational issues and drive sustainable corrective actions.

Monitoring & Observability

Build, maintain and continuously improve observability across production environments, including:

· Metrics

· Logs

· Dashboards

· Alerting

· Tracing where applicable

· Service/process monitoring

· System-health monitoring

· Performance monitoring

Hands-on experience with MONIT, Grafana and Prometheus, or equivalent technologies such as:

· Zabbix

· Nagios/Icinga

· ELK / Elastic Stack

· OpenSearch

· Loki

The engineer should use monitoring proactively rather than relying solely on customer-reported incidents.

Troubleshooting & Root-Cause Analysis

Perform advanced troubleshooting across:

· Enterprise applications

· Linux/OS

· Kubernetes

· Cloud

· Infrastructure

· Networking

· Databases and DB connectivity

· Monitoring

· Integrations

· Application dependencies

Use logs, metrics, configuration and dependency analysis to distinguish between symptoms and underlying root causes.

Perform application-level analysis and work together with Development for deeper code-level investigation where required.

Automation

Develop scripts and automation using technologies such as:

· Shell/Bash

· Python

· Infrastructure/deployment automation tools

Automate:

· Operational tasks

· Deployments

· Monitoring

· Health checks

· Diagnostics

· Data collection

· Repetitive Support activities

· Infrastructure provisioning where applicable

The objective is to reduce operational toil, improve consistency and minimize manual errors.

Security, Scalability & Resilience

Support and continuously improve:

· System security

· OS hardening

· Security patches

· Hotfixes

· Certificates

· Access controls

· Performance

· Scalability

· Reliability

· High Availability

· Disaster Recovery

Support controlled production changes according to defined operational and change-management processes.

High Availability & Disaster Recovery

Support enterprise HA and DR environments, including:

· HA architectures

· Failover concepts

· Disaster-recovery procedures

· Recovery testing

· Production recovery

· DC/DR environments

Participate in troubleshooting and recovery of HA/DR-related production incidents.

Cloud & Infrastructure Management

Deploy, manage and support cloud and infrastructure environments across:

· AWS

· Azure

· NTT

· Equivalent cloud platforms

· Customer-operated cloud

· On-premises infrastructure

· Hybrid environments

Responsibilities include:

· Compute/VM environments
· Networking
· Storage
· Cloud access and security
· Permissions and RBAC
· Infrastructure monitoring
· Troubleshooting
· Resource utilization
· Cloud resource optimization
· Infrastructure provisioning
· UAT, staging and production environments

The role must be capable of supporting different operating models where infrastructure may be managed by Cuculus, a cloud provider, a partner or the customer.

Kubernetes

Deploy, manage and troubleshoot Kubernetes environments and applications, including:

· Clusters

· Pods

· Deployments

· Services

· Networking

· Storage

· Configuration

· Logs

· Monitoring

· Application dependencies

· Production issue analysis

Strong hands-on Kubernetes capability is required.

Project Deployment & Operational Readiness

Support technical deployment and transition activities where required, including:

· Cloud environment preparation
· Kubernetes deployments
· Application deployment
· HA and DC/DR setup
· UAT/staging/production environments
· Deployment automation
· Technical validation

Work closely with Project and Engineering teams to ensure new solutions are operationally ready and supportable before transition into Support.

Operational readiness should include:

· Monitoring

· Logging

· Runbooks

· Support documentation

· Architecture/dependency information

· Deployment procedures

· Recovery procedures

· Known limitations

· Escalation paths

· Knowledge transfer

· Production-readiness validation

Project ownership remains with the respective Project organization; the L3 Support role contributes technical expertise and operational-readiness requirements.

Incident Management & SLA Operations

Handle major incidents, outages and technical escalations, including:

· Incident leadership

· Technical investigation

· Customer communication

· Stakeholder coordination

· Expectation management

· Escalation

· Service recovery

· RCA

· Post-incident review

Experience with structured ITSM/ticketing systems, preferably Jira / Jira Service Management, including:

· Incident tracking

· Prioritization

· Assignment

· Escalation

· Documentation

· Customer updates

· SLA follow-up

The engineer is expected to actively manage SLA obligations and escalate potential SLA breaches before they occur.

---

24/7/365 Follow-the-Sun Support

The role participates in the global 24/7/365 Follow-the-Sun Support organization through planned shift rotation.

This includes scheduled:

· Day shifts

· Night shifts

· Weekends

· Public holidays

Structured handover between shifts and time zones is mandatory.

The handover should cover:

Ticket → Severity/SLA → Customer impact → Current status → Findings → Actions completed → Monitoring status → Pending actions → Risks → Specialist required → Next owner

Documentation & Knowledge Management

Create, maintain and continuously improve:

· SOPs

· Runbooks

· Troubleshooting guides

· Operational procedures

· Known-error documentation

· Reliability standards

· Knowledge-base articles

· Recovery procedures

Knowledge gained through incidents must be documented and reused rather than remaining with individual engineers.

Training, Mentoring & Knowledge Replication

Support the development of the wider Support organization through:

· Knowledge transfer

· Mentoring

· Cross-training

· Train-the-trainer

· Knowledge replication

· Practical troubleshooting sessions

· Documentation

Senior engineers are expected to actively transfer their knowledge to other Support resources.

AI Knowledge Management

Capture validated knowledge from resolved L2/L3 incidents, including:

· Symptoms

· Diagnostics

· Technical findings

· RCA

· Workarounds

· Recovery procedures

· Known solutions

Structure this knowledge for incorporation into the AI-based tool L1/L2 Support capability.

The continuous improvement cycle is:

Resolve → RCA → Document → Cross-train → Replicate → Update AI tool → Assess competency → Increase independence

Cross-Skilling – DBA

As part of the Support cross-skilling model, the L3 E2E Support Expert is expected to develop a minimum target of 50% operational DBA competency.

Operational cross-skilling includes:

· Database health/status

· Monitoring and alert interpretation

· DB connectivity

· Basic SQL/query investigation

· Logs

· Sessions/connections

· Capacity/disk checks

· Standard DB health checks

· Approved runbook-based activities

The 50% target is competency-based, not simply completion of 50% of training material.

Competencies are classified as:

A – Independent: Can perform independently.

B – Guided: Can perform with specialist consultation/approval.

C – Specialist Only: Remains with the DBA specialist.

Advanced DBA activities such as complex RAC/Data Guard administration, advanced PostgreSQL/ClickHouse changes, major migrations/upgrades, advanced tuning and high-risk recovery remain specialist responsibilities.

Required skills and Qualifications:

· Bachelor's degree in Computer Science, Information Technology or related field, or equivalent relevant professional experience.

· Minimum 5 years relevant IT experience, including at least 3 years in production/application/infrastructure Support.

· Senior/L3 troubleshooting capability.

· Strong Linux administration and troubleshooting skills.

· Strong Kubernetes deployment and troubleshooting experience.

· Hands-on cloud experience with AWS, Azure or equivalent.

· Strong networking fundamentals, including TCP/IP, DNS, routing, connectivity, ports and firewalls.

· Experience with scripting using Shell/Bash, Python or equivalent.

· Strong monitoring and observability experience.

· Understanding of HA, DR and enterprise architectures.

· Enterprise-application troubleshooting experience.

· Experience working in SLA-driven production environments.

· Experience handling critical incidents and escalations.

· Strong analytical and root-cause-analysis skills.

· Strong documentation and knowledge-sharing capabilities.

· Ability to communicate technical topics clearly to customers and internal stakeholders.

· Willingness and capability to participate in 24/7/365 Follow-the-Sun shift rotation.

Preferred Skills:

· Utility-industry experience.

· HES and/or MDM experience.

· Multi-cloud or hybrid-cloud experience.

· Experience with MONIT, Grafana, Prometheus, Zabbix, Nagios/Icinga, ELK, OpenSearch or Loki.

· Experience with Terraform, Ansible, Docker or equivalent automation technologies.

· Jira / Jira Service Management experience.

· Experience with large-scale enterprise infrastructure and production environments.

· Experience with technical customer communication and de-escalation.

· Training or mentoring experience.

The L3 End-to-End Support Expert combines:

Technical expertise + E2E ownership + production reliability + proactive monitoring + automation + customer communication + SLA ownership + cross-skilling + knowledge replication + continuous improvement.

The role is not limited to reacting to incidents. It proactively contributes to service reliability, operational readiness, Support quality and the continuous improvement of the global Support organization.