On October 1, 2026, an internal shared service became unavailable to several dependent applications. This affected multiple API endpoints and reduced availability of the charger page in the customer portal.
The incident primarily impacted users trying to access charger-related information through the portal. The issue was mitigated by restoring the affected service and deploying an additional safeguard to prevent dependent applications from waiting indefinitely on slow service responses.

## Impact

• Several API endpoints experienced degraded availability or timeouts.
  
• The charger page in the customer portal was unavailable or intermittently failing for affected users during the incident.
  
• No data loss has been identified as part of this incident.

## Timeline

All times are in CEST on October 1, 2026.

• 15:21: Support reported slowness in the portal.
  
• 15:24: Engineers identified that one of the APIs was experiencing timeouts when calling an internal shared service.
  
• 16:00: The affected API was rolled back to the previous version to restore access to a previous fallback mechanism.
  
• 17:05: The fallback mechanism was enabled, but the charger page continued to fail.
  
• 18:35: Engineers identified that another API was calling the same internal shared service without a timeout.
  
• 18:51: The affected internal service instance was restarted, after which the charger page started working again.
  
• 19:00: A production fix was deployed to add a timeout for requests from the portal API to the internal shared service.

## Root Cause

The immediate cause of the incident was that dependent APIs were affected by slow or unavailable responses from a shared internal service.
The underlying cause of the service instability is still under investigation. During the incident, service health metrics showed significantly elevated memory-management activity, which indicates that the service was under heavy load at the time.
The affected shared service currently handles both resource-intensive background processing and real-time request serving in the same runtime environment. This makes it vulnerable to resource contention. When heavy processing occurs, real-time requests can be delayed or fail, which can then affect applications that depend on the service.

## Resolution

The incident was mitigated in two steps.
First, engineers rolled back one affected API to a previous version that still supported a fallback mechanism.
Second, the affected shared service instance was restarted, which restored functionality for the charger page.
A production fix was then deployed to add a timeout to one of the dependent APIs. This reduces the risk of the portal becoming unavailable when the shared service is slow or unresponsive.

## Current Status

The charger page functionality was restored on October 1, 2026 at approximately 18:51 CEST. A production fix was deployed at 19:00 CEST to reduce the risk of similar cascading failures.
Further investigation is ongoing to determine the exact cause of the shared service instability and to define longer-term architectural improvements.