Skip to content

Troubleshooting

Before you start

  • A running platform install (Deploy with Helm or Ansible).
  • kubectl pointed at that cluster, namespace usually oop.
  • For request tracing: an x-correlator / correlation_id — either the one your client sent, or the value OEG returned (see below).
  • Optional: Helm release name (oop-platform) if you need helm status.

Trace a request

App Provider may send x-correlator. OEG passes that value on to SRM:

  • Async work (deploy, terminate, network capability lifecycle, …): as correlation_id on the DataBus command and on the matching events.
  • Sync queries (e.g. location): as the x-correlator header on OEG’s internal HTTP call to SRM (and as correlation_id in the query payload where the contract requires it).

SRM keeps the same id on the path; OEG echoes it on the northbound response.

If the client omits x-correlator, OEG generates a UUID for that request and returns it on the response as the x-correlator header (success and error responses). Use that header value to search logs — it is the same id OEG used toward SRM.

To follow one request (adjust deployment names if needed):

CORR='<paste-correlator>'   # from your request, or from the OEG response header

kubectl logs -n oop deploy/oegcontroller --since=30m | grep -F "$CORR"
kubectl logs -n oop deploy/srmcontroller --since=30m | grep -F "$CORR"

You should see the same id on the OEG intake and on the SRM command/event path. If it stops at one component, dig there next (pod health, then that chart’s troubleshooting).

For the end-to-end correlator path across components, see Correlation ID Flow in the architecture repository.

By symptom

Start with pods, then the correlator, then the row that matches what you see:

kubectl get pods -n oop
helm status oop-platform -n oop
Symptom First checks More detail
Pod stuck in Init / CrashLoop DB role/password for that component (postgres-credentials); init container logs (wait-for-database) SRM, OEG, FM chart troubleshooting
OEG up but every call is 401 Token iss / audience oeg; Keycloak up if login is on OEG chart troubleshooting
Zones list empty / SRM fails during bootstrap Check srm.topology and SRM startup logs. An unknown control_path_ref causes InvalidTargetRef and prevents SRM startup. How to define zones, SRM chart
App deploy fails with “forbidden” Check permissions for the selected identity on the selected cluster. For in-cluster execution, check srm.workloads.namespaces; for remote execution, check the kubeconfig user's remote permissions. SRM chart
Async operation hangs / never completes Trace with x-correlator (OEG → SRM) Trace a request above
Async operation or instance ends FAILED Trace with x-correlator, then read SRM failure_detail on the matching service_order (and on capability_instance when a realization row exists) — why the order or capability failed Read SRM's failure detail below
Partner / federation callbacks never arrive FM public/callback URLs reachable from the partner FM chart
Network capability / NEF errors NEF reachability and adapter settings EURECOM NEF checks

Read SRM's failure detail

SRM records why an order or capability failed in its PostgreSQL database srm. With CORR set as in Trace a request:

# the order
kubectl exec -n oop postgres-0 -- psql -U postgres -d srm -c \
  "SELECT order_type, state, failure_detail FROM service_order WHERE correlation_id = '$CORR';"

# its capabilities, when SRM got that far
kubectl exec -n oop postgres-0 -- psql -U postgres -d srm -c \
  "SELECT ci.ref, ci.state, ci.failure_detail FROM capability_instance ci
   JOIN service_order so ON ci.originating_service_order_id = so.id
   WHERE so.correlation_id = '$CORR';"

General pod/Helm commands also live under Deploy with Helm › Troubleshooting.

Check it worked

The original symptom is gone. For a request-level issue, the same x-correlator shows a continuous path through the components you expect (or a clear failure at one hop, with failure_detail populated when SRM recorded one).

All options