Deploy with Ansible¶

Deploy one OpenOP platform per operator, using existing Kubernetes clusters or Ubuntu VMs on which Ansible first installs k3s. With multiple operators, a separate playbook configures partner identities, agreements and federation.

What you end up with¶

Ansible runs Helm on the operator host to install the platform. SRM deploys application workloads into the oop-apps namespace.

flowchart LR
  ctl["Ansible controller"]
  host["Operator host<br/>Helm · kubectl · kubeconfig"]
  subgraph cluster["Operator Kubernetes cluster"]
    subgraph ns["Namespace oop · Helm release oop"]
      public["Portal · OEG · FM"]
      srm["SRM"]
      deps["Keycloak · PostgreSQL · NATS"]
    end
    apps["Namespace oop-apps<br/>Application workloads"]
  end
  ctl -->|SSH| host
  host -->|"helm upgrade --install"| ns
  ctl -. "Public endpoint checks" .-> public
  srm -->|"Helm / Kubernetes API"| apps

Each FM obtains a client-credentials token from the partner’s Keycloak and uses it to call the partner’s FM.

flowchart LR
  subgraph A["operator-a"]
    fmA["FM"]
    kcA["Keycloak · federation realm<br/>Client: operator-b"]
  end
  subgraph B["operator-b"]
    fmB["FM"]
    kcB["Keycloak · federation realm<br/>Client: operator-a"]
  end
  fmA -->|"Get token as operator-a"| kcB
  fmB -->|"Get token as operator-b"| kcA
  fmA -->|"OPG.04 EWBI requests"| fmB
  fmB -->|"OPG.04 EWBI requests"| fmA

Before you start¶

On the controller (the machine you run Ansible from):

• Python 3.12 or newer, Git, and SSH access to each remote operator host.
• network access to each operator's public platform endpoints for the deployment checks and end-to-end tests.
• For the end-to-end tests, kubectl and a kubeconfig for each operator's cluster.

For each operator:

• A Kubernetes cluster and, on the host Ansible connects to, kubectl, Helm, the Python Kubernetes client and a kubeconfig for that cluster. The optional k3s playbook installs these tools and configures the kubeconfig.
• A public address (oop_platform_public_host) reachable by users, the controller and partner operators. With the default configuration, allow access to Keycloak 30081, Portal 30082 when enabled, OEG 32263 and FM 30989.
• Resources for a complete platform, as listed in the Helm guide.

The example uses HTTP and NodePorts on a test network. Configure TLS, DNS and access to the container registry to suit your deployment environment.

Steps¶

1. Get the playbooks¶

git clone https://labs.etsi.org/rep/oop/deploy/ansible.git
cd ansible
git checkout v2.0.0-release
cd ansible

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
ansible-galaxy collection install -r requirements.yml

Every command below runs from this ansible/ansible folder, with the virtual environment active.

2. Write the inventory¶

Copy the example; *.local.yml files are ignored by Git:

cp inventory/r2.example.yml inventory/r2.local.yml

Each operator is one host in the group oop_platform_operators. Replace the example addresses and set ansible_user in the group variables to your SSH user. ansible_host is the address used for SSH; oop_platform_public_host is the address clients and partner operators use to reach the platform.

all:
  children:
    oop_platform_operators:
      vars:
        ansible_user: ubuntu
        oop_platform_chart_version: v2.0.0-release
        oop_platform_fm_image_tag: "2.0.0"
        oop_platform_oeg_image_tag: "2.0.0"
        oop_platform_srm_image_tag: "2.0.0"
        oop_platform_portal_enabled: true
        oop_platform_portal_image_tag: "2.0.0"
        oop_platform_allow_insecure_partner_endpoints: true   # plain HTTP between operators
        oop_platform_extra_values:          # lets SRM deploy apps into oop-apps
          srm:
            workloads:
              namespaces: [oop-apps]
        # ... plus oop_platform_partner_agreement and oop_platform_bootstrap,
        #     copied unchanged from r2.example.yml
      hosts:
        operator_a:
          ansible_host: 192.0.2.10
          oop_platform_public_host: 192.0.2.10
          oop_platform_federation_id: operator-a
          oop_platform_country_code: ES
          oop_platform_mcc: "214"
          oop_platform_mncs: ["07"]
          oop_platform_federation_clients: [operator-b]
          srm_zone_name: Barcelona
        operator_b:
          ansible_host: 192.0.2.20
          oop_platform_public_host: 192.0.2.20
          oop_platform_federation_id: operator-b
          oop_platform_country_code: FR
          oop_platform_mcc: "208"
          oop_platform_mncs: ["01"]
          oop_platform_federation_clients: [operator-a]
          srm_zone_name: Paris

What matters most:

• Versions. Set the chart and the four image tags as above. The example file in the v2.0.0-release tag still carries the pre-release values (a chart commit, ci-develop-* images, and the portal turned off), so don't keep them. For an API-only deployment, set oop_platform_portal_enabled: false and follow the API instructions in the federation tutorial.
• Federation identity. oop_platform_federation_id identifies this operator to its partners and is the Keycloak client ID those partners create for it. oop_platform_federation_clients lists partners accepted by this operator. For federation, configure these lists reciprocally.
• Zones. The example’s oop_platform_bootstrap declares one zone per operator, named by srm_zone_name, with applications running in namespace oop-apps on that operator’s cluster. oop_platform_partner_agreement defines the zones and resource terms offered to partners, including the available resource profiles (flavours).
• Application endpoints. In the Kubernetes control path's infrastructure, set endpoint_host to the address application clients use to reach workload NodePorts. For the sample in-cluster backend, use endpoint_host: "{{ oop_platform_public_host }}" when that public address reaches the workload ports. See Kubernetes for local and remote connection settings.
• Plain Ubuntu VMs. Also add the hosts to the group k3s_nodes. A host whose k3s_server names another host joins that one as an agent; the commented block at the end of r2.example.yml shows how.
• Secrets. When secret variables are unset, the chart generates the values and stores them in the cluster. For deployments you need to preserve, back up the generated credentials securely or provide your own through Ansible Vault before the first installation.

3. Install k3s, only if needed¶

ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/deploy.yml

The playbook installs k3s on the hosts in k3s_nodes, waits for the nodes to become Ready, and prepares the server hosts with a kubeconfig and the tools needed to deploy OpenOP. Skip this step when using existing Kubernetes clusters.

4. Deploy one platform per operator¶

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/deploy.yml

For each operator, the role validates the inventory, checks the FM partner and OEG callback encryption keys against any values supplied through Vault, fetches the selected Helm chart and installs or upgrades the platform. It then checks readiness and the public endpoints from the controller, and prints a deployment summary.

The run ends with one summary per operator:

OOP Release 2 is up on operator_a (namespace oop).
  Portal:           http://192.0.2.10:30082
  OEG:              http://192.0.2.10:32263
  Federation (FM):  http://192.0.2.10:30989
  Partner tokens:   http://192.0.2.10:30081/realms/federation
Generated in the cluster and stored nowhere else:
  - Federation Manager partner secret key (federation-manager-keys)
  - OEG callback key (oegcontroller-keys)
  - database passwords (postgres-credentials)
  - Keycloak admin password (keycloak-credentials)

Re-run the same command to apply inventory changes to an existing deployment. See Known issues when changing the chart version.

5. Federate the operators¶

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/federate.yml

The playbook needs access to all participating operators in the inventory. For each pair, it:

1. Retrieves the partner client credentials from Keycloak.
2. Registers each operator as a partner in the other’s FM.
3. Applies the agreements defined by oop_platform_partner_agreement.
4. Creates the federation contexts using OPG.04 CreateFederation.

If an agreement was edited in the Portal, the next run restores the inventory’s terms.

The playbook does not subscribe to the partner’s zones, and applications can’t be deployed in a partner zone until you do. Subscribe in each operator’s Portal, or through the API, as in Federate two operators › Subscribe to the partner’s zone. The same tutorial shows every step by hand.

Check it worked¶

The tests create temporary Keycloak clients on each platform and remove them after the run.

pip install -r tests/r2_e2e/requirements.txt
ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/e2e.yml

Pass extra pytest arguments with -e '{"e2e_pytest_args":"-k federation"}', for example to select federation-related tests.

The release tag still marks test_apps_are_private_to_their_tenant as an expected failure. OEG 2.0.0 now passes that read/delete isolation check, so pytest reports XPASS(strict) and fails the run. See Known issues for the workaround.

First, check federation status in each operator’s Portal. Sign in with the operator role and open Federation. Each configured partner should appear as active and Federated.

Then, check the platform services from each operator host. Use the kubeconfig and platform namespace configured in that operator’s inventory entry:

kubectl --kubeconfig OPERATOR_KUBECONFIG -n oop get pods
curl --fail-with-body -sS http://OPERATOR_PUBLIC_HOST:30989/healthz

Replace the placeholders with that operator’s inventory values; change oop if its platform namespace differs. Expect the platform pods to be ready and the FM health response to be {"status":"ok"}.

Remove it¶

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/undeploy.yml

This runs helm uninstall and waits for completion. The chart retains its generated Secrets and volumes, allowing a later deployment to reuse the stored data.

To remove the retained Secrets and PVCs as well, delete the platform namespace after uninstalling the release.

If k3s was installed for this setup, you can also remove it with:

ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/undeploy.yml -e k3s_confirm_uninstall=true

This removes k3s, its workloads and locally stored cluster data from those machines.

Known issues¶

Changing the chart version. Deployment may fail at Fetch the chart because of the cached shallow checkout. Remove the cached Helm repository on the affected operator host and rerun deploy.yml. Its default location is ~/.cache/oop-platform/<inventory-host>/helm.

A test reports XPASS(strict). For v2.0.0-release, run the remaining tests while excluding the case with the outdated expected-failure mark:

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/e2e.yml \
  -e '{"e2e_pytest_argv":["-k","not test_apps_are_private_to_their_tenant"]}'

All options¶

See the scenario README for the full inventory reference, Secrets and Vault variables, agreement settings and partners outside the inventory (oop_platform_partners). Variable defaults are in roles/oop-platform/defaults/main.yml.

Use oop_platform_extra_values for additional chart settings. Configure topology and control-path credentials through oop_platform_bootstrap, as described in Define zones and Kubernetes integration.