Publish a dataset in the Data Lab¶

In this tutorial you turn on the Data Lab, submit a dataset description to the staging catalogue as a data steward, promote it to production, and find it as an anonymous visitor.

The Data Lab is a catalogue: it describes datasets and links to them, and it does not store the data files. See Data Lab for how it works.

Before you start¶

You need:

• A running OpenOP on kind, deployed as in Deploy with Helm, with kubectl and helm pointing at it. Run the commands from the helm directory of that guide.
• The Keycloak password of the sample users (step 2).

The examples use the kind addresses, where the Data Lab UI is on http://localhost:30083.

1. Turn on the Data Lab¶

The Data Lab is off by default. Enable it together with its Keycloak realm dataspace; set both, because with only one of them nobody can log in or nothing uses the realm:

helm upgrade oop-platform ./oop-platform-chart -n oop --reuse-values \
  --set datalab.enabled=true --set keycloak.realms.dataspace.enabled=true

Keycloak restarts to import the realm, and a setup job creates two catalogues: oop-org (production) and oop-org-staging (submissions awaiting review). Wait until the Data Lab pods are ready:

kubectl -n oop get pods

2. Sign in as the steward¶

On kind, realm dataspace has three sample users: steward, alice and bob. They share one password:

kubectl -n oop get secret keycloak-credentials -o jsonpath='{.data.dev-user-password}' | base64 -d; echo

Open http://localhost:30083. The home page lists the catalogues and anyone can browse it without signing in. Choose Sign in, log in as steward, then open the management console at http://localhost:30083/#/admin.

A steward administers every catalogue. A normal user works in a catalogue they were added to, as a reader, contributor or publisher.

3. Submit a dataset to staging¶

1. In the console, open the catalogue oop-org-staging and choose New dataset.

2. Fill in the description. The form fields below are required:

   Field	Example
   Title	Edge latency traces
   Description	Round-trip times between UEs and an edge application.
   Issued	today's date, YYYY-MM-DD
   Access rights	PUBLIC
   Licence	CC-BY-4.0
   GDPR compliant, FAIR compliant	state true or false for each
   Distribution: title	Latency traces (CSV)
   Distribution: access URL	https://example.org/edge-latency/data
   Identifier	filled in for you from the dataset id

   The form proposes the dataset id from the title (edge-latency-traces). It can only contain a-z, 0-9 and -, and it cannot be changed later.

3. Save. The Data Lab API checks the record against the 6G-DALI Metadata Application Profile and shows an error next to any field that fails, without writing anything.

The record is written to the staging catalogue, which is private: only its members and stewards can see it.

4. Promote it to production¶

Staging holds submissions awaiting review. A steward moves a reviewed dataset to the public catalogue:

1. In oop-org-staging, open your dataset and choose Promote.
2. The console confirms that the dataset moved to oop-org.

The dataset leaves staging and appears in oop-org.

5. Find it as an anonymous visitor¶

Open a private browser window, which has no session, and go to http://localhost:30083/#/datasets. Search for latency. The dataset is listed, and its page shows the distribution and its access link. You can narrow the results with the facets on the left: catalogue, theme, format, licence, keyword and publisher.

The same search is available as an API through the UI's address:

curl "http://localhost:30083/api/public/datasets?q=latency"

The answer has a count, the matching items and the facets. Anonymous calls only ever return public catalogues and Public datasets.

What you learned¶

• How to turn on the Data Lab with its Keycloak realm.
• That the Data Lab holds descriptions of datasets, validated against the MAP, and not the data files.
• The two-step flow: submit to oop-org-staging, then a steward promotes to oop-org.
• That public search needs no login, and that private catalogues stay private.

Next steps¶

• Add a colleague to a catalogue as reader, contributor or publisher from the console, and sign in as alice to see what each level can do.
• Create a private catalogue for your own testbed with New catalogue.
• Read the Data Lab architecture page, and its known gaps before you rely on it: direct writes to the hub are not validated, and editing a dataset in the console loses fields the form does not show.
• To reset the Data Lab on kind, delete both its stores together (the Virtuoso and Elasticsearch volumes, Secret datalab-credentials and realm dataspace). See the Helm README, section Data Lab.