Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Create experiment, workflow, and tasks documentation from configuration, as well as metadata #1884

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kinow
Copy link
Member

@kinow kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:16

Closes #939

Creating a merge request to demonstrate the idea of the automated documentation of experiments. This is not super new, as you can achieve almost the same with Cylc and ecFlow. In Cylc you have DOC and TITLE too. In ecFlow they use MANUAL.

I am extending the scope of the issue (created 11 months ago) to add something that was discussed more recently, about the documentation for models, metadata, ...

The idea is documented in the docgen/__init__.py docstrings. Pasting it here:

Goes over experiment configuration, gathering each job's
``TITLE`` and ``DOC`` to create the documentation for the
jobs.

Goes over the ``METADATA`` configuration entry, looking
for sections with ``key=value`` pairs, or links to
configuration keys, e.g.:

.. code-block:: yaml

  METADATA:
    MODEL:
      # This is a static value that will produce name = IFS
      - name: name
      # This is a dynamic value that will produce resolution = abc...
      - resolution: %RUN.IFS.RESOLUTION%
      # This resolves the key and value automatically and will produce RUN.IFS.INIPATH=/path/path/...
      - %RUN.IFS.INIPATH%
      # This is the extended format
      - name: source
        value: https://git@...
        documentation: |
          The source code is hosted at the private repository...
    DATA:
      - PROVENANCE: ...


Each property of the ``METADATA`` YAML object is rendered as
a separate section. These sections, in YAML, are arrays that
contain metadata references in each line. Extra documentation
about each key can be entered using the extended format.

The gist is that the maintainers of the experiment would be able to select configuration values from the workflow that must be exported as workflow metadata (in the METADATA YAML object/entry above).

The new command in Autosubmit will take care to go over the experiment configuration, resolve variables like %EXPID%, %RUN.IFS.RESOLUTION.ETC%, and produce the metadata list, with the current values used in the experiment.

I think this is key here, to have the documentation generated with the values used in the workflow -- i.e. the workflow run configuration is the single source of truth, to avoid having a PDF or Wiki page manually written that might be out of sync with what's actually happening in the workflow.

@kinow kinow added this to the Autosubmit 4.1 milestone Dec 12, 2024
@kinow kinow self-assigned this Dec 12, 2024
@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:19

The current code is a draft and only produces:

$ autosubmit docgen a001
Autosubmit is running with 4.0.99

Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
Autosubmit general sections OK
Configuration files OK

# Workflow Jobs

These are the jobs in the workflow “a001
”.
## INI

**Initialize the experiment**
This task runs locally, the expid is %EXPID%

## SIM

## ASIM

## POST

# Metadata

This is valid Markdown, that could be used to generate a web page, PDF, docx, etc.. The contents of the ini job configuration:

  ini:
      FILE: 1.sh
      RUNNING: once
      TITLE: Initialize the experiment
      doc: This task runs locally, the expid is %EXPID%

Pasting the content below so GitLab can render the markdown (expanding the %EXPID% is easy but I will add that later):

Workflow Jobs

These are the jobs in the workflow “a001
”.

INI

Initialize the experiment
This task runs locally, the expid is %EXPID%

SIM

ASIM

POST

Metadata

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:20

/cc @mcastril , @GinkaVanThielen , @ainagaya , @Lerriola , @franra9 , @sebastianbeyer , @fdoblasreyes wrote a quick merge request just now to demonstrate the idea of automated documentation generation ☝️

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:20

Commented on autosubmit/docgen/init.py line 1

Note to self, update to 2024

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:22

And this doesn't need to replace any other documentation planned for models; this can be used to complement any documentation/wiki/deliverable/etc., but automating the parts that contain variables from the workflow.

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 19, 2024, 12:36

mentioned in issue digital-twins/de_340/project_management#579

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @mcastril on Jan 19, 2024, 19:09

This is wonderful Bruno. Can you clarify one doubt?

Goes over the ``METADATA`` configuration entry, looking
for sections with ``key=value`` pairs, or links to
configuration keys, e.g.:

Wouldn't it be key: value?

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @ainagaya on Jan 22, 2024, 09:55

That's great @kinow !!

So, if I understood correctly, what should be done in the workflow is to create a configuration file with the METADATA entries and fill them with placeholders? Sounds good to me.

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 22, 2024, 10:01

Yup! That's the initial idea. But we can work on it and decide what would be the best way for that. I think with a few more hours (2...3) I can get it to work and produce the final markdown.

With a working prototype we can test in the climatedt workflow or another example workflow, and then discuss what needs changing.

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @kinow on Jan 22, 2024, 10:33

Wouldn't it be key: value?

Ah, yes, so that it's valid YAML, good catch. Will fix that in the docs later. Thanks @mcastril !

@kinow
Copy link
Member Author

kinow commented Dec 12, 2024

In GitLab by @ainagaya on Jan 24, 2024, 13:07

mentioned in issue digital-twins/de_340/workflow#174

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[enhancement] Create experiment, workflow, and tasks documentation from Autosubmit configuration files
1 participant