Feature/4980 add json schema to submitted data of json dump plugin

[viktorvanwijk]

Closes #4980

Changes

Added JSON schema definition to the submitted data of the JSON dump registration plugin

Checklist

Check off the items that are completed or not relevant.

• Impact on features

  • Checked copying a form
  • Checked import/export of a form
  • Config checks in the configuration overview admin page
  • Problem detection in the admin email digest is handled

• Release management

  • I have labelled the PR as "needs-backport" accordingly

• I have updated the translations assets (you do NOT need to provide translations)

  • Ran ./bin/makemessages_js.sh
  • Ran ./bin/compilemessages_js.sh

• Dockerfile/scripts

  • Updated the Dockerfile with the necessary scripts from the ./bin folder

• Commit hygiene

  • Commit messages refer to the relevant Github issue
  • Commit messages explain the "why" of change, not the how

[codecov]

Codecov Report

Attention: Patch coverage is 99.36102% with 2 lines in your changes missing coverage. Please review.

Project coverage is 96.70%. Comparing base (8c29ae8) to head (54ceb6f).
Report is 20 commits behind head on master.

Files with missing lines	Patch %	Lines
...penforms/registrations/contrib/json_dump/plugin.py	96.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5007      +/-   ##
==========================================
+ Coverage   96.64%   96.70%   +0.05%     
==========================================
  Files         764      768       +4     
  Lines       26088    26409     +321     
  Branches     3406     3446      +40     
==========================================
+ Hits        25212    25538     +326     
+ Misses        610      607       -3     
+ Partials      266      264       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sergei-maertens]

First round, focusing on the architectural aspect and not so much on the actual schemas.

You should probably start from #5007 (comment)

[sergei-maertens]

openforms.variables.service.get_static_variables() is public API, but only returns the FormVariable instances and not the static variables around it that have more information.

On the other hand, looking at all possible flavours we have:

• Formio component -> FormVariable with source set to component (we can get more info from the component)
• User defined variable -> FormVariable with source set to user_defined (we can't get more info at all, the shittiest of all flavours)
• Static variables -> FormVariable that's only exists in-memory and not in the DB, but the implementation details are outside of it

I think it would be nice if we can define the schema generation fully in terms of the FormVariable class. It already has some methods/helpers to do some Formio specific stuff, so it's not weird to have it handle the first flavour too.

Roughly I'm thinking:

# easy to extend (future & probably registration variables) generator
def _iter_form_variables(form: Form) -> Iterator[FormVariable]:
    # static variables are always available
    yield from get_static_variables()
    # then handle all form variables holding dynamic data (component + user defined)
    yield from form.formvariable_set.all()

then you can consume that in your top-level function:

def generate_json_schema(...):
    requested_variable_schemas = {
        key: variable.as_json_schema()
        for variable in _iter_form_variables(form)
        if (key := variable.key) in limit_to_variables
    }

    ...  # process this with deep objects etc., the FormioData data structure might be useful here too

Using generators like this is quite elegant, since you decouple the "give me all the variables that can possibly exist for a form" from the actual schema generation and you can do a single dict comprehension instead of having to check limit_to_variables membership for every flavour of variable, which also lowers the chances of forgetting to perform such a check. And, if more flavours of variables come into existence in the future, then it's a matter of updating the generator above and we're set.

[sergei-maertens]

do we not send a value at all?

[viktorvanwijk]

'not filled' is not the best description. In the actual not-filled case, the value is an object with all the options listed as False. Here, I mean in the case it is hidden. Then it's an empty object, so I remove the required list.