Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically create dataset JSON schemas through introspection #964

Open
chrisschopp opened this issue Dec 12, 2024 · 1 comment
Open
Labels
enhancement New feature or request help wanted Contribution task, outside help would be appreciated!

Comments

@chrisschopp
Copy link
Contributor

Description

When a new dataset is added to kedro-datasets, the JSON schema that enables Kedro catalog validation in an IDE must be manually updated.

These schemas could be created programmatically through introspection (as suggested by @datajoely).

Context

Manually creating a JSON schemas is an additional maintenance step (that may be easy to overlook).

Possible Implementation

  1. The inspect module could be used to get the dataset's attributes.
  2. A pre-commit hook could generate the required kedro-catalog-X.XX.json (only for the current package version).
    a. When a developer commits a new dataset, the JSON schema is created for them.
  3. Remove references to updating jsonschema from CONTRIBUTING.md and PULL_REQUEST_TEMPLATE.md.

Possible Alternatives

The introspection code could be triggered manually instead of by a pre-commit hook. The hook would fully automate it, though.

@merelcht merelcht added the Community Issue/PR opened by the open-source community label Dec 12, 2024
@github-project-automation github-project-automation bot moved this to Wizard inbox in Kedro Wizard 🪄 Dec 12, 2024
@merelcht merelcht added the enhancement New feature or request label Dec 12, 2024
@merelcht merelcht removed the Community Issue/PR opened by the open-source community label Dec 12, 2024
@astrojuanlu
Copy link
Member

This would be nice to have for the Kedro team, but it's not something we'll be doing in the short term. If someone wants to open a PR, go for it!

@astrojuanlu astrojuanlu added the help wanted Contribution task, outside help would be appreciated! label Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Contribution task, outside help would be appreciated!
Projects
None yet
Development

No branches or pull requests

3 participants