Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add origin metadata to database items #3066

Merged
merged 4 commits into from
Nov 15, 2023
Merged

Conversation

koesie10
Copy link
Member

@koesie10 koesie10 commented Nov 13, 2023

This adds an origin property to the database item to store the source of the database, for example GitHub or an internet URL. This will be used to automatically check for updates to GitHub-downloaded databases in the future.

Checklist

  • CHANGELOG.md has been updated to incorporate all user visible changes made by this pull request.
  • Issues have been created for any UI or other user-facing changes made by this pull request.
  • [Maintainers only] If this pull request makes user-facing changes that require documentation changes, open a corresponding docs pull request in the github/codeql repo and add the ready-for-doc-review label there.

This adds a source property to the database item to store the source of
the database, for example GitHub or an internet URL. This will be used
to automatically check for updates to GitHub-downloaded databases in the
future.
@koesie10 koesie10 force-pushed the koesie10/add-database-source branch from 5d50d93 to bca7ecb Compare November 13, 2023 10:13
@koesie10 koesie10 changed the title Add source to database item Add source metadata to database items Nov 13, 2023
@koesie10 koesie10 marked this pull request as ready for review November 13, 2023 10:37
@koesie10 koesie10 requested review from a team as code owners November 13, 2023 10:37
Copy link
Contributor

@robertbrignull robertbrignull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of points I just want to clarify, but it seems ok.

@@ -88,6 +90,7 @@ export interface LocalDatabase {
name: string;
dateAdded: number;
language: string;
source: DatabaseSource;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the idea that this doesn't need to be an optional field because the UI currently doesn't allow you to add local databases to this config? This is the config for the new, combined databases panel, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. This type isn't actually used anywhere, but once we start using it, we should have the source for every database that is added. This is the same as the language and dateAdded, both of which are also optional in the FullDatabaseOptions, but required in this type.

return {
databaseUrl: `https://api.github.com/repos/${owner}/${repo}/code-scanning/codeql/databases/${language}`,
owner,
name: repo,
commitOid: databaseForLanguage.commit_oid,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth noting that this is the commit oid at the point of checking the database info and it might have changed by the point that we download the database contents. However the download should happen immediately and I'm not sure we can do better anyway since there isn't an API endpoint that returns both the metadata and the contents.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something we are aware of. The chance of this race condition happening is unlikely, but if it does, it's also not problematic. The only reason the commit_oid will be used is for checking updates to the database. If we run into this race condition, it's probably pretty likely that it's an active repository and we would have new commits coming in shortly after anyways.

@adityasharad
Copy link
Contributor

Passing suggestion: if you're concerned about the potential confusion around source or provenance, perhaps "origin"?

@koesie10
Copy link
Member Author

Passing suggestion: if you're concerned about the potential confusion around source or provenance, perhaps "origin"?

Thanks! I think that's a better name for this.

@koesie10 koesie10 changed the title Add source metadata to database items Add origin metadata to database items Nov 14, 2023
@koesie10 koesie10 requested a review from a team November 15, 2023 12:24
Copy link
Contributor

@robertbrignull robertbrignull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@koesie10 koesie10 merged commit 6fc567b into main Nov 15, 2023
@koesie10 koesie10 deleted the koesie10/add-database-source branch November 15, 2023 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants