Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support secrets in tool requirements #19084

Draft
wants to merge 91 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
946469c
init add secrets to tools
arash77 Oct 29, 2024
6af870e
add secret requirement in tools schema
arash77 Oct 30, 2024
25e293d
check the required field with user_preferences_extra
arash77 Oct 30, 2024
f0a5459
validate secret type and store for tool interface in user preferences
arash77 Oct 30, 2024
4ade06a
Add secrets into tools
arash77 Oct 30, 2024
8d47dc3
Add tests for secrets in tools
arash77 Oct 30, 2024
b52a907
Avoid log vault_key
arash77 Oct 31, 2024
f9541ec
fix typo
arash77 Oct 31, 2024
1623763
cast app for using vault into StructuredApp
arash77 Oct 31, 2024
b7c1eb8
Add secrets parameter to parse_requirements_and_containers method in …
arash77 Oct 31, 2024
924ede4
add secrets into cwl and yml
arash77 Oct 31, 2024
1aefc45
Fix tool parsing test to get secrets
arash77 Oct 31, 2024
fd330ee
Fix tool tests to include secrets
arash77 Oct 31, 2024
10e3673
Rename 'secrets' to 'credentials' in tool parsing
arash77 Nov 28, 2024
9e5c140
Refactor test cases to remove unused TestSecretsInExtraUserPreference…
arash77 Dec 2, 2024
e5e0186
updating the credentials to the new format
arash77 Dec 3, 2024
6076634
Refactor credential classes (Variable and Secret)
arash77 Dec 3, 2024
ad7a332
user credential model
arash77 Dec 6, 2024
2d1d746
Add API and schema for user credentials management
arash77 Dec 6, 2024
67188ba
Remove unused Union import from credentials service
arash77 Dec 6, 2024
660ebd7
Add basic ToolCredentials component and related interfaces for managi…
davelopez Dec 5, 2024
89c211e
Refactor ToolCredentials component
davelopez Dec 5, 2024
ea5e437
Add badges to indicate optional and required credentials in Credentia…
davelopez Dec 5, 2024
d848cb8
Add WIP user credentials store
davelopez Dec 5, 2024
5bd5d5f
Refactor ToolCredentials component to enhance user messaging
davelopez Dec 5, 2024
e447c9d
Initialize credentials in ManageToolCredentials to use a copy
davelopez Dec 5, 2024
cda08de
Refactor credential validation logic
davelopez Dec 5, 2024
9062009
update OpenAPI schema for credentials endpoints
arash77 Dec 9, 2024
c183c51
Add update credentials API and payload models
arash77 Dec 9, 2024
fa00530
Refactor credentials API schema
arash77 Dec 9, 2024
3479824
Enhance ToolEvaluator to read secrets from UserVaultWrapper and query…
arash77 Dec 9, 2024
1b3ccba
Add new models for user credentials and tool credentials management
davelopez Dec 9, 2024
e6e7c71
fixing model name
arash77 Dec 16, 2024
2a1d3e0
updating models for user credentials
arash77 Dec 16, 2024
4ba9518
partial update for credentials api to match the new changes
arash77 Dec 16, 2024
1c25f12
fix linting and schema
arash77 Dec 16, 2024
f1f97c3
update credentials api
arash77 Dec 16, 2024
230d2e3
update credentials models
arash77 Dec 16, 2024
1660e4c
update schema
arash77 Dec 16, 2024
7a29a03
fix linting
arash77 Dec 16, 2024
9b7e435
update
arash77 Dec 18, 2024
daa8a5d
refactor credentials API and schema
arash77 Dec 18, 2024
8e8d7a6
refactor user credentials model and introduce variable and secret model
arash77 Dec 18, 2024
0adafe3
add user credentials, user credentials group,
arash77 Dec 19, 2024
6191e5c
update openapi schema
arash77 Dec 19, 2024
c0fe98d
rename variable and secret tables to credential_variable and credenti…
arash77 Dec 19, 2024
36b3e88
add credentials attribute to MockTool class for enhanced testing
arash77 Dec 19, 2024
7f5f52d
rename parse_requirements_and_containers to parse_requirements,
arash77 Dec 20, 2024
ee215e2
remove vault in evaluation and MinimalToolApp
arash77 Dec 20, 2024
1aafdef
Fix credentials access control in API
davelopez Dec 20, 2024
e911c31
Refactor user credentials store
davelopez Dec 20, 2024
3992795
Refactor ToolCredentials and related UI components
davelopez Dec 20, 2024
d3c0963
Update user_id type to support "current" in credentials
arash77 Jan 2, 2025
f49624b
Fix user credentials retrieval to handle None case and ensure proper …
arash77 Jan 2, 2025
29f27d7
Add integration tests for user credentials API
arash77 Jan 2, 2025
b585802
Fix API endpoint string formatting in integration tests for user cred…
arash77 Jan 2, 2025
c36ed04
Add ondelete cascade to user credential group foreign keys and update…
arash77 Jan 3, 2025
fca65b8
Refactor credential deletion logic to use lists instead of sets and a…
arash77 Jan 3, 2025
8add0ac
Add ondelete cascade to foreign keys in user credentials and credenti…
arash77 Jan 3, 2025
2a6c755
Draft: Implement UserCredentialsConfigurator to manage environment va…
arash77 Jan 9, 2025
a9d70fe
Refactor UserCredentialsConfigurator to handle multiple results
arash77 Jan 10, 2025
24a19f1
Refactor CredentialsService to improve credential handling and deleti…
arash77 Jan 14, 2025
3ae8b8a
Enhance test coverage for credential management
arash77 Jan 14, 2025
3779f8b
Refactor ToolEvaluator
arash77 Jan 14, 2025
2737d90
Refactor CredentialsService to skip None values for variables and sec…
arash77 Jan 14, 2025
c5672bd
Fix v-model bindings in ServiceCredentials component for variable and…
davelopez Jan 14, 2025
134df63
Add secret placeholder handling in ManageToolCredentials and userCred…
davelopez Jan 14, 2025
e07d2fd
Try to avoid password manager autocompletion
davelopez Jan 14, 2025
c065095
Improve variable and secret handling in ManageToolCredentials
davelopez Jan 14, 2025
2cc833d
Fix new group not selected on creation by default
davelopez Jan 14, 2025
107b2d8
Enhance manage new credential sets
davelopez Jan 14, 2025
1027e26
Fix bug when updating user credentials to get the updated one
arash77 Jan 15, 2025
5e57c72
Add integration test for adding a new group to user credentials
arash77 Jan 15, 2025
e68721f
Refactoring CredentialsService and Add CredentialsManager for seperat…
arash77 Jan 15, 2025
9a03b09
Fix parameter naming in delete_credentials method calls for clarity
arash77 Jan 15, 2025
a10449c
Remove group_name parameter from user credentials API and related met…
arash77 Jan 15, 2025
d9e6bcf
Enhance credential management interface
davelopez Jan 15, 2025
a6fbb26
Add functionality to delete credential groups in UI
davelopez Jan 15, 2025
29feae1
Introduces update_current_group method for better group management
arash77 Jan 15, 2025
32c83f6
Refactor integration tests for user credentials
arash77 Jan 15, 2025
9f4d622
Refactor + add more integration tests for user credential management
davelopez Jan 16, 2025
c7e0f45
Refactors credentials management methods to only include db connectio…
arash77 Jan 16, 2025
6c26250
Add tool_id attribute to MockTool class to fix the test
arash77 Jan 20, 2025
2e5953a
Adds session instance to CredentialsManager constructor
arash77 Jan 20, 2025
904202d
Add unit tests for CredentialsManager functionality
arash77 Jan 20, 2025
286a865
Add test to ensure anonymous users cannot provide credentials
davelopez Jan 20, 2025
18282cc
Add test to ensure other users cannot list credentials
davelopez Jan 20, 2025
c8350bb
Refactors credential management and improves type annotations
arash77 Jan 21, 2025
402e14c
Refactor test_CredentialsManager to improve type annotations for rows…
arash77 Jan 21, 2025
398e969
Refactor credentials management to improve type annotations and conso…
arash77 Jan 21, 2025
c33b5c1
Refactor test_CredentialsManager to consolidate imports
arash77 Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
344 changes: 344 additions & 0 deletions client/src/api/schema/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4692,6 +4692,58 @@ export interface paths {
patch?: never;
trace?: never;
};
"/api/users/{user_id}/credentials": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/** Lists all credentials the user has provided */
get: operations["list_user_credentials_api_users__user_id__credentials_get"];
put?: never;
/** Allows users to provide credentials for a secret/variable */
post: operations["provide_credential_api_users__user_id__credentials_post"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/api/users/{user_id}/credentials/{user_credentials_id}": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
post?: never;
/** Deletes all credentials for a specific service */
delete: operations["delete_service_credentials_api_users__user_id__credentials__user_credentials_id__delete"];
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/api/users/{user_id}/credentials/{user_credentials_id}/{group_id}": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
post?: never;
/** Deletes a specific credential */
delete: operations["delete_credentials_api_users__user_id__credentials__user_credentials_id___group_id__delete"];
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/api/users/{user_id}/custom_builds": {
parameters: {
query?: never;
Expand Down Expand Up @@ -7288,6 +7340,18 @@ export interface components {
*/
url: string;
};
/** CreateSourceCredentialsPayload */
CreateSourceCredentialsPayload: {
/** Credentials */
credentials: components["schemas"]["ServiceCredentialPayload"][];
/** Source Id */
source_id: string;
/**
* Source Type
* @constant
*/
source_type: "tool";
};
/**
* CreateType
* @enum {string}
Expand Down Expand Up @@ -7383,6 +7447,27 @@ export interface components {
*/
username: string;
};
/** CredentialGroupResponse */
CredentialGroupResponse: {
/**
* Id
* @example 0123456789ABCDEF
*/
id: string;
/** Name */
name: string;
/** Secrets */
secrets: components["schemas"]["SecretResponse"][];
/** Variables */
variables: components["schemas"]["VariableResponse"][];
};
/** CredentialPayload */
CredentialPayload: {
/** Name */
name: string;
/** Value */
value: string | null;
};
/** CustomArchivedHistoryView */
CustomArchivedHistoryView: {
/**
Expand Down Expand Up @@ -15687,6 +15772,18 @@ export interface components {
} & {
[key: string]: unknown;
};
/** SecretResponse */
SecretResponse: {
/** Already Set */
already_set: boolean;
/**
* Id
* @example 0123456789ABCDEF
*/
id: string;
/** Name */
name: string;
};
/** ServerDirElement */
ServerDirElement: {
/** Md5 */
Expand Down Expand Up @@ -15805,6 +15902,27 @@ export interface components {
*/
version: string;
};
/** ServiceCredentialPayload */
ServiceCredentialPayload: {
/**
* Current Group
* @default default
*/
current_group: string | null;
/** Groups */
groups: components["schemas"]["ServiceGroupPayload"][];
/** Reference */
reference: string;
};
/** ServiceGroupPayload */
ServiceGroupPayload: {
/** Name */
name: string;
/** Secrets */
secrets: components["schemas"]["CredentialPayload"][];
/** Variables */
variables: components["schemas"]["CredentialPayload"][];
};
/** ServiceType */
ServiceType: {
/**
Expand Down Expand Up @@ -17499,6 +17617,36 @@ export interface components {
*/
username: string;
};
/** UserCredentialsListResponse */
UserCredentialsListResponse: components["schemas"]["UserCredentialsResponse"][];
/** UserCredentialsResponse */
UserCredentialsResponse: {
/** Current Group Name */
current_group_name: string;
/** Groups */
groups: {
[key: string]: components["schemas"]["CredentialGroupResponse"];
};
/**
* Id
* @example 0123456789ABCDEF
*/
id: string;
/** Reference */
reference: string;
/** Source Id */
source_id: string;
/**
* Source Type
* @constant
*/
source_type: "tool";
/**
* User Id
* @example 0123456789ABCDEF
*/
user_id: string;
};
/** UserDeletionPayload */
UserDeletionPayload: {
/**
Expand Down Expand Up @@ -17767,6 +17915,18 @@ export interface components {
*/
username?: string | null;
};
/** VariableResponse */
VariableResponse: {
/**
* Id
* @example 0123456789ABCDEF
*/
id: string;
/** Name */
name: string;
/** Value */
value: string | null;
};
/** Visualization */
Visualization: Record<string, never>;
/** VisualizationCreatePayload */
Expand Down Expand Up @@ -33715,6 +33875,190 @@ export interface operations {
};
};
};
list_user_credentials_api_users__user_id__credentials_get: {
parameters: {
query?: {
/** @description The type of source to filter by. */
source_type?: "tool" | null;
/** @description The ID of the source to filter by. */
source_id?: string | null;
};
header?: {
/** @description The user ID that will be used to effectively make this API call. Only admins and designated users can make API calls on behalf of other users. */
"run-as"?: string | null;
};
path: {
user_id: string | "current";
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["UserCredentialsListResponse"];
};
};
/** @description Request Error */
"4XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
/** @description Server Error */
"5XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
};
};
provide_credential_api_users__user_id__credentials_post: {
parameters: {
query?: never;
header?: {
/** @description The user ID that will be used to effectively make this API call. Only admins and designated users can make API calls on behalf of other users. */
"run-as"?: string | null;
};
path: {
user_id: string | "current";
};
cookie?: never;
};
requestBody: {
content: {
"application/json": components["schemas"]["CreateSourceCredentialsPayload"];
};
};
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["UserCredentialsListResponse"];
};
};
/** @description Request Error */
"4XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
/** @description Server Error */
"5XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
};
};
delete_service_credentials_api_users__user_id__credentials__user_credentials_id__delete: {
parameters: {
query?: never;
header?: {
/** @description The user ID that will be used to effectively make this API call. Only admins and designated users can make API calls on behalf of other users. */
"run-as"?: string | null;
};
path: {
user_id: string | "current";
user_credentials_id: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Request Error */
"4XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
/** @description Server Error */
"5XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
};
};
delete_credentials_api_users__user_id__credentials__user_credentials_id___group_id__delete: {
parameters: {
query?: never;
header?: {
/** @description The user ID that will be used to effectively make this API call. Only admins and designated users can make API calls on behalf of other users. */
"run-as"?: string | null;
};
path: {
user_id: string | "current";
user_credentials_id: string;
group_id: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
/** @description Request Error */
"4XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
/** @description Server Error */
"5XX": {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MessageExceptionModel"];
};
};
};
};
get_custom_builds_api_users__user_id__custom_builds_get: {
parameters: {
query?: never;
Expand Down
Loading
Loading