[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib #2328

Electronic-Waste · 2024-05-15T03:07:15Z

What this PR does / why we need it:

This PR converts the GSoC Proposal into KEP, which describes the details of implementing "Project 6: Push-based Metrics Collection for Katib". There are some related issues discussing about this :

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Checklist:

Docs included if any changes are user facing

Signed-off-by: Electronic-Waste <[email protected]>

andreyvelich · 2024-05-24T19:36:34Z

/area gsoc

andreyvelich

Thank you for this @Electronic-Waste!
I left a few comments.
/assign @kubeflow/wg-training-leads

docs/proposals/push-based-metrics-collection.md

Electronic-Waste · 2024-06-05T03:23:08Z

ref issue: #2340

Signed-off-by: Electronic-Waste <[email protected]>

docs/proposals/push-based-metrics-collection.md

Signed-off-by: Electronic-Waste <[email protected]>

andreyvelich

Thank you @Electronic-Waste!
I think, we almost ready with this proposal, just a few small comments.
Please take a look @tenzen-y @johnugeorge

andreyvelich · 2024-06-14T13:29:26Z

docs/proposals/push-based-metrics-collection.md

+    # The newly added parameter metrics_collector_config.
+    # It specifies the config of metrics collector, for example, 
+    # metrics_collector_config={"kind": "Push"},
+    metrics_collector_config: Dict[str, Any] = None, 


Let's by default use the StdOut or Push metrics collector, so user doesn't need to configure it.

Okay, I'll use StdOut as the default metrics collector since Push metrics collector is under development.

docs/proposals/push-based-metrics-collection.md

Signed-off-by: Electronic-Waste <[email protected]>

Electronic-Waste · 2024-06-17T05:19:50Z

Sincere thanks for your time @andreyvelich . I modified the proposal according to your review.

Could you please review the updated version of the proposal? And We can discuss about it in the meeting today. If there are no extra reviews to make, I can convert the proposal to PRs in this week.

andreyvelich

Thank you for these updates @Electronic-Waste!
It looks good.
/lgtm
/assign @tenzen-y @johnugeorge

docs/proposals/push-based-metrics-collection.md

Signed-off-by: Electronic-Waste <[email protected]>

nsingl00 · 2024-06-24T17:28:40Z

docs/proposals/push-based-metrics-collection.md

+
+### New Parameter in Python SDK Function `tune`
+
+We decided to add `metrics_collection_mechanism` to `tune` function in Python SDK.


It should be metrics_collector_config right?

Yes, I forgot to change it. Thank you!

Signed-off-by: Electronic-Waste <[email protected]>

johnugeorge · 2024-06-28T20:12:37Z

/lgtm

andreyvelich

I think, we can merge this proposal.
We can create followup PRs if we need to modify it.
Thanks again for driving this @Electronic-Waste!
/lgtm
/approve

google-oss-prow · 2024-06-28T22:35:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [andreyvelich]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…beflow#2328) * doc: initial commit of gsoc proposal(project 6). Signed-off-by: Electronic-Waste <[email protected]> * doc: complete KEP for gsoc proposal(Project 6). Signed-off-by: Electronic-Waste <[email protected]> * chore: add non-goals and examples. Signed-off-by: Electronic-Waste <[email protected]> * chore: add . Signed-off-by: Electronic-Waste <[email protected]> * chore: add compatibility changes in trial controller. Signed-off-by: Electronic-Waste <[email protected]> * chore: update architecture figure. Signed-off-by: Electronic-Waste <[email protected]> * chore: update format. Signed-off-by: Electronic-Waste <[email protected]> * chore: update doc after the review in 10th, June. Signed-off-by: Electronic-Waste <[email protected]> * chore: add code link and remove namespace env variable. Signed-off-by: Electronic-Waste <[email protected]> * chore: modify proposal after the review in 14th, June. Signed-off-by: Electronic-Waste <[email protected]> * chore: delete WIP label. Signed-off-by: Electronic-Waste <[email protected]> * chore: add timeout param into report_metrics. Signed-off-by: Electronic-Waste <[email protected]> * fix: metrics_collector_config spelling. Signed-off-by: Electronic-Waste <[email protected]> --------- Signed-off-by: Electronic-Waste <[email protected]>

doc: initial commit of gsoc proposal(project 6).

ef09f05

Signed-off-by: Electronic-Waste <[email protected]>

google-oss-prow bot added the size/S label May 15, 2024

google-oss-prow bot requested review from andreyvelich, anencore94 and gaocegege May 15, 2024 03:07

doc: complete KEP for gsoc proposal(Project 6).

aff97bf

Signed-off-by: Electronic-Waste <[email protected]>

google-oss-prow bot added size/M and removed size/S labels May 15, 2024

google-oss-prow bot added the area/gsoc label May 24, 2024

Electronic-Waste mentioned this pull request Jun 3, 2024

[GSoC] Project 6: Push-based Metrics Collection for Katib #2340

Open

6 tasks

andreyvelich reviewed Jun 3, 2024

View reviewed changes

chore: add non-goals and examples.

7ece3ed

Signed-off-by: Electronic-Waste <[email protected]>

google-oss-prow bot added size/L and removed size/M labels Jun 5, 2024

Electronic-Waste requested a review from andreyvelich June 6, 2024 06:16

Electronic-Waste added 5 commits June 9, 2024 16:19

chore: add .

e923a33

Signed-off-by: Electronic-Waste <[email protected]>

chore: add compatibility changes in trial controller.

169857c

Signed-off-by: Electronic-Waste <[email protected]>

chore: update architecture figure.

0cfac1d

Signed-off-by: Electronic-Waste <[email protected]>

chore: update format.

6969dd7

Signed-off-by: Electronic-Waste <[email protected]>

chore: update doc after the review in 10th, June.

1a2599d

Signed-off-by: Electronic-Waste <[email protected]>

andreyvelich reviewed Jun 12, 2024

View reviewed changes

docs/proposals/push-based-metrics-collection.md Outdated Show resolved Hide resolved

docs/proposals/push-based-metrics-collection.md Outdated Show resolved Hide resolved

chore: add code link and remove namespace env variable.

d2439f2

Signed-off-by: Electronic-Waste <[email protected]>

Electronic-Waste requested a review from andreyvelich June 14, 2024 07:25

andreyvelich reviewed Jun 14, 2024

View reviewed changes

chore: modify proposal after the review in 14th, June.

3208e67

Signed-off-by: Electronic-Waste <[email protected]>

Electronic-Waste requested a review from andreyvelich June 17, 2024 05:15

andreyvelich reviewed Jun 17, 2024

View reviewed changes

docs/proposals/push-based-metrics-collection.md Outdated Show resolved Hide resolved

google-oss-prow bot assigned johnugeorge, tenzen-y and andreyvelich Jun 17, 2024

google-oss-prow bot added the lgtm label Jun 17, 2024

chore: delete WIP label.

cc95ef0

Signed-off-by: Electronic-Waste <[email protected]>

google-oss-prow bot removed the lgtm label Jun 18, 2024

Electronic-Waste requested a review from andreyvelich June 18, 2024 13:32

chore: add timeout param into report_metrics.

4161cff

Signed-off-by: Electronic-Waste <[email protected]>

nsingl00 reviewed Jun 24, 2024

View reviewed changes

fix: metrics_collector_config spelling.

c32a84a

Signed-off-by: Electronic-Waste <[email protected]>

google-oss-prow bot added the lgtm label Jun 28, 2024

andreyvelich reviewed Jun 28, 2024

View reviewed changes

google-oss-prow bot added the approved label Jun 28, 2024

google-oss-prow bot merged commit f06906d into kubeflow:master Jun 28, 2024
60 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib #2328

[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib #2328

Electronic-Waste commented May 15, 2024 •

edited

Loading

andreyvelich commented May 24, 2024

andreyvelich left a comment

Electronic-Waste commented Jun 5, 2024

andreyvelich left a comment

andreyvelich Jun 14, 2024

Electronic-Waste Jun 15, 2024

Electronic-Waste commented Jun 17, 2024

andreyvelich left a comment

nsingl00 Jun 24, 2024

Electronic-Waste Jun 25, 2024

johnugeorge commented Jun 28, 2024

andreyvelich left a comment

google-oss-prow bot commented Jun 28, 2024


		### New Parameter in Python SDK Function `tune`

		We decided to add `metrics_collection_mechanism` to `tune` function in Python SDK.

[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib #2328

[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib #2328

Conversation

Electronic-Waste commented May 15, 2024 • edited Loading

andreyvelich commented May 24, 2024

andreyvelich left a comment

Choose a reason for hiding this comment

Electronic-Waste commented Jun 5, 2024

andreyvelich left a comment

Choose a reason for hiding this comment

andreyvelich Jun 14, 2024

Choose a reason for hiding this comment

Electronic-Waste Jun 15, 2024

Choose a reason for hiding this comment

Electronic-Waste commented Jun 17, 2024

andreyvelich left a comment

Choose a reason for hiding this comment

nsingl00 Jun 24, 2024

Choose a reason for hiding this comment

Electronic-Waste Jun 25, 2024

Choose a reason for hiding this comment

johnugeorge commented Jun 28, 2024

andreyvelich left a comment

Choose a reason for hiding this comment

google-oss-prow bot commented Jun 28, 2024

Electronic-Waste commented May 15, 2024 •

edited

Loading