Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape all the specfiles from Fedora for source URLs #2390

Closed
4 of 8 tasks
mfocko opened this issue Apr 4, 2024 · 3 comments · Fixed by packit/research#216
Closed
4 of 8 tasks

Scrape all the specfiles from Fedora for source URLs #2390

mfocko opened this issue Apr 4, 2024 · 3 comments · Fixed by packit/research#216
Assignees
Labels
area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. gain/high This brings a lot of value to (not strictly a lot of) users. impact/low This issue impacts only a few users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related.

Comments

@mfocko
Copy link
Member

mfocko commented Apr 4, 2024

To be able to adjust the firewall rules »beforehand« on the MP+, we need to know the domains/forges from which users are using sources for the sync-release actions.

Scraping the specfiles and configs of current users didn't provide enough benefit, since it can be still a pain point during an onboarding of new users

  • scrape all the specfiles
  • pinpoint forges/URLs that we do not support as of now
    • (optionally) ask the packagers if they'd be interested in onboarding
    • also check for activity within the packages
  • edit comment with the “waiting list” in Firewall rules that need to be enabled deployment#575
  • (optionally) open a ticket to adjust the firewall rules
    • (“optional”) ticket per URL :)
  • (optionally) verify that the sources can be obtained
@mfocko mfocko added area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. impact/low This issue impacts only a few users. gain/high This brings a lot of value to (not strictly a lot of) users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related. labels Apr 4, 2024
@xsuchy xsuchy self-assigned this Apr 4, 2024
@xsuchy
Copy link
Contributor

xsuchy commented Apr 4, 2024

I am running:

for i in *spec; do
spectool $i |grep Source
done >/tmp/output.txt

@xsuchy
Copy link
Contributor

xsuchy commented Apr 4, 2024

Here is the list of all Sources from Fedora:
output.txt.gz

@xsuchy xsuchy removed their assignment Apr 4, 2024
@mfocko mfocko moved this from new to priority-backlog in Packit Kanban Board Apr 8, 2024
@nforro
Copy link
Member

nforro commented May 20, 2024

  • scrape all the specfiles

Maybe we should go also through Packit configs, because (at least in theory) upstream_project_url can be blocked while Source0 URL is fine.

@lbarcziova lbarcziova moved this from priority-backlog to refined in Packit Kanban Board May 23, 2024
@mfocko mfocko self-assigned this Jul 8, 2024
@mfocko mfocko moved this from refined to in-progress in Packit Kanban Board Jul 8, 2024
@mfocko mfocko added the blocked We are blocked! label Jul 22, 2024
@mfocko mfocko removed the blocked We are blocked! label Aug 12, 2024
@mfocko mfocko moved this from in-progress to in-review in Packit Kanban Board Aug 14, 2024
mfocko added a commit to mfocko/research that referenced this issue Aug 15, 2024
github-merge-queue bot pushed a commit to packit/research that referenced this issue Aug 27, 2024
@github-project-automation github-project-automation bot moved this from in-review to done in Packit Kanban Board Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. gain/high This brings a lot of value to (not strictly a lot of) users. impact/low This issue impacts only a few users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants