Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a list of downstream repos, and script to update #1304

Merged
merged 63 commits into from
Jun 13, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
fdf91da
Add a list of downstream repos, and script to update
MichaelChirico May 27, 2022
71e1e44
readability
MichaelChirico May 27, 2022
03274b1
more readability
MichaelChirico May 27, 2022
6e9fce1
switch to read from DCF; improve regex; improve matching
MichaelChirico May 29, 2022
10a3d5c
Merge branch 'main' into rev-urls
MichaelChirico May 29, 2022
51cbff2
move to .dev
MichaelChirico May 29, 2022
8c146ac
Merge branch 'rev-urls' of github.com:r-lib/lintr into rev-urls
MichaelChirico May 29, 2022
2e6916d
use regexpr, also save repo-less packages
MichaelChirico May 29, 2022
9f3fbbc
<- for assignment
MichaelChirico May 29, 2022
f9b43c0
debug
MichaelChirico May 29, 2022
d53fc2f
output of new approach
MichaelChirico May 30, 2022
d18b748
sort no-repos output
MichaelChirico May 30, 2022
5b4403d
Merge branch 'main' of github.com:r-lib/lintr into rev-urls
MichaelChirico May 30, 2022
9de86f0
use BugReports as a backup
MichaelChirico May 30, 2022
30249f5
output of including BugReports (+11 hits)
MichaelChirico May 30, 2022
5edd089
better naming
MichaelChirico May 30, 2022
b4bb0cb
stub of file tracking manual repos
MichaelChirico May 30, 2022
6624268
include maintainers alongside unmatched packages
MichaelChirico May 30, 2022
cb57789
s for consistency
MichaelChirico Jun 1, 2022
a33e806
use revdep as a prefix for related files
MichaelChirico Jun 1, 2022
800efca
add commented-out .gitignore for editing in web later
MichaelChirico Jun 1, 2022
552fc71
first version of script to compare versions
MichaelChirico Jun 1, 2022
c9fb6ef
executable
MichaelChirico Jun 1, 2022
f779490
fix regex for commas_linter not to hang on certain files
MichaelChirico Jun 2, 2022
3cca886
Merge branch 'main' into commas-performance
AshesITR Jun 2, 2022
a513853
Merge branch 'main' into commas-performance
MichaelChirico Jun 2, 2022
5f52f9f
fix paren_body_linter as well
MichaelChirico Jun 2, 2022
18160ab
Merge branch 'commas-performance' of github.com:jimhester/lintr into …
MichaelChirico Jun 2, 2022
3157bb9
Merge branch 'main' into rev-urls
MichaelChirico Jun 2, 2022
c7b2af0
Merge branch 'commas-performance' into rev-urls
MichaelChirico Jun 2, 2022
f71978c
Merge branch 'main' into rev-urls
MichaelChirico Jun 4, 2022
0c8a5ed
add timings, start to add analysis
MichaelChirico Jun 4, 2022
64f93d1
flow for installing required packages
MichaelChirico Jun 5, 2022
3cb0982
new list of packages
MichaelChirico Jun 5, 2022
1eb6115
debugging
MichaelChirico Jun 5, 2022
1bf0085
better state reset
MichaelChirico Jun 5, 2022
553743b
first pass at post-analysis insights
MichaelChirico Jun 5, 2022
7ee7f8e
prepare handling of CRAN-only package
MichaelChirico Jun 5, 2022
8dab66a
use CSV to keep package<->repo mapping
MichaelChirico Jun 5, 2022
3b05784
incorporate metadata table used to store all metadata
MichaelChirico Jun 5, 2022
e4e9081
add a first pass at an e-mail template
MichaelChirico Jun 5, 2022
b6b7b40
first pass at generating e-mails
MichaelChirico Jun 5, 2022
676244a
add e-mails (to be reverted)
MichaelChirico Jun 5, 2022
89bbb29
tweak printing order
MichaelChirico Jun 5, 2022
e1ead96
robust shebang usage
MichaelChirico Jun 5, 2022
5230284
reset repo state for fresh run from scratch
MichaelChirico Jun 5, 2022
0c35544
must supply repos= in batch mode
MichaelChirico Jun 5, 2022
a7d682a
fresh run
MichaelChirico Jun 5, 2022
0685bb4
improve display
MichaelChirico Jun 5, 2022
cafdd9e
adding back template e-mails after a fresh run
MichaelChirico Jun 5, 2022
67a4240
add arrow
MichaelChirico Jun 6, 2022
8535d28
fine-tune output, add comment
MichaelChirico Jun 6, 2022
9f98544
Merge branch 'main' into rev-urls
MichaelChirico Jun 6, 2022
5d8d42e
Merge branch 'main' into rev-urls
MichaelChirico Jun 7, 2022
87cc073
add shims for deleted functions
MichaelChirico Jun 7, 2022
e138be9
Merge branch 'main' into rev-urls
MichaelChirico Jun 10, 2022
cf568c3
Merge remote-tracking branch 'origin/rev-urls' into rev-urls
MichaelChirico Jun 10, 2022
c3620e1
remove emails
MichaelChirico Jun 10, 2022
4e5ceb4
remove temp files
MichaelChirico Jun 10, 2022
50c4119
handle gitignore
MichaelChirico Jun 10, 2022
1aaa681
Merge branch 'main' into rev-urls
MichaelChirico Jun 13, 2022
98a932a
improve e-mail template
MichaelChirico Jun 13, 2022
2549de9
stop if missing file, improve skip=
MichaelChirico Jun 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .dev/get_urls.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/Rscript
# Script to get URL keys from CRAN
# of packags that Suggest or Import lintr

cran_db = as.data.frame(available.packages())
MichaelChirico marked this conversation as resolved.
Show resolved Hide resolved
lintr_re = "\\blintr\\b"
lintr_pkg = cran_db[
with(cran_db, grepl(lintr_re, Suggests) | grepl(lintr_re, Imports)),
"Package"
]

# simple retry mechanism
desc_url_fmt = "https://cran.r-project.org/web/packages/%s/DESCRIPTION"
extract_url = function(pkg, initial_sleep = 1, retry_sleep = 60) {
desc_url = sprintf(desc_url_fmt, pkg)
retry = function(cond) {
message("Sleepy... 😴")
Sys.sleep(retry_sleep)
url(desc_url, open = "r")
}
Sys.sleep(initial_sleep)
url_conn = tryCatch(url(desc_url, open = "r"), error = retry, warning = retry)
pkg_data = read.dcf(url_conn, "URL")
drop(pkg_data)
}

extract_github_repo = function(urls) {
matches = gregexpr("https?://git(hub|lab).com/[a-zA-Z0-9_-]+/[a-zA-Z0-9._-]+", urls)
MichaelChirico marked this conversation as resolved.
Show resolved Hide resolved
starts = sapply(matches, `[[`, 1L)
matched = starts > 0L

urls = urls[matched]
starts = starts[matched]
matches = matches[matched]

unname(substr(urls, starts, starts + sapply(matches, attr, "match.length") - 1L))
}

urls = character(length(lintr_pkg))
# for loop makes debugging easier
for (ii in seq_along(urls)) {
urls[ii] <- extract_url(lintr_pkg[ii])
}

urls = sort(extract_github_repo(urls[!is.na(urls)]))
MichaelChirico marked this conversation as resolved.
Show resolved Hide resolved

writeLines(urls, "reverse-imports-urls")
67 changes: 67 additions & 0 deletions .dev/reverse-imports-urls
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
http://github.com/soodoku/abbyyR
http://github.com/soodoku/tuber
https://github.com/alan-turing-institute/PosteriorBootstrap
https://github.com/bearloga/WikidataQueryServiceR
https://github.com/bfgray3/cattonum
https://github.com/bokeh/rbokeh
https://github.com/Crunch-io/rcrunch
https://github.com/DARTH-git/dampack
https://github.com/dirmeier/datastructures
https://github.com/dirmeier/diffusr
https://github.com/dmlc/xgboost
https://github.com/donlelef/tsviz
https://github.com/Dschaykib/newsmd
https://github.com/ehrlinger/ggRandomForests
https://github.com/hafen/geofacet
https://github.com/ilarischeinin/rasterpdf
https://github.com/jakubnowicki/fixtuRes
https://github.com/jbaileyh/geogrid
https://github.com/joundso/cleaR
https://github.com/jrnold/ggthemes
https://github.com/jumpingrivers/prettyB
https://github.com/kapsner/BiasCorrector
https://github.com/kapsner/rBiasCorrection
https://github.com/kloppen/rde
https://github.com/lifebit-ai/cloudos
https://github.com/lorenzwalthert/precommit
https://github.com/MazamaScience/PWFSLSmoke
https://github.com/mbertolacci/WoodburyMatrix
https://github.com/mi2-warsaw/FSelectorRcpp
https://github.com/mlflow/mlflow
https://github.com/mlr-org/mlr
https://github.com/mlr-org/mlrCPO
https://github.com/mplatzer/BTYDplus
https://github.com/mwaldstein/edgarWebR
https://github.com/nik01010/dashboardthemes
https://github.com/nik01010/openbankeR
https://github.com/openml/openml-r
https://github.com/paulhendricks/describer
https://github.com/philips-software/latrend
https://github.com/r-dbi/DBItest
https://github.com/REditorSupport/languageserver
https://github.com/rexyai/RestRserve
https://github.com/rich-iannone/i18n
https://github.com/r-lib/devtools
https://github.com/r-lib/lifecycle
https://github.com/ropensci/babette
https://github.com/ropensci/beautier
https://github.com/ropensci/dittodb
https://github.com/ropensci/osfr
https://github.com/ropensci/roadoi
https://github.com/ropensci/unifir
https://github.com/rOpenStats/RSQL
https://github.com/rstudio/connectwidgets
https://github.com/russHyde/dupree
https://github.com/sagiegurari/scriptexec
https://github.com/sinhrks/ggfortify
https://github.com/stencila/schema
https://github.com/SwissClinicalTrialOrganisation/secuTrialR
https://github.com/themains/virustotal
https://github.com/thijsjanzen/nLTT
https://github.com/thomas-neitmann/ggcharts
https://github.com/UCLATALL/supernova
https://github.com/wahani/modules
https://github.com/zachmayer/caretEnsemble
https://github.com/zzawadz/DepthProc
https://gitlab.com/fvafrcu/fakemake
https://gitlab.com/fvafrCU/packager