Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix case-sensitivity for automated snyk update script #906

Merged
merged 11 commits into from
Apr 4, 2023
Merged

Fix case-sensitivity for automated snyk update script #906

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4470b67
new: fix case-sensitivity for snyk updates
nickumia-reisys Apr 3, 2023
3a48337
fix: reset update to False for the next package
nickumia-reisys Apr 3, 2023
d9936b7
fix: ckanext-qa doesn't like newer setuptools in virtualenv?
nickumia-reisys Apr 3, 2023
373f241
new: add special docker-compose service for requirements
nickumia-reisys Apr 3, 2023
fa213ec
Update Pip Requirements
datagov-bot Apr 3, 2023
ad00d4f
Merge pull request #907 from GSA/requirement-patches
nickumia-reisys Apr 3, 2023
1ee3095
revert: standalone ckan-requirements is not any faster
nickumia-reisys Apr 3, 2023
b40836c
Merge branch 'fix-snyk-update-script' of github.com:GSA/catalog.data.…
nickumia-reisys Apr 3, 2023
6297907
revert: only update if explicit reference did not work
nickumia-reisys Apr 3, 2023
f964267
revert: spatial is not ready for upgrade
nickumia-reisys Apr 3, 2023
6c2f527
revert: harvest? maybe something breaking here?
nickumia-reisys Apr 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion ckan/freeze-requirements.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ trap cleanup EXIT
pip3 install virtualenv

virtualenv $venv
${venv}/bin/pip3 install setuptools==67.1.0
${venv}/bin/pip3 install -r /app/ckan/requirements.in

${venv}/bin/pip3 freeze --all > /app/ckan/requirements.txt
${venv}/bin/pip3 freeze --all > /app/ckan/requirements.txt
1 change: 1 addition & 0 deletions ckan/requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,3 +116,4 @@ gevent>=21.8.0
newrelic
certifi>=2022.12.7
cryptography>=39.0.1
redis>=4.5.4
30 changes: 15 additions & 15 deletions ckan/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@ async-timeout==4.0.2
Babel==2.9.1
Beaker==1.11.0
bleach==3.3.0
boto3==1.26.73
botocore==1.29.73
boto3==1.26.104
botocore==1.29.104
certifi==2022.12.7
cffi==1.15.1
chardet==3.0.4
ckan==2.9.8
-e git+https://github.com/ckan/ckanext-archiver.git@c96e3c81bfc430cdb0372f3307c7abd4109a80f1#egg=ckanext_archiver
-e git+https://github.com/ckan/ckanext-archiver.git@cbfadf9fbf10405958fdef9f77a7faedc05aa20b#egg=ckanext_archiver
ckanext-datagovcatalog==0.0.5
ckanext-datagovtheme==0.1.24
ckanext-datajson==0.1.17
ckanext-dcat @ git+https://github.com/ckan/ckanext-dcat@618928be5a211babafc45103a72b6aab4642e964
ckanext-envvars==0.0.2
ckanext-envvars==0.0.3
ckanext-geodatagov==0.1.33
ckanext-googleanalyticsbasic==0.2.0
-e git+https://github.com/ckan/ckanext-harvest.git@89a98d7ff5aa3445d8158921669b8d0b04fa41c3#egg=ckanext_harvest
Expand All @@ -25,10 +25,10 @@ ckanext-saml2auth @ git+https://github.com/keitaroinc/ckanext-saml2auth.git@b2c6
-e git+https://github.com/ckan/ckanext-spatial.git@17d5a341cf8f40b35b25df91a18ce72c31195ba3#egg=ckanext_spatial
ckantoolkit==0.0.7
click==7.1.2
cryptography==39.0.1
cryptography==40.0.1
defusedxml==0.7.1
dominate==2.4.0
elementpath==4.0.1
elementpath==4.1.0
fanstatic==1.1
feedgen==0.9.0
Flask==2.0.0
Expand All @@ -42,7 +42,7 @@ greenlet==2.0.2
gunicorn==20.1.0
html5lib==1.1
idna==2.10
importlib-resources==5.10.2
importlib-resources==5.12.0
isodate==0.6.1
itsdangerous==2.1.2
Jinja2==3.0.0
Expand All @@ -54,7 +54,7 @@ Mako==1.2.4
Markdown==3.1.1
MarkupSafe==2.1.2
messytables==0.15.2
newrelic==8.7.0
newrelic==8.8.0
nose==1.3.7
numpy==1.24.2
OWSLib==0.18.0
Expand All @@ -63,15 +63,15 @@ passlib==1.7.3
PasteDeploy==2.0.1
pathtools==0.1.2
pika==1.3.1
pip==23.0
pip==23.0.1
ply==3.11
polib==1.0.7
progressbar==2.5
progressbar2==3.53.3
psycopg2==2.8.6
pycparser==2.21
PyJWT==2.4.0
pyOpenSSL==23.0.0
pyOpenSSL==23.1.1
pyparsing==3.0.9
pyproj==2.6.1
pysaml2==7.0.1
Expand All @@ -85,7 +85,7 @@ PyUtilib==5.7.1
PyYAML==5.4
PyZ3950 @ git+https://github.com/danizen/PyZ3950@6d44a4ab85c8bda3a7542c2c9efdfad46c830219
rdflib==4.2.2
redis==4.5.1
redis==4.5.4
repoze.lru==0.7
repoze.who==2.3
requests==2.25.0
Expand All @@ -102,14 +102,14 @@ SQLAlchemy==1.3.5
sqlparse==0.4.2
tzlocal==1.3
unicodecsv==0.14.1
urllib3==1.26.14
urllib3==1.26.15
webassets==0.12.1
webencodings==0.5.1
WebOb==1.8.7
Werkzeug==2.0.0
wheel==0.38.4
xlrd==2.0.1
xmlschema==2.2.1
zipp==3.13.0
xmlschema==2.2.2
zipp==3.15.0
zope.event==4.6
zope.interface==5.5.2
zope.interface==6.0
6 changes: 3 additions & 3 deletions tools/snyk-update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
new_version = v['upgradeTo'].split('@')[1]
print(package, old_version, new_version)

# Remove old version
os.system('sed -i "/%s/d" ckan/requirements.in' % (package + "==" + old_version))
os.system('sed -i "/%s/d" ckan/requirements.in' % (package + ">=" + old_version))
# TODO: Handle case when vulnerable package isn't explicitly in requirements.in

# Remove old version
os.system('sed -i "/^%s\\(=\\|>\\|$\\)/Id" ckan/requirements.in' % (package))
# Add new version
os.system("echo '%s' >> ckan/requirements.in" % (package + ">=" + new_version))

Expand Down