Merge pull request #47 from metaodi/develop

Release 2.0.0

.github/workflows/lint_python.yml

@@ -10,7 +10,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.6, 3.7, 3.8]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
 
     steps:
     - uses: actions/checkout@v2

.gitignore

@@ -2,3 +2,4 @@ sruthi.egg-info
 *.pyc
 *.swp
 .coverage
+pyenv

CHANGELOG.md

@@ -4,6 +4,17 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/) and this p
 
 ## [Unreleased]
 
+## [2.0.0] - 2023-07-06
+### Added
+- Pass in a custom requests session with the `session` parameter
+
+### Changed
+- Use `black` code style
+
+### Removed
+- BC-break: `requests_kwargs` was removed since we can now pass in a custom requests session
+- BC-break: no more support for Python 3.6, minimum required version is now Python 3.7
+
 ## [1.0.0] - 2021-12-06
 ### Added
 - Add support for SRU 1.1 by passing `sru_version='1.1'` to the client or the operation calls.
@@ -76,7 +87,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/) and this p
 - `Fixed` for any bug fixes.
 - `Security` to invite users to upgrade in case of vulnerabilities.
 
-[Unreleased]: https://github.com/metaodi/sruthi/compare/v1.0.0...HEAD
+[Unreleased]: https://github.com/metaodi/sruthi/compare/v2.0.0...HEAD
+[2.0.0]: https://github.com/metaodi/sruthi/compare/v1.0.0...v2.0.0
 [1.0.0]: https://github.com/metaodi/sruthi/compare/v0.1.2...v1.0.0
 [0.1.2]: https://github.com/metaodi/sruthi/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/metaodi/sruthi/compare/v0.1.0...v0.1.1

Makefile

@@ -12,7 +12,11 @@ deps:  ## Install dependencies
 	python -m pip install -r test-requirements.txt
 
 lint:  ## Linting of source code
-	python -m flake8 --statistics --show-source .
+	python -m black --check sruthi examples tests
+	python -m flake8 --statistics --show-source sruthi examples tests
+
+format:  ## Format source code (black codestyle)
+	python -m black sruthi examples tests
 
 test:  ## Run tests
 	python -m pytest --cov=sruthi tests/

README.md

@@ -1,5 +1,6 @@
 [![PyPI Version](https://img.shields.io/pypi/v/sruthi)](https://pypi.org/project/sruthi/)
 [![Tests + Linting Python](https://github.com/metaodi/sruthi/actions/workflows/lint_python.yml/badge.svg)](https://github.com/metaodi/sruthi/actions/workflows/lint_python.yml)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
 # sruthi
 
@@ -67,7 +68,8 @@ Verordnung der Stadt Zürich betreffend die Erfüllung von Amtspflichten durch d
 https://suche.staatsarchiv.djiktzh.ch/detail.aspx?Id=3796980
 ```
 
-The return value of `searchretrieve` is iterable, so you can easily loop over it. Or you can use indices to access elements, e.g. `records[1]` to get the second elemenet, or `records[-1]` to get the last one.
+The return value of `searchretrieve` is iterable, so you can easily loop over it.
+Or you can use indices to access records, e.g. `records[1]` to get the second record, or `records[-1]` to get the last one.
 
 Even [slicing](https://python-reference.readthedocs.io/en/latest/docs/brackets/slicing.html) is supported, so you can do things like only iterate over the first 5 elements using
 
@@ -122,6 +124,23 @@ By default sruthi uses SRU 1.2 to make requests, but you can specify the SRU ver
 8985
 ```
 
+### Custom parameters and settings
+
+If an SRU endpoint needs additional (custom) parameters, you can create your own session object and pass it to the client.
+This is useful for adding authentication (username, password), custom headers or parameters, SSL verification settings etc.
+
+```python
+>>> import sruthi
+>>> import requests
+>>> # customize session
+>>> session = requests.Session()
+>>> session.params = {"x-collection": "GGC"}
+>>> # pass the customized session to sruthi
+>>> records = sruthi.searchretrieve("https://jsru.kb.nl/sru", query="gruninger", session=session)
+>>> records.count
+4
+```
+
 ## Schemas
 
 sruthi does not make any assumptions about the record data schema.
@@ -136,7 +155,7 @@ sruthi has been tested with the following schemas:
 
 To contribute to sruthi simply clone this repository and follow the instructions in [CONTRIBUTING.md](/CONTRIBUTING.md).
 
-This project ha a Makefile with the most common commands.
+This project has a `Makefile` with the most common commands.
 Type `make help` to get an overview.
 
 ## Release

examples/authentication.py

@@ -0,0 +1,16 @@
+import os
+import requests
+from sruthi import Client
+
+# create authenticated session
+user = os.getenv("CATALOG_USER")
+pw = os.getenv("CATALOG_PASS")
+session = requests.Session()
+session.auth = (user, pw)
+
+# pass authenticated session to client
+sru_client = Client("https://suche.staatsarchiv.djiktzh.ch/SRU/", session=session)
+
+# get records for query
+records = sru_client.searchretrieve(query="Zürich")
+print(records)

examples/configure_client_with_explain.py

@@ -1,24 +1,24 @@
 from sruthi import Client
 
 # create a new client and call explain()
-sru_client = Client('https://suche.staatsarchiv.djiktzh.ch/SRU/')
+sru_client = Client("https://suche.staatsarchiv.djiktzh.ch/SRU/")
 info = sru_client.explain()
 
 for name, details in info.schema.items():
     print(f"This SRU endpoint supports the metadata schema {details['title']}.")
 
 # configure the maximum records based on the config
 try:
-    sru_client.maximum_records = info.config['maximumRecords']
+    sru_client.maximum_records = info.config["maximumRecords"]
     print(f"Set maximum_records to {sru_client.maximum_records}.")
 except KeyError:
     print("Config `maximum_records` not available, keep original value")
 
 # get records for query
-records = sru_client.searchretrieve(query='Zürich')
+records = sru_client.searchretrieve(query="Zürich")
 
 # display 5 records
-print('')
-print('First 5 results for `Zürich`')
+print("")
+print("First 5 results for `Zürich`")
 for r in records[:5]:
-    print("* ", r['title'])
+    print("* ", r["title"])

examples/custom_parameter.py

@@ -0,0 +1,25 @@
+import requests
+import sruthi
+from pprint import pprint
+
+
+def print_url(r, *args, **kwargs):
+    print(r.url)
+
+
+# create session with custom paramter session
+session = requests.Session()
+
+# here some example of how a session can be used to customize parameters, settings etc.
+session.params = {"x-collection": "GGC"}  # add custom request parameter
+session.verify = False  # disable SSL verfications
+session.hooks["response"].append(print_url)  # add custom hook
+
+# pass custom session to client
+sru_client = sruthi.Client("https://jsru.kb.nl/sru", session=session)
+
+# get records for query
+records = sru_client.searchretrieve(query="gruninger")
+pprint(records)
+print("---")
+pprint(records[0])

examples/explain.py

@@ -4,52 +4,52 @@
 import yaml
 
 sru_endpoints = [
-    'https://suche.staatsarchiv.djiktzh.ch/SRU/',
-    'https://amsquery.stadt-zuerich.ch/SRU/',
-    'http://lx2.loc.gov:210/LCDB?',
-    'https://na01.alma.exlibrisgroup.com/view/sru/TR_INTEGRATION_INST',
+    "https://suche.staatsarchiv.djiktzh.ch/SRU/",
+    "https://amsquery.stadt-zuerich.ch/SRU/",
+    "http://lx2.loc.gov:210/LCDB?",
+    "https://na01.alma.exlibrisgroup.com/view/sru/TR_INTEGRATION_INST",
 ]
 
 
 def print_header(s):
-    cprint(s, 'green', attrs=['bold'])
+    cprint(s, "green", attrs=["bold"])
 
 
 def print_title(s):
-    cprint(s, attrs=['bold'])
+    cprint(s, attrs=["bold"])
 
 
 def dump(d):
     print(yaml.dump(d, allow_unicode=True, default_flow_style=False))
 
 
 for endpoint in sru_endpoints:
-    print_header(20 * '=')
-    print_header('=')
-    print_header(f'= {endpoint}')
-    print_header('=')
-    print_header(20 * '=')
+    print_header(20 * "=")
+    print_header("=")
+    print_header(f"= {endpoint}")
+    print_header("=")
+    print_header(20 * "=")
     info = sruthi.explain(endpoint)
 
-    print_title('Server:')
+    print_title("Server:")
     dump(info.server)
-    print('')
+    print("")
 
-    print_title('Database:')
+    print_title("Database:")
     dump(info.database)
-    print('')
+    print("")
 
-    print_title('Index:')
+    print_title("Index:")
     dump(info.index)
-    print('')
+    print("")
 
-    print_title('Schema:')
+    print_title("Schema:")
     dump(info.schema)
-    print('')
+    print("")
 
-    print_title('Config:')
+    print_title("Config:")
     dump(info.config)
-    print('')
+    print("")
 
-    print('')
-    print('')
+    print("")
+    print("")

examples/generate_csv.py

@@ -4,33 +4,33 @@
 import traceback
 
 records = sruthi.searchretrieve(
-    'https://amsquery.stadt-zuerich.ch/SRU/',
-    query="isad.reference = V.B.b.43.:1 AND isad.descriptionlevel = Dossier"
+    "https://amsquery.stadt-zuerich.ch/SRU/",
+    query="isad.reference = V.B.b.43.:1 AND isad.descriptionlevel = Dossier",
 )
 
 try:
     header = [
-       'reference',
-       'title',
-       'year',
-       'url',
+        "reference",
+        "title",
+        "year",
+        "url",
     ]
     writer = csv.DictWriter(
         sys.stdout,
         header,
-        delimiter=',',
+        delimiter=",",
         quotechar='"',
-        lineterminator='\n',
-        quoting=csv.QUOTE_MINIMAL
+        lineterminator="\n",
+        quoting=csv.QUOTE_MINIMAL,
     )
     writer.writeheader()
 
     for record in records:
         row = {
-           'reference': record['reference'],
-           'title': record['title'],
-           'year': record['date'],
-           'url': record['extra']['link'],
+            "reference": record["reference"],
+            "title": record["title"],
+            "year": record["date"],
+            "url": record["extra"]["link"],
         }
         writer.writerow(row)
 except Exception as e:

examples/isad.py

@@ -2,19 +2,15 @@
 from pprint import pprint
 
 # check supported schemas of server
-server_url = 'https://suche.staatsarchiv.djiktzh.ch/SRU/'
-schema = 'isad'
+server_url = "https://suche.staatsarchiv.djiktzh.ch/SRU/"
+schema = "isad"
 server = sruthi.explain(server_url)
 
 
-print(20 * '=')
-print('=')
+print(20 * "=")
+print("=")
 print(f"= Record with schema: {schema}")
-print('=')
-print(20 * '=')
-records = sruthi.searchretrieve(
-    server_url,
-    query='Zurich',
-    record_schema=schema
-)
+print("=")
+print(20 * "=")
+records = sruthi.searchretrieve(server_url, query="Zurich", record_schema=schema)
 pprint(records[0])

examples/library_of_congress.py

@@ -1,27 +1,27 @@
 import sruthi
 import sys
 
-LOC_BASE = 'http://lx2.loc.gov:210/LCDB?'
+LOC_BASE = "http://lx2.loc.gov:210/LCDB?"
 
 
 def loc_search(isbn, sru_base):
     loc_lcc = None
     try:
         records = sruthi.searchretrieve(sru_base, query=isbn)
         record = records[0]
-        fields = record.get('datafield', [])
+        fields = record.get("datafield", [])
         for field in fields:
-            if field['tag'] != '050':
+            if field["tag"] != "050":
                 continue
-            if len(field.get('subfield', [])) > 0:
-                loc_lcc = (field['subfield'][0]['text'])
+            if len(field.get("subfield", [])) > 0:
+                loc_lcc = field["subfield"][0]["text"]
                 break
     except Exception as e:
         print("Error: %s" % e, file=sys.stderr)
         return None
     return loc_lcc
 
 
-isbn = '0062509470'
+isbn = "0062509470"
 result = loc_search(isbn, LOC_BASE)
 print(f"Tag 050 of ISBN '{isbn}': {result}")

examples/schemas.py

@@ -2,22 +2,20 @@
 from pprint import pprint
 
 # check supported schemas of server
-server = sruthi.explain('http://lx2.loc.gov:210/LCDB?')
+server = sruthi.explain("http://lx2.loc.gov:210/LCDB?")
 
 print(f"Supported schemas: {', '.join(server.schema.keys())}")
 
 
 for schema in server.schema.keys():
-    print(20 * '=')
-    print('=')
+    print(20 * "=")
+    print("=")
     print(f"= Record with schema: {schema}")
-    print('=')
-    print(20 * '=')
+    print("=")
+    print(20 * "=")
     records = sruthi.searchretrieve(
-        'http://lx2.loc.gov:210/LCDB?',
-        query="human",
-        record_schema=schema
+        "http://lx2.loc.gov:210/LCDB?", query="human", record_schema=schema
     )
     pprint(records[0])
-    print('')
-    print('')
+    print("")
+    print("")