Skip to content

Commit

Permalink
Merge branch 'master' into snyk-fix-cd0df4562b0a872d3083d7bb36dd10cb
Browse files Browse the repository at this point in the history
  • Loading branch information
jmbraybrook authored Nov 20, 2024
2 parents 3b77ad3 + 6199e7f commit 281262f
Show file tree
Hide file tree
Showing 22 changed files with 195 additions and 172 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11" ]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13" ]

env:
## Environment variable
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## [1.3.0] - 2024/11/18
- Removing Support for Python 3.7
- Adding Support for Python 3.12, 3.13

- Resolved issues on installation on later python versions
- Updated Typeguard Version
- Updated Packages to latest versions
- Fixed typing on GeoJson Outputs -> Feature Collection to Dict.

## [1.2.11] - 2024/07/08
- Package Resupported - Supported under new team [jmbraybrook]

## [1.2.10] - 2024/03/08

### Changed
Expand Down
13 changes: 3 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
# osdatahub <!-- omit in toc -->


> **Important Notice**:
>
> - This Python package is no longer actively maintained and will not receive updates.
> - It may become incompatible with future [OS Data Hub APIs](https://osdatahub.os.uk/).

[![GitHub issues](https://img.shields.io/github/issues/OrdnanceSurvey/osdatahub)](https://github.com/OrdnanceSurvey/osdatahub/issues)
[![Python package](https://github.com/OrdnanceSurvey/osdatahub/actions/workflows/python-package.yml/badge.svg)](https://github.com/OrdnanceSurvey/osdatahub/actions/workflows/python-package.yml)
<a href="https://codeclimate.com/github/dchirst/osdatahub/maintainability"><img src="https://api.codeclimate.com/v1/badges/471fd53dbb22e9e28546/maintainability" /></a>
Expand Down Expand Up @@ -271,7 +264,7 @@ results = linked_ids.query(200001025758)
## Downloads API

If you'd like to download an entire dataset instead of querying the API on demand, the OS Data Hub has the
[Downloads API](https://osdatahub.os.uk/docs/downloads/technicalSpecification). This API allows you to search,m explore, and download both [Open Data Products](https://osdatahub.os.uk/downloads/open) (e.g. OS Open Rivers, Boundary-Line, and a 1:250,000 scale
[Downloads API](https://osdatahub.os.uk/docs/downloads/technicalSpecification). This API allows you to search, explore, and download both [Open Data Products](https://osdatahub.os.uk/downloads/open) (e.g. OS Open Rivers, Boundary-Line, and a 1:250,000 scale
colour raster of Great Britain) and Premium Data Packages using Python.

It is possible to download Open Data products without an API key, but the Premium Data Packages require you to have
Expand Down Expand Up @@ -341,7 +334,7 @@ and will apply to all the osdatahub api requests.

# Contribute

This package is still under active developement and we welcome contributions from the community via issues and pull requests.
This package is still under active development and we welcome contributions from the community via issues and pull requests.

To install osdatahub, along with the tools you need to develop and run tests,
run the following in your environment:
Expand All @@ -352,4 +345,4 @@ pip install -e .[dev]

## Support

For any kind of issues or suggestions please see the [**documentation**](https://osdatahub.readthedocs.io/en/latest/), open a **[github issue](https://github.com/OrdnanceSurvey/osdatahub/issues)** or contact us via Email **[rapidprototyping@os.uk](mailto:rapidprototyping@os.uk)**
For any kind of issues or suggestions please see the [**documentation**](https://osdatahub.readthedocs.io/en/latest/), open a **[github issue](https://github.com/OrdnanceSurvey/osdatahub/issues)** or contact us via Email **[datascience@os.uk](mailto:datascience@os.uk)**
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
author = "OS Rapid Prototyping Team"

# The full version, including alpha/beta/rc tags
release = "1.2.3"
release = "1.3.0"

# -- General configuration ---------------------------------------------------

Expand Down
4 changes: 3 additions & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
myst-parser==0.15.2
sphinx-rtd-theme
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
setuptools>=75.3.0
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[build-system]
requires = ["setuptools>=65.5.1"]
requires = ["setuptools>=75.3.0"]
build-backend = "setuptools.build_meta"

21 changes: 10 additions & 11 deletions recipes/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

{% set name = "osdatahub" %}
{% set version = "1.2.3" %}
{% set version = "1.3.0" %}


package:
Expand All @@ -19,15 +19,14 @@ build:
requirements:
host:
- pip
- python >=3.7
- python >=3.8
run:
- geojson ==2.5.0
- python >=3.7
- requests ==2.25.0
- shapely ==1.8.0
- typeguard ==2.13.0
- tqdm ~=4.62.3

- geojson ~=3.1.0
- python >=3.8
- requests ~=2.32.3
- shapely ~=2.0.6
- typeguard ~=4.4.0
- tqdm ~=4.67.0
test:
imports:
- osdatahub
Expand All @@ -45,6 +44,6 @@ about:
dev_url: https://github.com/OrdnanceSurvey/osdatahub
extra:
recipe-maintainers:
- dchirst
- JEPooley
- FHunt-OS
- FHunt-OS
- jmbraybrook
14 changes: 8 additions & 6 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
geojson~=3.0.1
requests~=2.31.0
typeguard~=2.13.0
shapely~=2.0.0
tqdm~=4.65.0
setuptools~=67.7.2
geojson~=3.1.0
typeguard~=4.4.0
shapely~=2.0.6
tqdm~=4.67.0
setuptools>=75.3.0
requests~=2.32.3
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability

25 changes: 14 additions & 11 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
[metadata]
name = osdatahub
version = 1.2.10
author = OS Rapid Prototyping
author_email = rapidprototyping@os.uk
version = 1.3.0
author = OS Data Science
author_email = datascience@os.uk
classifiers =
Natural Language :: English
Intended Audience :: Developers
Intended Audience :: Science/Research
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13
Topic :: Utilities
Topic :: Scientific/Engineering :: GIS
description = osdatahub is Ordnance Survey's (OS) Python API wrapper, designed to make data from the OS Data Hub APIs readily accessible to developers.
Expand All @@ -33,13 +34,15 @@ url = https://github.com/OrdnanceSurvey/osdatahub
[options]
include_package_data = True
install_requires =
geojson~=3.0.1
requests~=2.31.0
typeguard~=2.13.0
shapely~=2.0.0
tqdm~=4.65.0
setuptools~=67.7.2
python_requires = >=3.7
geojson~=3.1.0
requests~=2.32.3
typeguard~=4.4.0
shapely~=2.0.6
tqdm~=4.67.0
setuptools>=75.3.0
urllib3>=2.2.2
zipp>=3.19.1
python_requires = >=3.8
package_dir=
=src
packages=find:
Expand Down
4 changes: 2 additions & 2 deletions src/osdatahub/DownloadsAPI/data_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,9 @@ def product_list(self, version_id: str, return_downloadobj: bool = False) -> Uni
def download(self,
version_id: str,
output_dir: Union[str, Path] = ".",
file_name: str = None,
file_name: Union[str, None] = None,
overwrite: bool = False,
processes: int = None) -> list:
processes: Union[int, None] = None) -> list:
"""
Downloads Data Package files to your local machine
Expand Down
12 changes: 9 additions & 3 deletions src/osdatahub/DownloadsAPI/downloads_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ def __init__(self, url: str, file_name: str, size: int):
self.file_name = file_name
self.size = size

def download(self, output_dir: Union[str, Path], overwrite: bool = False, pbar: tqdm = None) -> str:
def download(self,
output_dir: Union[str, Path],
overwrite: bool = False,
pbar: Union[tqdm, None] = None) -> str:
"""
Downloads file to given directory
Expand Down Expand Up @@ -179,8 +182,11 @@ def product_list(self):
pass

@staticmethod
def _download(download_list: Union[list, _DownloadObj], output_dir: Union[str, Path], overwrite: bool = False,
download_multiple: bool = False, processes: int = None) -> list:
def _download(download_list: Union[list, _DownloadObj],
output_dir: Union[str, Path],
overwrite: bool = False,
download_multiple: bool = False,
processes: Union[int, None] = None) -> list:
"""
Downloads product/datapackage to the given directory. Can download a single format or can download multiple
formats in parallel
Expand Down
21 changes: 13 additions & 8 deletions src/osdatahub/DownloadsAPI/opendata.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,12 @@ class OpenDataDownload(_DownloadsAPIBase):

# TODO: change name
@typechecked
def product_list(self, file_name: str = None, file_format: str = None, file_subformat: str = None,
area: str = None, return_downloadobj: bool = False) -> Union[list, dict]:
def product_list(self,
file_name: Union[str, None] = None,
file_format: Union[str, None] = None,
file_subformat: Union[str, None] = None,
area: Union[str, None] = None,
return_downloadobj: bool = False) -> Union[list, dict]:
"""
Returns a list of possible downloads for a specific OS OpenData Product based on given filters
Expand Down Expand Up @@ -60,14 +64,15 @@ def product_list(self, file_name: str = None, file_format: str = None, file_subf
else:
return response.json()

def download(self, output_dir: Union[str, Path] = ".",
file_name: str = None,
file_format: str = None,
file_subformat: str = None,
area: str = None,
def download(self,
output_dir: Union[str, Path] = ".",
file_name: Union[str, None] = None,
file_format: Union[str, None] = None,
file_subformat: Union[str, None] = None,
area: Union[str, None] = None,
download_multiple: bool = False,
overwrite: bool = False,
processes: int = None) -> list:
processes: Union[int, None] = None) -> list:
"""
Downloads Product files to your local machine
Expand Down
10 changes: 6 additions & 4 deletions src/osdatahub/FeaturesAPI/features_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import requests
from geojson import FeatureCollection
from typeguard import check_argument_types
from typeguard import typechecked

import osdatahub
from osdatahub.extent import Extent
Expand Down Expand Up @@ -82,6 +82,7 @@ def product(self, product_name: str):
def xml_filter(self):
return self.__construct_filter()

@typechecked
def query(self, limit: int = 100) -> FeatureCollection:
"""Run a query of the OS Features API
Expand All @@ -92,7 +93,7 @@ def query(self, limit: int = 100) -> FeatureCollection:
Returns:
FeatureCollection: The results of the query in GeoJSON format
"""
assert check_argument_types()

params = self.__params
data = GrowList()
n_required = min(limit, 100)
Expand Down Expand Up @@ -138,14 +139,15 @@ def __params(self) -> dict:
"typeName": self.product.name,
"filter": self.__construct_filter(),
}


@typechecked
def add_filters(self, *xml_filters: Filter) -> None:
"""Add XML filter strings to the final query
Args:
xml_filters (str): Valid OGC XML filter objects
"""
assert check_argument_types()

self.filters.extend(xml_filters)


Expand Down
26 changes: 12 additions & 14 deletions src/osdatahub/NGD/ngd_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,14 @@
from typing import Union

import requests
from geojson import Feature, FeatureCollection
from typeguard import check_argument_types
from typeguard import typechecked

import osdatahub
from osdatahub import Extent
from osdatahub.NGD.crs import get_crs


def _merge_geojsons(gj1: FeatureCollection, gj2: FeatureCollection) -> FeatureCollection:
def _merge_geojsons(gj1: Union[dict], gj2: Union[dict]) -> Union[dict]:
"""
Combines 2 geojsons from NGD api into a single valid geojson
Expand Down Expand Up @@ -84,16 +83,17 @@ def get_collections(cls) -> dict:
response = osdatahub.get(cls.__ENDPOINT, proxies=osdatahub.get_proxies())
response.raise_for_status()
return response.json()


@typechecked
def query(self,
extent: Extent = None,
crs: Union[str, int] = None,
start_datetime: datetime = None,
end_datetime: datetime = None,
cql_filter: str = None,
filter_crs: Union[str, int] = None,
extent: Union[Extent, None] = None,
crs: Union[str, int, None] = None,
start_datetime: Union[datetime, None] = None,
end_datetime: Union[datetime, None] = None,
cql_filter: Union[str, None] = None,
filter_crs: Union[str, int, None] = None,
max_results: int = 100,
offset: int = 0) -> FeatureCollection:
offset: int = 0) -> Union[dict]:
"""
Retrieves features from a Collection
Expand Down Expand Up @@ -123,7 +123,6 @@ def query(self,
FeatureCollection: The results of the query in GeoJSON format
"""

assert check_argument_types()
assert max_results > 0, f"Argument max_results must be greater than 0 but was {max_results}"
assert offset >= 0, f"Argument offset must be greater than 0 but was {offset}"
params = {}
Expand Down Expand Up @@ -183,7 +182,6 @@ def query(self,
raise e

resp_json = response.json()

data = _merge_geojsons(data, resp_json)

if resp_json["numberReturned"] < limit:
Expand All @@ -193,7 +191,7 @@ def query(self,

return data

def query_feature(self, feature_id: str, crs: Union[str, int] = None) -> Feature:
def query_feature(self, feature_id: str, crs: Union[str, int] = None) -> dict:
"""
Retrieves a single feature from a collection
Expand Down
Loading

0 comments on commit 281262f

Please sign in to comment.