Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Made diff about *.pbf ignore features order #1630

Open
sharkAndshark opened this issue Dec 23, 2024 · 9 comments · May be fixed by #1642
Open

Made diff about *.pbf ignore features order #1630

sharkAndshark opened this issue Dec 23, 2024 · 9 comments · May be fixed by #1642
Labels
github_actions Pull requests that update GitHub Actions code

Comments

@sharkAndshark
Copy link
Collaborator

sharkAndshark commented Dec 23, 2024

CI failed due to the features order changed.

feature test on latest(ubuntu 24.04 + postgis/postgis:16-3.5) other pg version in CI
cmp_0_0_0.pbf.txt, feature of layer 1 which its gid == 19 29th feature 18th feature

A  vt-show which don't care features order might be needed.

@sharkAndshark sharkAndshark changed the title CI failed Made diff about *.pbf ignore features order Dec 25, 2024
@sharkAndshark sharkAndshark added the github_actions Pull requests that update GitHub Actions code label Dec 25, 2024
@sharkAndshark
Copy link
Collaborator Author

To duplicate this failure just update the docker-compose.yml and run just bless or just test

........
  db-is-ready:
    # This should match the version of postgres used in the CI workflow
    image: postgis/postgis:16-3.5
........
  db:
    # This should match the version of postgres used in the CI workflow
    image: postgis/postgis:16-3.5
........

@sharkAndshark
Copy link
Collaborator Author

@nyurik we should fix this how about make a new vt-show tool? CC @CommanderStorm

@nyurik
Copy link
Member

nyurik commented Dec 26, 2024

yeah, perhaps a simple tile content dumper, possibly using geozero for the conversion into geojson or csv, and sorting it? I don't think Martin should contain the needed code to decode protobuf itself.

@sharkAndshark
Copy link
Collaborator Author

sharkAndshark commented Dec 26, 2024

Agree. It should be a repo outside which read *.pbf, sort the features before writing. CSV or GeoJSON To Be Decided.

How to sort?

Join all the properties key value paris, geometry type and coordiantes and then make a hash?

@nyurik
Copy link
Member

nyurik commented Dec 26, 2024

I don't think we need a separate repo - this is just a few lines of code we would need, and maybe no code at all - just install the geozero cli tool. See https://github.com/georust/geozero - and if the list of features is exported to CSV, you can run it through ... | sort in a linux shell. Or if you want to store geojson, perhaps use jq to sort a specific array (this one is a bit trickier as you would need to use jq script)

@sharkAndshark
Copy link
Collaborator Author

Not be aware that it's a CLI tool either. I thought it's a lib only.. lollll Yeah. It's better make this in shell script.

@nyurik
Copy link
Member

nyurik commented Dec 26, 2024

yeah, it is not auto-published (i just created a ticket for that). I guess we can simply do a few line binary if we can't get cli to do what we need, perhaps as a unpublished test tool.

@sharkAndshark
Copy link
Collaborator Author

So let's try shell + csv first. Seems it's the best solution.

@CommanderStorm
Copy link
Collaborator

perhaps use jq to sort a specific array (this one is a bit trickier as you would need to use jq script)

For reference: we already have merged something similar ^^
https://github.com/maplibre/martin/blob/26249e217eb1d802a11e511bbaab92372ef91d69/tests/test.sh#L86C18-L86C32

@sharkAndshark sharkAndshark linked a pull request Jan 4, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants