Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HttpOpener shall be able to handle GZIP encoding #511

Closed
dr0i opened this issue Nov 27, 2023 · 1 comment · Fixed by #513
Closed

HttpOpener shall be able to handle GZIP encoding #511

dr0i opened this issue Nov 27, 2023 · 1 comment · Fixed by #513
Assignees

Comments

@dr0i
Copy link
Member

dr0i commented Nov 27, 2023

HttpOpener , aka decode-html when speaking flux, is not able to decode gziped data atm.

This was discovered by @TobiasNx failing to lookup schema.org .

@dr0i dr0i self-assigned this Nov 27, 2023
@dr0i dr0i added this to Metafacture Nov 27, 2023
@dr0i dr0i moved this to Selected in Metafacture Nov 27, 2023
@dr0i
Copy link
Member Author

dr0i commented Nov 28, 2023

Working on it I noticed

r -> r.withHeader("Content-Encoding", "UTF-8")

This seems invalid because:

The content-encoding specifies the data transfer encoding used by the issuer of the content. UTF-8 is not a content encoding, it is a character set. Specifying the character set is done in the content-type header

(https://stackoverflow.com/questions/17154967/is-content-encoding-being-set-to-utf-8-invalid)

[EDIT] there seems a principal misunderstanding of encoding in HttpOpener as a synonym for charset. I Propose to rename variables and methods to charset , probably mark setEncoding(String) as deprecated.

@dr0i dr0i moved this from Selected to Working in Metafacture Nov 28, 2023
dr0i added a commit that referenced this issue Dec 18, 2023
- follows redirects
- fixes misconception of "Content-Encoding"
@dr0i dr0i closed this as completed in #513 Jan 2, 2024
@github-project-automation github-project-automation bot moved this from Working to Done in Metafacture Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant