Skip to content
This repository has been archived by the owner on Nov 14, 2019. It is now read-only.

lastModified header #132

Open
viktor-svirsky opened this issue Sep 8, 2017 · 0 comments
Open

lastModified header #132

viktor-svirsky opened this issue Sep 8, 2017 · 0 comments

Comments

@viktor-svirsky
Copy link

Hi @marevol,
Our main idea to get actual page modified date. In our system, we already stored pages with timestamp values. In this case, we need to reuse lastModified header value if it present in crawled records.
We need to convert lastModified date value to timestamp value like '1504887075' for future ordering. If the record doesn't include lastModified value, we prefer use @timestamp value.

Example:

lastModified: "2017-09-04T14:58:57.000Z",
timestamp: "1504887075", //<<<==== this value generated based on lastModified or @timestamp values
@timestamp: "2017-09-08T16:11:31.546Z"

Is it possible to rewrite logic, that could cover our expectations?

Thanks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant