You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Nov 14, 2019. It is now read-only.
Hi @marevol,
Our main idea to get actual page modified date. In our system, we already stored pages with timestamp values. In this case, we need to reuse lastModified header value if it present in crawled records.
We need to convert lastModified date value to timestamp value like '1504887075' for future ordering. If the record doesn't include lastModified value, we prefer use @timestamp value.
Example:
lastModified: "2017-09-04T14:58:57.000Z",
timestamp: "1504887075", //<<<==== this value generated based on lastModified or @timestamp values @timestamp: "2017-09-08T16:11:31.546Z"
Is it possible to rewrite logic, that could cover our expectations?
Thanks
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Hi @marevol,
Our main idea to get actual page modified date. In our system, we already stored pages with timestamp values. In this case, we need to reuse lastModified header value if it present in crawled records.
We need to convert lastModified date value to timestamp value like '1504887075' for future ordering. If the record doesn't include lastModified value, we prefer use @timestamp value.
Example:
lastModified: "2017-09-04T14:58:57.000Z",
timestamp: "1504887075", //<<<==== this value generated based on lastModified or @timestamp values
@timestamp: "2017-09-08T16:11:31.546Z"
Is it possible to rewrite logic, that could cover our expectations?
Thanks
The text was updated successfully, but these errors were encountered: