Fix incorrect handling of chain rollbacks (#914)

CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog], and this project adheres to [Semantic Versioning].
 
+## [Unreleased]
+
+### Fixed
+
+- tezos.tzkt: Fixed an issue with approving schema after reindexing triggered by rollback.
+- evm.subsquid: Last mile indexing is significantly faster now.
+
 ## [7.2.1] - 2023-12-12
 
 ### Added

docs/6.deployment/7.monitoring.md

@@ -11,20 +11,20 @@ This check says that DipDup is not stuck and keeps receiving new data (the last
 
 ## URI format
 
-```
-https://<your-indexer-host>/api/rest/dipdup_head_status?name=<datasource-uri>
+```text
+https://<your-indexer-host>/api/rest/dipdup_head_status?name=<datasource>
 ```
 
 If you have camel case enabled in the Hasura config:
 
-```
-https://<your-indexer-host>/api/rest/dipdupHeadStatus?name=<datasource-uri>
+```text
+https://<your-indexer-host>/api/rest/dipdupHeadStatus?name=<datasource>
 ```
 
 For example:
 
-* [https://domains.dipdup.net/api/rest/dipdup_head_status?name=https://api.tzkt.io](https://domains.dipdup.net/api/rest/dipdup_head_status?name=https://api.tzkt.io)
-* [https://juster.dipdup.net/api/rest/dipdupHeadStatus?name=https://api.tzkt.io](https://domains.dipdup.net/api/rest/dipdup_head_status?name=https://api.tzkt.io)
+* [https://domains.dipdup.net/api/rest/dipdup_head_status?name=mainnet](https://domains.dipdup.net/api/rest/dipdup_head_status?name=mainnet)
+* [https://juster.dipdup.net/api/rest/dipdupHeadStatus?name=mainnet](https://domains.dipdup.net/api/rest/dipdup_head_status?name=mainnet)
 
 ### Response
 
@@ -57,16 +57,7 @@ Otherwise, the state is considered **OUTDATED**:
 The default check looks like the following:
 
 ```sql
-CREATE
-OR REPLACE VIEW dipdup_head_status AS
-SELECT
-    name,
-    CASE
-        WHEN timestamp < NOW() - interval '3 minutes' THEN 'OUTDATED'
-        ELSE 'OK'
-    END AS status
-FROM
-    dipdup_head;
+{{ #include ../src/dipdup/sql/dipdup_head_status.sql }}
 ```
 
 You can also create your custom alert endpoints using SQL views and functions and then convert them to Hasura REST endpoints.

docs/7.references/2.config.md

@@ -169,7 +169,7 @@ description: "Config file reference"
 
 <dl class="py class">
 <dt class="sig sig-object py" id="ResolvedHttpConfig">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre"></span></span><span class="sig-name descname"><span class="pre">ResolvedHttpConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">retry_count</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_multiplier</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">2.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_rate</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_period</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_limit</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">60</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10000</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">replay_path</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">alias</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#ResolvedHttpConfig" title="Link to this definition">¶</a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre"></span></span><span class="sig-name descname"><span class="pre">ResolvedHttpConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">retry_count</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_multiplier</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">2.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_rate</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_period</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_limit</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">60</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">request_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">60</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">10000</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">replay_path</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">alias</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#ResolvedHttpConfig" title="Link to this definition">¶</a></dt>
 <dd><p>HTTP client configuration with defaults</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -182,6 +182,7 @@ description: "Config file reference"
 <li><p><strong>ratelimit_sleep</strong> (<em>float</em>) – </p></li>
 <li><p><strong>connection_limit</strong> (<em>int</em>) – </p></li>
 <li><p><strong>connection_timeout</strong> (<em>int</em>) – </p></li>
+<li><p><strong>request_timeout</strong> (<em>int</em>) – </p></li>
 <li><p><strong>batch_size</strong> (<em>int</em>) – </p></li>
 <li><p><strong>replay_path</strong> (<em>str</em><em> | </em><em>None</em>) – </p></li>
 <li><p><strong>alias</strong> (<em>str</em><em> | </em><em>None</em>) – </p></li>
@@ -192,7 +193,7 @@ description: "Config file reference"
 
 <dl class="py class">
 <dt class="sig sig-object py" id="HttpConfig">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre"></span></span><span class="sig-name descname"><span class="pre">HttpConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">retry_count</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_multiplier</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_rate</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_period</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_limit</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">replay_path</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">alias</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#HttpConfig" title="Link to this definition">¶</a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre"></span></span><span class="sig-name descname"><span class="pre">HttpConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">retry_count</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">retry_multiplier</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_rate</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_period</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratelimit_sleep</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_limit</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">connection_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">request_timeout</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">replay_path</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">alias</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#HttpConfig" title="Link to this definition">¶</a></dt>
 <dd><p>Advanced configuration of HTTP client</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -205,6 +206,7 @@ description: "Config file reference"
 <li><p><strong>ratelimit_sleep</strong> (<em>float</em><em> | </em><em>None</em>) – Sleep time between requests when rate limit is reached</p></li>
 <li><p><strong>connection_limit</strong> (<em>int</em><em> | </em><em>None</em>) – Number of simultaneous connections</p></li>
 <li><p><strong>connection_timeout</strong> (<em>int</em><em> | </em><em>None</em>) – Connection timeout in seconds</p></li>
+<li><p><strong>request_timeout</strong> (<em>int</em><em> | </em><em>None</em>) – Request timeout in seconds</p></li>
 <li><p><strong>batch_size</strong> (<em>int</em><em> | </em><em>None</em>) – Number of items fetched in a single paginated request</p></li>
 <li><p><strong>replay_path</strong> (<em>str</em><em> | </em><em>None</em>) – Use cached HTTP responses instead of making real requests (dev only)</p></li>
 <li><p><strong>alias</strong> (<em>str</em><em> | </em><em>None</em>) – Alias for this HTTP client (dev only)</p></li>