Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry_policy session property docs #21520

Merged
merged 1 commit into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/.vale/config/vocabularies/Base/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ CPU
Datafaker
DNS
ETL
GCS
Guice
gzip
hdfs
HDFS
ipsum
JDBC
Expand All @@ -38,5 +40,6 @@ rowId
SPI
subnet
TLS
trino
Trino
truststore
37 changes: 20 additions & 17 deletions docs/src/main/sphinx/admin/fault-tolerant-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ fault-tolerant execution to improve query processing resilience, read

## Configuration

Fault-tolerant execution is disabled by default. To enable the feature, set the
Fault-tolerant execution is turned off by default. To enable the feature, set the
`retry-policy` configuration property to either `QUERY` or `TASK`
depending on the desired {ref}`retry policy <fte-retry-policy>`.

Expand Down Expand Up @@ -66,7 +66,10 @@ execution on a Trino cluster:
* - `retry-policy`
- Configures what is retried in the event of failure, either `QUERY` to retry
the whole query, or `TASK` to retry tasks individually if they fail. See
[retry policy](fte-retry-policy) for more information.
[retry policy](fte-retry-policy) for more information. Use the equivalent
mosabua marked this conversation as resolved.
Show resolved Hide resolved
session property `retry_policy` only on clusters configured for
fault-tolerant execution and typically only to deactivate with `NONE`, since
switching between modes on a cluster is not tested.
- `NONE`
* - `exchange.deduplication-buffer-size`
- [Data size](prop-type-data-size) of the coordinator's in-memory buffer used
Expand All @@ -76,8 +79,8 @@ execution on a Trino cluster:
the failure recovery capabilities to be fully functional" error message unless an
[exchange manager](fte-exchange-manager) is configured.
- `32MB`
* - `fault-tolerant-execution-exchange-encryption-enabled`
- Enable encryption of spooling data, see [Encryption](fte-encryption) for details.
* - `fault-tolerant-execution.exchange-encryption-enabled`
- Enable encryption of spooling data, see [Encryption](fte-encryption) for details.
Setting this property to false is not recommended if Trino processes sensitive data.
- ``true``
:::
Expand All @@ -88,13 +91,14 @@ Find further related properties in [](/admin/properties), specifically in
(fte-retry-policy)=
## Retry policy

The `retry-policy` configuration property designates whether Trino retries
entire queries or a query's individual tasks in the event of failure.
The `retry-policy` configuration property, or the `retry_policy` session
property, designates whether Trino retries entire queries or a query's
mosabua marked this conversation as resolved.
Show resolved Hide resolved
individual tasks in the event of failure.

### QUERY

A `QUERY` retry policy instructs Trino to automatically retry a query in the
event of an error occuring on a worker node. A `QUERY` retry policy is
event of an error occurring on a worker node. A `QUERY` retry policy is
recommended when the majority of the Trino cluster's workload consists of many
small queries.

Expand Down Expand Up @@ -338,9 +342,9 @@ fault-tolerant execution:
- Maximum number of partitions to use for distributed joins and aggregations,
similar in function to the ``query.max-hash-partition-count`` [query
management property](/admin/properties-query-management). It is not
recommended to increase this property value above the default of `50`, which
may result in instability and poor performance. May be overridden for the
current session with the `fault_tolerant_execution_max_partition_count`
recommended to increase this property value higher than the default of `50`,
which may result in instability and poor performance. May be overridden for
the current session with the `fault_tolerant_execution_max_partition_count`
[session property](session-properties-definition).
- `50`
- Only `TASK`
Expand Down Expand Up @@ -389,7 +393,7 @@ property to `filesystem` or `hdfs`, and set additional configuration properties
for your storage solution.

The following table lists the available configuration properties for
`exchange-manager.properties`, their default values, and which filesystem(s)
`exchange-manager.properties`, their default values, and which file systems
the property may be configured for:

:::{list-table} Exchange manager configuration properties
Expand Down Expand Up @@ -449,12 +453,11 @@ the property may be configured for:
- AWS S3, GCS
* - `exchange.s3.endpoint`
- S3 storage endpoint server if using an S3-compatible storage system that
is not AWS. If using AWS S3, this can be ignored unless HTTPS is required
by an AWS bucket policy. If TLS is required, then this property can be
set to an https endpoint such as ``https://s3.us-east-1.amazonaws.com``.
Note that TLS is redundant due to {ref}`automatic encryption <fte-encryption>`.
If using GCS, set it
to `https://storage.googleapis.com`.
is not AWS. If using AWS S3, this can be ignored unless HTTPS is required
by an AWS bucket policy. If TLS is required, then this property can be
set to an https endpoint such as ``https://s3.us-east-1.amazonaws.com``.
Note that TLS is redundant due to {ref}`automatic encryption <fte-encryption>`.
If using GCS, set it to `https://storage.googleapis.com`.
-
- Any S3-compatible storage
* - `exchange.s3.max-error-retries`
Expand Down
Loading