RDoc-2571 - set data subscription worker timeout

ravendb · Feb 4, 2024 · 6abc80e · 6abc80e
1 parent b6543e0
commit 6abc80e
Show file tree

Hide file tree

Showing 5 changed files with 245 additions and 3 deletions.
diff --git a/...data-subscriptions/consumption/how-to-consume-data-subscription.dotnet.markdown b/...data-subscriptions/consumption/how-to-consume-data-subscription.dotnet.markdown
@@ -0,0 +1,114 @@
+# Data Subscriptions: How to Consume a Data Subscription
+
+---
+
+{NOTE: }
+
+Subscriptions are consumed by processing batches of documents received from the server. 
+A `SubscriptionWorker` object manages the documents processing and the communication between the client and the server according to a set of configurations received upon it's creation. 
+We've introduced several ways to create and configure a SubscriptionWorker, starting from just giving a subscription name, and ending with a detailed configuration object - `SubscriptionWorkerOptions`.
+
+In this page:
+
+[SubscriptionWorker lifecycle](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#subscriptionworker-lifecycle)  
+[Error handling](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#error-handling)  
+[Worker interplay](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#worker-interplay)
+
+{NOTE/}
+
+---
+
+{PANEL:SubscriptionWorker lifecycle}
+A `SubscriptionWorker` object starts its life from being generated by the `DocumentsStore.Subscriptions`:
+
+{CODE subscription_open_simple@ClientApi\DataSubscriptions\DataSubscriptions.cs /}
+
+At this point, the worker has only got its configuration. No connection or processing happens at this moment. 
+In order to start processing, the `Run` method should be called. The Run method receives the batch processing logic that should be performed:
+
+{CODE subscription_run_simple@ClientApi\DataSubscriptions\DataSubscriptions.cs /}
+
+From this point on, the subscription worker will start processing batches. If for any reason, the processing is aborted, and the returned task (`subscriptionRuntimeTask`) will be finished with an exception.
+
+{PANEL/}
+
+
+{PANEL:Error handling}
+There are two categories of errors that may occur during subscription processing:
+
+{INFO:Internal mechanism errors}
+Those errors occur during the normal server-client communication between the worker and the server.  
+If an unexpected error occurs, the worker will try to reconnect to the server. There are conditions in which the worker will cease its operation and will not try to reconnect:  
+
+* The subscription does not exist or was deleted  
+* Another worker took over the subscription (see connection strategy)
+* The worker could not connect to any of the servers
+* The worker could not receive the node in charge of the task (this can happen when there is no leader)
+* Authorization exception
+* Exception during connection establishment
+
+{INFO/}
+
+{INFO:User's batch processing logic unhandled exception}
+Example:
+{CODE throw_during_user_logic@ClientApi\DataSubscriptions\DataSubscriptions.cs /}
+
+If an exception is thrown, the worker will abort the current batch process. 
+A worker can be configured to treat the thrown exception by either of the following two ways:  
+
+* By default, the worker will wrap the thrown exception with a `SubscriberErrorException` exception and rethrow it,  
+  terminating the subscription execution without acknowledging progress or retrying. The task returned by the `Run` function will
+  be terminated with an erroneous state, throwing a `SubscriberErrorException` exception.
+
+* If `SubscriptionWorkerOptions`'s value `IgnoreSubscriberErrors` is set to true, the erroneous batch will get acknowledged without retrying and the next batches will continue processing. 
+{INFO/}
+
+{INFO: Reconnecting}
+In the cases above, we described situations in which a worker will try to reconnect with the server. There are two key `SubscriptionWorkerOptions` fields controlling this state:
+
+*  `TimeToWaitBeforeConnectionRetry` - The time that the worker will 'sleep' before trying to reconnect.
+* `MaxErroneousPeriod` - The maximum time in which the worker is allowed to be in an erroneous state. After that time passes, the worker will stop trying to reconnect.
+{INFO/}
+
+{INFO: Timing out}
+If a worker fails to connect the server for a given time period, it will throw an exception and stop trying.  
+
+* Timeout period: `ConnectionStreamTimeout`  
+* Exception generated when the timeout period is exceeded: `OperationCanceledException`  
+* Default timeout period: 30 second  
+
+Example:  
+{CODE worker_timeout@ClientApi\DataSubscriptions\DataSubscriptions.cs /}
+
+{INFO/}
+
+{INFO: OnUnexpectedSubscriptionError}
+`OnUnexpectedSubscriptionError` is the event raised when a connection failure occurs 
+between the subscription worker and the server and it throws an unexpected exception. 
+When this occurs, the worker will automatically try to reconnect again. This event is 
+useful for logging these unexpected exceptions.
+{INFO/}
+
+{PANEL/}
+
+{PANEL: Worker interplay}
+There can only be one active subscription worker working on a subscription. 
+Nevertheless, there are scenarios where it is required to interact between an existing subscription worker and one that tries to connect. 
+This relationship and interoperation is configured by the `SubscriptionWorkerOptions` `Strategy` field.  
+The strategy field is the enum `SubscriptionOpeningStrategy`, which has the following values:  
+
+* `OpenIfFree` - the server will allow the worker to connect only if there isn't any other currently connected workers.  
+  If there is a existing connection, the incoming worker will throw a `SubscriptionInUseException`.  
+* `WaitForFree` - If the client currently cannot open the subscription because it is used by another client, it will wait for the previous client to disconnect and only then will connect.  
+  This is useful in client failover scenarios where there is one active client and another one already waiting to take its place.  
+* `TakeOver` - the server will allow an incoming connection to overthrow an existing one. It will behave according to the existing connection strategy:
+  * The existing connection has a strategy that is not `TakeOver`. In this case, the incoming connection will take over it causing the existing connection to throw a `SubscriptionInUseException`.  
+  * The existing connection has a strategy that is `TakeOver`. In this case, the incoming connection will throw a SubscriptionInUseException exception.  
+{PANEL/}
+
+## Related Articles  
+
+### Data Subscriptions  
+
+- [What are Data Subscriptions](../../../client-api/data-subscriptions/what-are-data-subscriptions)  
+- [How to Create a Data Subscription](../../../client-api/data-subscriptions/creation/how-to-create-data-subscription)  
diff --git a/...i/data-subscriptions/consumption/how-to-consume-data-subscription.java.markdown b/...i/data-subscriptions/consumption/how-to-consume-data-subscription.java.markdown
@@ -0,0 +1,97 @@
+# Data Subscriptions: How to Consume a Data Subscription
+
+---
+
+{NOTE: }
+
+Subscriptions are consumed by processing batches of documents received from the server. 
+A `SubscriptionWorker` object manages the documents processing and the communication between the client and the server according to a set of configurations received upon it's creation. 
+We've introduced several ways to create and configure a SubscriptionWorker, starting from just giving a subscription name, and ending with a detailed configuration object - `SubscriptionWorkerOptions`.
+
+In this page:
+
+[SubscriptionWorker lifecycle](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#subscriptionworker-lifecycle)  
+[Error handling](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#error-handling)  
+[Workers interplay](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription#workers-interplay)
+
+{NOTE/}
+
+---
+
+{PANEL:SubscriptionWorker lifecycle}
+A `SubscriptionWorker` object starts its life from being generated by the `DocumentsStore.subscriptions`:
+
+{CODE:java subscription_open_simple@ClientApi\DataSubscriptions\DataSubscriptions.java /}
+
+At this point, the worker has only got its configuration. No connection or processing happens at this moment. 
+In order to start processing, the `run` method should be called. The `run` method receives the batch processing logic that should be performed:
+
+{CODE:java subscription_run_simple@ClientApi\DataSubscriptions\DataSubscriptions.java /}
+
+From this point on, the subscription worker will start processing batches. If for any reason, the processing is aborted, and the returned task (`subscriptionRuntimeTask`) will be finished with an exception.
+
+{PANEL/}
+
+
+{PANEL:Error handling}
+There are two categories of errors that may occur during subscription processing:
+
+{INFO:Internal mechanism errors}
+Those errors occur during the normal server-client communication between the worker and the server.  
+If an unexpected error occurs, the worker will try to reconnect to the server. There are conditions in which the worker will cease its operation and will not try to reconnect:  
+
+* The subscription does not exist or was deleted  
+* Another worker took over the subscription (see connection strategy)
+* The worker could not connect to any of the servers
+* The worker could not receive the node in charge of the task (this can happen when there is no leader)
+* Authorization exception
+* Exception during connection establishment
+
+{INFO/}
+
+{INFO:User's batch processing logic unhandled exception}
+Example:
+{CODE:java throw_during_user_logic@ClientApi\DataSubscriptions\DataSubscriptions.java /}
+
+If an exception is thrown, the worker will abort the current batch process. 
+A worker can be configured to treat the thrown exception by either of the following two ways:  
+
+* By default, the worker will wrap the thrown exception with a `SubscriberErrorException` exception and rethrow it,  
+  terminating the subscription execution without acknowledging progress or retrying. The task returned by the `Run` function will
+  be terminated with an erroneous state, throwing a `SubscriberErrorException` exception.
+
+* If `SubscriptionWorkerOptions`'s value `ignoreSubscriberErrors` is set to true, the erroneous batch will get acknowledged without retrying and the next batches will continue processing. 
+{INFO/}
+
+{INFO: Reconnecting}
+In the cases above, we described situations in which a worker will try to reconnect with the server. There are two key `SubscriptionWorkerOptions` fields controlling this state:
+
+*  `timeToWaitBeforeConnectionRetry` - The time that the worker will 'sleep' before trying to reconnect.
+* `maxErroneousPeriod` - The maximum time in which the worker is allowed to be in erroneous state. After that time passes, the worker will stop trying to reconnect
+{INFO/}
+
+
+{PANEL/}
+
+{PANEL: Workers interplay}
+There can only be one active subscription worker working on a subscription. 
+Nevertheless, there are scenarios where it is required to interact between an existing subscription worker and one that tries to connect. 
+This relationship and interoperation is configured by the `SubscriptionConnectionOptions` `Strategy` field.  
+The strategy field is an enum, having the following values:  
+
+* `OPEN_IF_FREE` - the server will allow the worker to connect only if there isn't any other currently connected workers.  
+  If there is a existing connection, the incoming worker will throw a SubscriptionInUseException.  
+* `WAIT_FOR_FREE` - If the client currently cannot open the subscription because it is used by another client, it will wait for the previous client to disconnect and only then will connect.  
+  This is useful in client failover scenarios where there is one active client and another one already waiting to take its place.  
+* `TAKE_OVER` - the server will allow an incoming connection to overthrow an existing one. It will behave according to the existing connection strategy:
+  * The existing connection has a strategy that is not `TAKE_OVER`. In this case, the incoming connection will take over it causing the existing connection to throw a SubscriptionInUseException exception.  
+  * The existing connection has a strategy that is `TAKE_OVER`. In this case, the incoming connection will throw a SubscriptionInUseException exception.  
+{PANEL/}
+
+## Related Articles
+
+**Data Subscriptions**:
+
+- [What are Data Subscriptions](../../../client-api/data-subscriptions/what-are-data-subscriptions)
+- [How to Create a Data Subscription](../../../client-api/data-subscriptions/creation/how-to-create-data-subscription)
+- [How to Consume a Data Subscription](../../../client-api/data-subscriptions/consumption/how-to-consume-data-subscription)
diff --git a/...mples/csharp/Raven.Documentation.Samples/ClientApi/DataSubscriptions/DataSubscriptions.cs b/...mples/csharp/Raven.Documentation.Samples/ClientApi/DataSubscriptions/DataSubscriptions.cs
@@ -583,10 +583,19 @@ e is SubscriptionInvalidStateException ||
                     subscriptionWorker.Dispose();
                 }
             }
+            #endregion
 
+            while (true)
+            {
+                #region worker_timeout
+                var options = new SubscriptionWorkerOptions(subscriptionName);
 
-            #endregion
+                // Set the orker's timeout period
+                options.ConnectionStreamTimeout = TimeSpan.FromSeconds(45);
 
+                subscriptionWorker = store.Subscriptions.GetSubscriptionWorker<Order>(options);
+                #endregion
+            }
 
             async Task ProcessOrder(Order o)
             {

diff --git a/...data-subscriptions/consumption/how-to-consume-data-subscription.dotnet.markdown b/...data-subscriptions/consumption/how-to-consume-data-subscription.dotnet.markdown
@@ -74,7 +74,19 @@ A worker can be configured to treat the thrown exception in either of the follow
 In the cases above, we described situations in which a worker will try to reconnect with the server. There are two key `SubscriptionWorkerOptions` fields controlling this state:
 
 *  `TimeToWaitBeforeConnectionRetry` - The time that the worker will 'sleep' before trying to reconnect.
-* `MaxErroneousPeriod` - The maximum time in which the worker is allowed to be in an erroneous state. After that time passes, the worker will stop trying to reconnect
+* `MaxErroneousPeriod` - The maximum time in which the worker is allowed to be in an erroneous state. After that time passes, the worker will stop trying to reconnect.
+{INFO/}
+
+{INFO: Timing out}
+If a worker fails to connect the server for a given time period, it will throw an exception and stop trying.  
+
+* Timeout period: `ConnectionStreamTimeout`  
+* Exception generated when the timeout period is exceeded: `OperationCanceledException`  
+* Default timeout period: 30 second  
+
+Example:  
+{CODE worker_timeout@ClientApi\DataSubscriptions\DataSubscriptions.cs /}
+
 {INFO/}
 
 {INFO: OnUnexpectedSubscriptionError}

diff --git a/...mples/csharp/Raven.Documentation.Samples/ClientApi/DataSubscriptions/DataSubscriptions.cs b/...mples/csharp/Raven.Documentation.Samples/ClientApi/DataSubscriptions/DataSubscriptions.cs
@@ -677,9 +677,19 @@ e is SubscriptionInvalidStateException ||
                 }
             }
 
-
             #endregion
 
+            while (true)
+            {
+                #region worker_timeout
+                var options = new SubscriptionWorkerOptions(subscriptionName);
+
+                // Set the orker's timeout period
+                options.ConnectionStreamTimeout = TimeSpan.FromSeconds(45);
+
+                subscriptionWorker = store.Subscriptions.GetSubscriptionWorker<Order>(options);
+                #endregion
+            }
 
             async Task ProcessOrder(Order o)
             {