You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we handle tablet split by modifying the polling list in place where we remove the parent tablet from the list and add the child tablets.
The above logic causes an issue in case one of the task is restarted. Let's say when we start the connector, we divide the tablets into task configs. Now this tablet list gets interpreted at the task level and is polled onto. Now let's say a tablet A gets split into B and C, we will modify the polling list but the task config remains the same - in this case, if a task is restarted, we will again try to poll on the parent tablet A (which may be deleted by this time now) and thus the task will start throwing errors.
The proposed solution over here is that we modify the task configuration itself and let the Kafka Connect runtime restart the connector so that we start the whole process from the top again.
The text was updated successfully, but these errors were encountered:
Currently we handle tablet split by modifying the polling list in place where we remove the parent tablet from the list and add the child tablets.
The above logic causes an issue in case one of the task is restarted. Let's say when we start the connector, we divide the tablets into task configs. Now this tablet list gets interpreted at the task level and is polled onto. Now let's say a tablet A gets split into B and C, we will modify the polling list but the task config remains the same - in this case, if a task is restarted, we will again try to poll on the parent tablet A (which may be deleted by this time now) and thus the task will start throwing errors.
The proposed solution over here is that we modify the task configuration itself and let the Kafka Connect runtime restart the connector so that we start the whole process from the top again.
The text was updated successfully, but these errors were encountered: