Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hook gets deleted Pod in snapshot #210

Open
diafour opened this issue Nov 18, 2020 · 2 comments
Open

Hook gets deleted Pod in snapshot #210

diafour opened this issue Nov 18, 2020 · 2 comments
Labels
bug Something isn't working
Milestone

Comments

@diafour
Copy link
Contributor

diafour commented Nov 18, 2020

++ jq -r .snapshots.pods
+ pods='[
  {
    "filterResult": {
      "hostIP": null,
      "initialHostIP": "192.168.xxx.yyy",
      "name": "NAME-6c97fcc5f4-mh6f5"
    }
  },
  {
    "filterResult": {
      "hostIP": "192.168.199.135",
      "initialHostIP": null,
      "name": "NAME-6c97fcc5f4-mnx2c"
    }
  }
]'

kubectl:

root@kube-master-0 / # kubectl get pods -l app=NAME
NAME                         READY   STATUS    RESTARTS   AGE
NAME-6c97fcc5f4-mnx2c   0/1     Running   0          9h

Configuration:

configVersion: v1
    kubernetes:
    - name: pods
      group: main
      keepFullObjectsInMemory: false
      executeHookOnEvent: ["Added", "Modified"]
      executeHookOnSynchronization: true
      apiVersion: v1
      kind: Pod
      namespace:
        nameSelector:
          matchNames: ["namespace-name"]
      labelSelector:
        matchLabels:
          app: "NAME"
      jqFilter: |
        {
          "name": .metadata.name,
          "hostIP": .status.hostIP,
          "initialHostIP": .metadata.annotations."NAME/initial-host-ip"
        }

The first intention is a snapshot not invalidated properly because of the logic behind the option keepFullObjectsInMemory: false.

@diafour
Copy link
Contributor Author

diafour commented Dec 15, 2020

Update after some research.

There is a possibility to get nil from DeletedFinalStateUnknown.Obj: https://github.com/flant/shell-operator/blob/master/pkg/kube_events_manager/resource_informer.go#L271-L274. The best we can do here is to use same KeyFunc for index in CachedResources. For Event mode we can mimic Informers behavior and use cached filterResult and/or object with additional "stale" flag in binding context.

@diafour diafour added the bug Something isn't working label Dec 15, 2020
@diafour
Copy link
Contributor Author

diafour commented Dec 15, 2020

Also, we should consider implementing custom caching for Informers to make keepFullObjectsInMemory: false really working.

@diafour diafour added this to the 1.1.0 milestone Mar 31, 2021
@diafour diafour modified the milestones: 1.1.0, future Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant