Increase node heap size for adiff service #49

jake-low · 2025-01-02T18:49:04Z

Trying to fix this issue that appeared today:

λ kubectl logs osmcha-adiff-service-worker-66d4ff6645-66xjx
yarn run v1.22.19
$ node ./update.js
-- STARTING PROCESS AT Thu Jan 02 2025 18:44:34 GMT+0000
(node:28) NOTE: The AWS SDK for JavaScript (v2) will enter maintenance mode
on September 8, 2024 and reach end-of-support on September 8, 2025.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check blog post at https://a.co/cUPnyil
(Use `node --trace-warnings ...` to show where the warning was created)
Queueing replication files from 6391217 to 6413126 to process.

<--- Last few GCs --->

[28:0x7fe6d04530c0]    41485 ms: Mark-sweep (reduce) 2038.5 (2083.8) -> 2037.8 (2084.3) MB, 2033.5 / 0.0 ms  (average mu = 0.160, current mu = 0.002) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
error Command failed with signal "SIGABRT".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

github-actions · 2025-01-02T18:49:51Z

Terraform Format and Style 🖌`success`

Terraform Initialization ⚙️`success`

Terraform Plan 📖`success`

Terraform Validation 🤖`success`

Show Plan


terraform
module.resources.random_password.django_secret_key: Refreshing state... [id=none]
module.resources.module.eks.data.aws_caller_identity.current: Reading...
module.resources.module.eks.module.kms.data.aws_caller_identity.current[0]: Reading...
module.resources.module.eks.module.kms.data.aws_partition.current[0]: Reading...
module.resources.module.eks.aws_cloudwatch_log_group.this[0]: Refreshing state... [id=/aws/eks/osmcha-production-cluster/cluster]
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_partition.current: Reading...
module.resources.module.eks.data.aws_partition.current: Reading...
module.resources.data.aws_caller_identity.current: Reading...
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_partition.current: Reading...
module.resources.data.aws_availability_zones.available: Reading...
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_partition.current: Read complete after 0s [id=aws]
module.resources.module.eks.module.kms.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_partition.current: Read complete after 0s [id=aws]
module.resources.module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-0ff6ba7829e56e010]
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_caller_identity.current: Reading...
module.resources.module.eks.data.aws_partition.current: Read complete after 0s [id=aws]
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_caller_identity.current: Reading...
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_iam_policy_document.assume_role_policy[0]: Reading...
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_iam_policy_document.assume_role_policy[0]: Reading...
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=2560088296]
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=2560088296]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_iam_role.this[0]: Refreshing state... [id=default-eks-node-group-20231107054922198000000002]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_iam_role.this[0]: Refreshing state... [id=regular-eks-node-group-20231107054922197700000001]
module.resources.module.eks.data.aws_caller_identity.current: Read complete after 0s [id=003081160852]
module.resources.module.eks.module.kms.data.aws_caller_identity.current[0]: Read complete after 0s [id=003081160852]
module.resources.module.eks.data.aws_iam_session_context.current: Reading...
module.resources.module.eks.data.aws_iam_session_context.current: Read complete after 0s [id=arn:aws:iam::003081160852:user/devseed]
module.resources.module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Reading...
module.resources.module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=2764486067]
module.resources.module.eks.aws_iam_role.this[0]: Refreshing state... [id=osmcha-production-cluster-cluster-20231107054922198200000003]
module.resources.data.aws_caller_identity.current: Read complete after 0s [id=003081160852]
module.resources.module.eks.module.eks_managed_node_group["default"].data.aws_caller_identity.current: Read complete after 0s [id=003081160852]
module.resources.module.eks.module.eks_managed_node_group["regular"].data.aws_caller_identity.current: Read complete after 0s [id=003081160852]
module.resources.data.aws_availability_zones.available: Read complete after 0s [id=us-east-1]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_iam_role_policy_attachment.additional["AmazonEBSCSIDriverPolicy"]: Refreshing state... [id=default-eks-node-group-20231107054922198000000002-20231112185603604700000001]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]: Refreshing state... [id=regular-eks-node-group-20231107054922197700000001-20231107054924804200000006]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]: Refreshing state... [id=regular-eks-node-group-20231107054922197700000001-20231107054924700700000004]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]: Refreshing state... [id=regular-eks-node-group-20231107054922197700000001-20231107054924744600000005]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]: Refreshing state... [id=default-eks-node-group-20231107054922198000000002-20231107054925140200000009]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]: Refreshing state... [id=default-eks-node-group-20231107054922198000000002-20231107054925026700000007]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_iam_role_policy_attachment.additional["AmazonEBSCSIDriverPolicy"]: Refreshing state... [id=regular-eks-node-group-20231107054922197700000001-20231112185603642800000002]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]: Refreshing state... [id=default-eks-node-group-20231107054922198000000002-20231107054925077200000008]
module.resources.module.eks.aws_iam_role_policy_attachment.this["AmazonEKSClusterPolicy"]: Refreshing state... [id=osmcha-production-cluster-cluster-20231107054922198200000003-2023110705492595170000000b]
module.resources.module.eks.aws_iam_role_policy_attachment.this["AmazonEKSVPCResourceController"]: Refreshing state... [id=osmcha-production-cluster-cluster-20231107054922198200000003-2023110705492594830000000a]
module.resources.module.eks.module.kms.data.aws_iam_policy_document.this[0]: Reading...
module.resources.module.eks.module.kms.data.aws_iam_policy_document.this[0]: Read complete after 0s [id=2924970584]
module.resources.module.eks.module.kms.aws_kms_key.this[0]: Refreshing state... [id=64f7898b-3ee0-4a8b-a0d9-f9861be71e5a]
module.resources.module.eks.module.kms.aws_kms_alias.this["cluster"]: Refreshing state... [id=alias/eks/osmcha-production-cluster]
module.resources.module.eks.aws_iam_policy.cluster_encryption[0]: Refreshing state... [id=arn:aws:iam::003081160852:policy/osmcha-production-cluster-cluster-ClusterEncryption2023110705494813260000000f]
module.resources.module.eks.aws_iam_role_policy_attachment.cluster_encryption[0]: Refreshing state... [id=osmcha-production-cluster-cluster-20231107054922198200000003-20231107054949417900000010]
module.resources.module.vpc.aws_default_security_group.this[0]: Refreshing state... [id=sg-035e596106073cf62]
module.resources.module.vpc.aws_default_route_table.default[0]: Refreshing state... [id=rtb-09de68292979ea4bc]
module.resources.module.eks.aws_security_group.cluster[0]: Refreshing state... [id=sg-0192096028bce8b9c]
module.resources.module.vpc.aws_default_network_acl.this[0]: Refreshing state... [id=acl-096d4204f9df81729]
module.resources.module.vpc.aws_subnet.private[0]: Refreshing state... [id=subnet-0225432bfc41c3460]
module.resources.module.vpc.aws_route_table.public[0]: Refreshing state... [id=rtb-09b6383788590fd4c]
module.resources.module.vpc.aws_subnet.private[2]: Refreshing state... [id=subnet-0b0cbed43c2c6d28b]
module.resources.module.vpc.aws_subnet.private[1]: Refreshing state... [id=subnet-01379fd977a915c30]
module.resources.module.vpc.aws_subnet.public[0]: Refreshing state... [id=subnet-05462743417c78890]
module.resources.module.eks.aws_security_group.node[0]: Refreshing state... [id=sg-0fa5f0d23a3464a65]
module.resources.module.vpc.aws_subnet.public[1]: Refreshing state... [id=subnet-0444a55ffd754d386]
module.resources.module.vpc.aws_subnet.public[2]: Refreshing state... [id=subnet-027a255ffd5487e5b]
module.resources.module.vpc.aws_route_table.private[0]: Refreshing state... [id=rtb-08f840c254d408f66]
module.resources.module.vpc.aws_internet_gateway.this[0]: Refreshing state... [id=igw-001ac74d2eda24d7c]
module.resources.module.eks.aws_security_group_rule.node["ingress_nodes_ephemeral"]: Refreshing state... [id=sgrule-3774878186]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_4443_webhook"]: Refreshing state... [id=sgrule-115022675]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_6443_webhook"]: Refreshing state... [id=sgrule-3482942661]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_443"]: Refreshing state... [id=sgrule-4155399891]
module.resources.module.eks.aws_security_group_rule.node["ingress_self_coredns_udp"]: Refreshing state... [id=sgrule-3012666480]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_kubelet"]: Refreshing state... [id=sgrule-2156400892]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_9443_webhook"]: Refreshing state... [id=sgrule-3093590831]
module.resources.module.eks.aws_security_group_rule.node["ingress_self_coredns_tcp"]: Refreshing state... [id=sgrule-399378335]
module.resources.module.eks.aws_security_group_rule.node["ingress_cluster_8443_webhook"]: Refreshing state... [id=sgrule-3703914724]
module.resources.module.eks.aws_security_group_rule.node["egress_all"]: Refreshing state... [id=sgrule-2348016402]
module.resources.module.eks.aws_security_group_rule.cluster["ingress_nodes_443"]: Refreshing state... [id=sgrule-274735370]
module.resources.module.vpc.aws_route_table_association.private[2]: Refreshing state... [id=rtbassoc-032b6d59b2151c32c]
module.resources.module.vpc.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-01908ea82222da23e]
module.resources.module.vpc.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0ebaaeebe09f17002]
module.resources.module.vpc.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-00993017897174a09]
module.resources.module.vpc.aws_route_table_association.public[2]: Refreshing state... [id=rtbassoc-02d547b8c1ca2216c]
module.resources.module.vpc.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-0d7852c3624f5fa5c]
module.resources.module.vpc.aws_eip.nat[0]: Refreshing state... [id=eipalloc-00acc931ff3ef4897]
module.resources.module.vpc.aws_route.public_internet_gateway[0]: Refreshing state... [id=r-rtb-09b6383788590fd4c1080289494]
module.resources.module.vpc.aws_nat_gateway.this[0]: Refreshing state... [id=nat-0fcff0cb111a56930]
module.resources.module.eks.aws_eks_cluster.this[0]: Refreshing state... [id=osmcha-production-cluster]
module.resources.module.vpc.aws_route.private_nat_gateway[0]: Refreshing state... [id=r-rtb-08f840c254d408f661080289494]
module.resources.module.eks.time_sleep.this[0]: Refreshing state... [id=2023-11-07T06:00:08Z]
module.resources.module.eks.data.tls_certificate.this[0]: Reading...
module.resources.module.eks.data.tls_certificate.this[0]: Read complete after 0s [id=99d41e43229a4cdaf4141f3e8310e6d95c31dab9]
module.resources.module.eks.data.aws_eks_addon_version.this["aws-ebs-csi-driver"]: Reading...
module.resources.module.eks.aws_iam_openid_connect_provider.oidc_provider[0]: Refreshing state... [id=arn:aws:iam::003081160852:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/C8E12DA86808CD1A27FA43DCE7E709B2]
module.resources.helm_release.osmcha-ingress-nginx: Refreshing state... [id=ingress-nginx]
module.resources.helm_release.osmcha-cert-manager: Refreshing state... [id=cert-manager]
module.resources.helm_release.osmcha-redis: Refreshing state... [id=redis]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_launch_template.this[0]: Refreshing state... [id=lt-0e8461b46962b2afb]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_launch_template.this[0]: Refreshing state... [id=lt-06fc194f152986836]
module.resources.kubernetes_secret.django_secret_key: Refreshing state... [id=default/django-secret-key]
module.resources.module.eks.data.aws_eks_addon_version.this["aws-ebs-csi-driver"]: Read complete after 0s [id=aws-ebs-csi-driver]
module.resources.module.eks.module.eks_managed_node_group["default"].aws_eks_node_group.this[0]: Refreshing state... [id=osmcha-production-cluster:default-20231107060014586500000017]
module.resources.module.eks.module.eks_managed_node_group["regular"].aws_eks_node_group.this[0]: Refreshing state... [id=osmcha-production-cluster:regular-20231107060014586200000015]
module.resources.module.eks.aws_eks_addon.this["aws-ebs-csi-driver"]: Refreshing state... [id=osmcha-production-cluster:aws-ebs-csi-driver]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.resources.module.eks.aws_eks_cluster.this[0] has changed
  ~ resource "aws_eks_cluster" "this" {
        id                        = "osmcha-production-cluster"
        name                      = "osmcha-production-cluster"
      ~ platform_version          = "eks.29" -> "eks.32"
        tags                      = {}
        # (10 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.resources.module.eks.aws_eks_addon.this["aws-ebs-csi-driver"] will be updated in-place
  ~ resource "aws_eks_addon" "this" {
      ~ addon_version     = "v1.37.0-eksbuild.1" -> "v1.38.1-eksbuild.1"
        id                = "osmcha-production-cluster:aws-ebs-csi-driver"
        tags              = {}
        # (7 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Warning: Argument is deprecated

  with module.resources.module.eks.aws_eks_addon.this["aws-ebs-csi-driver"],
  on .terraform/modules/resources.eks/main.tf line 392, in resource "aws_eks_addon" "this":
 392:   resolve_conflicts        = try(each.value.resolve_conflicts, "OVERWRITE")

The "resolve_conflicts" attribute can't be set to "PRESERVE" on initial
resource creation. Use "resolve_conflicts_on_create" and/or
"resolve_conflicts_on_update" instead

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Pusher: @jake-low, Action: pull_request

Increase node heap size for adiff service

9f4d2c6

jake-low merged commit 8a6c458 into main Jan 2, 2025
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Increase node heap size for adiff service #49

Increase node heap size for adiff service #49

jake-low commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

Increase node heap size for adiff service #49

Increase node heap size for adiff service #49

Conversation

jake-low commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

Terraform Format and Style 🖌success

Terraform Initialization ⚙️success

Terraform Plan 📖success

Terraform Validation 🤖success

Terraform Format and Style 🖌`success`

Terraform Initialization ⚙️`success`

Terraform Plan 📖`success`

Terraform Validation 🤖`success`