Time outs on long running application

[michaelst]

I've seen various errors from time outs to :noproc after a couple of days of my app running. Here is the latest error I received. Let me know what else I can provide to help debug this.

** (exit) exited in: GenServer.call(Weddell.Client, {:publish, <<8, 17, 18, 7, 87, 97, 121, 102, 97, 105, 114, 26, 7, 36, 50, 49, 52, 46, 52, 56>>, "spendable.send-notification-request"}, 5000)
    ** (EXIT) time out
    (elixir 1.10.4) lib/gen_server.ex:1023: GenServer.call/3
    (spendable 0.1.0) lib/spendable/broadway/sync_member/sync_member.ex:117: Spendable.Broadway.SyncMember.sync_transaction/2
    (elixir 1.10.4) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2
    (spendable 0.1.0) lib/spendable/broadway/sync_member/sync_member.ex:89: Spendable.Broadway.SyncMember.sync_transactions/2
    (elixir 1.10.4) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2
    (broadway 0.6.2) lib/broadway/message.ex:48: Broadway.Message.update_data/2
    (broadway 0.6.2) lib/broadway/topology/processor_stage.ex:158: Broadway.Topology.ProcessorStage.handle_messages/4
    (broadway 0.6.2) lib/broadway/topology/processor_stage.ex:53: Broadway.Topology.ProcessorStage.handle_events/3

[michaelst]

It looks like the connection on the channel is getting closed but there is nothing to reinitialize it.

  def connect(project, opts \\ []) do
    scheme = Keyword.get(opts, :scheme, :https)
    ssl = ssl_opts(opts)
    cred = if scheme == :https, do: GRPC.Credential.new(ssl: ssl), else: nil
    host = Keyword.get(opts, :host, @default_host)
    port = Keyword.get(opts, :port, @default_port)
    {:ok, channel} =
      Stub.connect("#{host}:#{port}", cred: cred, adapter_opts: %{
        http2_opts: %{keepalive: :infinity}
      })
    {:ok, %__MODULE__{channel: channel,
                      project: project}}
  end

[medikent]

@michaelst I have seen these errors as well though I never got to the root cause. Any insight there?