Migration output should provide progress to completion

[ThomasFreedman]

I'm in the process of migrating a large ipfs repo (not sure of old version of this repo, I believe it was v11. History is now gone since upgrade was started) to latest kubo, v12. There is a file in the repo root named 11-to-12-cids.txt which is 497KB long.

Here are the stats currently reported:

~/.ipfs$ ipfs repo stat
NumObjects: 1500619
RepoSize: 391498113680
StorageMax: 6600000000000
RepoPath: /home/ipfs/.ipfs
Version: fs-repo@12

I started this upgrade early Monday and it's late Wednesday now.

I see many resource errors, and the numbers start over (308 in sample below, 509 is largest I've seen). I presume that is due to the migration level sequence being done but I'm not sure:
`2022-11-30T18:34:40.016-0600 ERROR resourcemanager libp2p/rcmgr_logging.go:53 Resource limits were exceeded 308 times with error "system: cannot reserve inbound connection: resource limit exceeded".

`
I resolved these errors on another repo after it completed by changing the swarms section of the config and restarting the daemon.

I am reluctant to stop the migration on this repo before it completes, but would like to and then resume it unless that will lengthen the time to finish it.

[Jorropo]

I'm a bit confused by your logs, if you see libp2p errors (which is a known unrelated issue btw ipfs/kubo#9432) that means Kubo is running, if Kubo is running that means the migration finished ? or at least isn't running.

We don't support migrating concurrently to running the node.

[ThomasFreedman]

I see 8 migration processes running:

~$ ps aux | grep migrat
root        12  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/0]
root        16  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/1]
root        21  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/2]
root        26  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/3]
root        31  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/4]
root        36  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/5]
root        41  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/6]
root        46  0.0  0.0      0     0 ?        S    Nov28   0:00 [migration/7]

The fact that I see modifications to files shows it was started. It was initiated by ipfs-update. The ipfs repo stat command was issued after the ipfs-update.

I also see a ipfs daemon process, thought that was part of the migration. I decided to take the risk and kill it, b/c the systemd service said the last start failed and it wasn't running. systemctl stop ipfs had no affect on the messages.

I stopped the process with kill, it exited cleanly. Restarted using systemctl and no errors reported, no messages in journalctl logs. All appears to be good now, ipfs repo stat is fine.

The key for resolving the messages is the changes to the swarm section suggested in another issue:

    "ResourceMgr": {
      "Limits": {
        "System": {
          "Memory": 1073741824,
          "FD": 1024,
          "Conns": 1024,
          "ConnsInbound": 1024,
          "ConnsOutbound": 1024,
          "Streams": 16384,
          "StreamsInbound": 4096,
          "StreamsOutbound": 16384
          }
      }
 }

[ThomasFreedman]

Problem solved as described.