referral analysis issue

[rewrew21]

Current Behavior

I am trying to run a referral analysis FROM 30'000 households TO 15 facilities. To save time, I've checked the "permute groups" box. But when I launch the analysis it crashes after a few seconds with the following error message: "2 nodes produced errors; first error: cannot open the connectionCallcheckForRemoteErrors(val)".

I have tried this analysis on MAC and Windows environments (both latest versions with the latest updates), and both times with the latest AM version on Docker (with latest update) AND the latest version of AM on VirtualBox (with latest udpate). I've also played around with the number of cores used and ram, still no luck

However, when I run the analysis without checking the "permute groups" box, it works fine (I stopped the analysis after an hour because it would have taken more than 4 days).

Finally, I tried to run the analysis FROM the 15 facilities TO the 30'000 households (isotropic) without the "permute groups" box checked, it crashed as well.

So I'm wondering if the issue is with the very high number of facility points (or households here)...

Thanks in advance for your help!
Andrew

[fxi]

Thanks for reporting this.

It should handle this fine.

I remember something – maybe related – occurring when subsetting a large number of facilities. Solved by : e504dce, issue #209

What you can do :

• Try with half the number, e.g. 15'000 instead of 30'000, or 20'000 ( don't select manually, use the menu above the table )
• Try with another id column, e.g. cat.
• If you can look at your log – or even better – drop your logs files here. That could help.
• Share your project in our Dropbox, so I can reproduce the issue and test the fix

---

Notes for myself:

The error text reported was not the real issue.

Internally, the referral module also use the faulty method e.g. here

[rewrew21]

Hi Fred,

So I tried a few things listed below:

• run on all 30'000, change ID (once with "cat", once with "id") --> fail
• run on the 29'000 FIRST facilities -->fail
• run on the 20'000 FIRST facilities listed --> SUCCESS!!! (and in under 5 minutes...)
• run on the 20'000 LAST facilities --> fail
• run on the 10'000 LAST facilities --> fail

I've sent you an e-mail with a link to my project and logs.
Let me know if you need more info.

Thanks!

[fxi]

Thanks.

So, this seems to be linked to the request size, as 10'000 LAST would still require a lots of bytes in the the SQL string :
SELECT cat in c( "20000","20001",...,"29999","30000")
Huge request. Hard to use logical comparator, as the user could have excluded a set within, eg exclude id=27891.
Other possibility, issue with some specific feature in range 20'001 > 30'000
So... I will look at it later, but I'm a bit overwhelmed by the VirtualBox update strategy today..

[fxi]

Should be solved in 5.8.1-alpha.6

[nicolasray]

@rewrew21 : great if you can check that it is now working for you in the latest version 5.8.1-alpha.6 (or 5.8.1 at the end of this week).