Using the lookup mapping strategy, LDAP proxy runs out of open files for invalid DNs

[fredreichbier]

We can reproduce this using locust:

• Use lookup mapping strategy on the LDAP proxy
• Edit locustfile.py and set USER_DN to a DN that does not exist in the LDAP backend
• Run locust analysis with 100 users and a hatch rate of 10
• Switch to Failures tab
• At first, all failures are '49: Invalid user.' as expected
• However, after some time, all new failures are '49: LDAP Proxy failed.'
• The server log reveals:

Feb 10 16:21:38 test twistd[32313]: 2017-02-10T16:21:38+0100 [pi_ldapproxy.proxy.ProxyServerFactory] Could not accept new connection (EMFILE)

• lsof -c twistd shows >1000 open connections from the LDAP proxy to the LDAP backend

[fredreichbier]

With f64e93c applied, the LDAP proxy seems to be able to handle >3000 bind requests with invalid DNs. However, it still fails with EMFILE at some point.

We should also have a closer look at this issue with regard to successful bind requests (is the connection between LDAP proxy and LDAP backend terminated properly?)

[fredreichbier]

• If the incoming bind request has invalid credentials, the LDAP proxy seems to properly close the connection to the LDAP backend
• However, if it fails due to some other error (e.g. a malformed DN), the connection between LDAP proxy and backend seems to persist

[fredreichbier]

I think 6b73309 has fixed the issue mentioned above (i.e. that the connection between LDAP proxy and backend persists if the DN is malformed). There seem to be cases in which the connection persists nevertheless, but it seems to be closed after a timeout. I suspect this could rather be a side-effect of our load testing approach using locust.

There seems to be a remaining issue if privacyIDEA answers with HTTP status codes != 200 (e.g. 500): In that case, HTTP connections between LDAP proxy and privacyIDEA are kept open for a pretty long time -- but they seem to get closed after a certain timeout.