Support pip

[mauritsvanrees]

We have the old PR #2 that adds support for pip. But it was never merged, and I am not sure if it works, and it has eight merge conflicts that would need to be resolved. So we may want to start over again, keeping that PR as a source of ideas and solutions.

I and others have done fixes over the years which may contain parts of that PR.
In Plone I have done some fixes so there are no core packages anymore that use the includeDependencies directive. If you wait for Plone 5.2.1 to come out, and you use the correct constraints, you can pip install Plone and it works.

I think we (Plone conference sprint Ferrara 2019) concluded that the setup.py plugin system still works: ZCML gets included.

But add-ons may still have includeDependencies in their zcml, and that does not work. You run into errors at startup for example when loading the zcml of zope.intid and it complains that the factory directive is unknown, though that is in the core of the zope zcml namespace.

And: it becomes much slower. The distributionForPackage or distributionForDottedName are looking in far too many places, taking many seconds.

Just running the utils.txt tests with the standard buildout is very fast:

$ time bin/test -t utils.txt
...
  Ran 1 tests with 0 failures, 0 errors and 0 skipped in 5.335 seconds.
...
real	0m5.670s
user	0m3.544s
sys	0m1.967s

With a virtualenv and pip install Plone, the tests take much much longer:

$ time bin/test -t utils.txt
...
  Ran 1 tests with 0 failures, 0 errors and 0 skipped in 4 minutes 46.169 seconds.
...
real	5m1.631s
user	3m33.626s
sys	1m27.483s

So 5 minutes instead of 5 seconds.

The whole distributionForDottedName could possibly be replaced by pkg_resources.get_distribution, but this may miss some of the corner cases that are currently caught. See the tests in utils.txt.

Someone may need to read and understand the pkg_resources documentation.

cc @jensens @tschorr