Implement threshold for the intersection of lists of validators

[Bronek]

High Level Overview of Change

Improve UNL security by allowing validators to set a minimum number of UNL publishers to agree on validators

Context of Change

In the current system, every node builds an internal list of validators to use, by performing union of UNLs received from all configured UNL publishers (validator lists), where the authenticity of each single UNL is verified by a manifest with a known master key.

This union is a weak security spot of the UNL publication system, as it allows a single UNL publisher to add an arbitrary number of validators, which will be subsequently used by all nodes trusting this UNL publisher. Currently the risk is managed by recommending only the use of two UNL publishers: vl.xrplf.org and vl.ripple.com . With this union approach, a hypothetical addition of a third (or more) UNL publisher would only increase the attack surface, in this way (counterintuitively) degrading the UNL system security.

This PR is replacing union with a configurable intersection threshold between UNL publishers, which by default is calculated as:

threshold =  size(validator_list_keys) < 3
  ? 1
  : floor(size(validator_list_keys) / 2) + 1

This means that for a default 2 UNL publishers, the default threshold will be 1. Since "intersection of 1 sets" is not an intersection (it's an union) this makes the default behaviour (for one or two UNL publishers) identical to the current union system.

If a node configuration contains a third UNL publisher, the default threshold will be calculated as floor(3 / 2) + 1, that is 1 + 1 that is 2. This means that, for 3 validator lists, the node will only use validators which are present on 2 or more lists (and will silently ignore validators which are on one list only).

If the configured validator list contains 4 or 5 publishers, the default threshold will be 3. If the configured validator list contains 6 or 7 publishers, the default threshold will be 4 etc.

This PR adds an optional configuration option [validator_list_threshold] to validators.txt which can be explicitly set to the minimum number of the lists on which a validator must be listed in order to be used (this number must not be greater than the size of [validator_list_keys]). If it is not set, or set to 0, the value will be calculated at startup from the size of [validator_list_keys] as explained above.

The actually used threshold value will be also displayed in the output of validators RPC method (alongside with the used list of validators, and the content of each configured validator list, as is the current output).

This change is not altering the list of the default UNL publishers. The intent of this PR is to make it safe to extend this list in the future.

---

The security problems of the union UNL publication system were originally described by @mDuo13 and the idea to use intersection of validator lists as a solution came from @ximinez

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Refactor (non-breaking change that only restructures code)
• Performance (increase or change in throughput and/or latency)
• Tests (you added tests for code that already exists, or your new feature included in this PR)
• Documentation update
• Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
• Release

API Impact

Add new value validator_list_threshold to the output from validators RPC method

Add new (optional) section [validator_list_threshold] to validators file.

• Public API: New feature (new methods and/or new fields)
• Public API: Breaking change (in general, breaking changes should only impact the next api_version)
• libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
• Peer protocol change (must be backward compatible or bump the peer protocol version)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.0%. Comparing base (3868c04) to head (a5e70c0).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #5112   +/-   ##
=======================================
  Coverage     78.0%   78.0%           
=======================================
  Files          789     789           
  Lines        66952   67007   +55     
  Branches      8110    8108    -2     
=======================================
+ Hits         52214   52277   +63     
+ Misses       14738   14730    -8     

Files with missing lines	Coverage Δ
src/xrpld/app/main/Application.cpp	68.8% <100.0%> (-0.1%)	⬇️
src/xrpld/app/misc/ValidatorList.h	100.0% <ø> (ø)
src/xrpld/app/misc/detail/ValidatorList.cpp	86.9% <100.0%> (+0.5%)	⬆️
src/xrpld/core/Config.h	85.7% <ø> (ø)
src/xrpld/core/ConfigSections.h	100.0% <ø> (ø)
src/xrpld/core/detail/Config.cpp	74.7% <100.0%> (+1.2%)	⬆️

... and 2 files with indirect coverage changes

[ximinez]

Looks good overall. I just left a few small suggestions, questions, and comments.

[ximinez]

Looks good!

[Bronek]

Proposed merge message

Add [validator_list_threshold] to validators.txt to improve UNL security