ip address geolocation lookup

[aptxx]

Support ip to geo location #2350

Looking for reviews

MaxMind is implemented. The MaxMind reader returns a 3166-1 Alpha2 country code. We've added a mapping for countrycode that converts the parsed Alpha2 code to Alpha3. To align with prebid-server-java and make it easier for non-developers to configure, the mapping data is stored in the country-codes.csv file within the code repository. This file is will be packaged into the binary at compile time.

The configuration files are mostly ported from the Java version to maintain consistency, so the default configurations are identical. The URL https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz is no longer valid. You now need to register a MaxMind account and get download link.

However, you can anonymously download the MMDB file from here: https://github.com/P3TERX/GeoLite.mmdb. But please remember, you will need to package it yourself into a .tar.gz format.

Configs

New section example:

geolocation:
    enabled: false
    type: maxmind
    maxmind:
        remote_file_syncer:
            http_client:
                max_connections_per_host: 0
                max_idle_connections: 40
                max_idle_connections_per_host: 2
                idle_connection_timeout_seconds: 60
            download_url: "https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz"
            save_filepath: "/var/tmp/prebid/GeoLite2-City.tar.gz"
            tmp_filepath: "/var/tmp/prebid/tmp/GeoLite2-City.tar.gz"
            retry_count: 3
            retry_interval_ms: 3000
            timeout_ms: 300000
            update_interval_ms: 0

GeoLocation feature need actived at account side. For example, actived in default account

account_defaults:
    geolocation:
        enabled: true

Privacy

Considering that the IP needs to be checked for privacy status before use, and to avoid unnecessary processing of private information, we've extracted a more general method to determine the privacy status.

Additionally, following the traffic diagram from Prebid Server and GDPR, we've also implemented the consent_string_means_in_scope configuration option, which can be configured in both account GDPR and global GDPR settings

[bsardo]

Thanks for the contribution @aptxx and sorry for the delay. We've been a bit overwhelmed. I'll give this a thorough review in a couple of weeks.

[bsardo]

Hi @aptxx, we recently released PBS 3.0, more specifically v3.1.0, which updates Prebid Server package import references throughout the project from v2 to v3.
For example:

import (
    "github.com/prebid/prebid-server/v3/adapters"
)

As a result, please merge with master (no rebase) and then ensure all Prebid Server package import references in the files you’ve changed are v3 such that the test suite passes so we can resume reviewing. Thanks!

[aptxx]

hi @bsardo , codes updated. Sorry for that I forgot to merge after reviewing recent changes and conflicts. This may require more time for review, thank you for your patience and hard work. Let me know if there’s anything I need to do.

Main changes:

1. Move path from v2 to v3
2. Fix conflicts of GDPR and user.ext.consent unit tests
3. Relocate geolocation/remotefilesync to util/
4. Move import _ "tz/data" from main.go to util/timeutil/time.go
5. Fix remotefilesync initialization timing issue (use time.Millisecond instead of time.Microsecond in router/router.go)

[bsardo]

@aptxx sorry for the long delay. This is quite a large PR and we have not had a large block of time that we can dedicate to reviewing it. Can you please resolve conflicts and we'll try to find some time in the next few weeks?

[aptxx]

@aptxx sorry for the long delay. This is quite a large PR and we have not had a large block of time that we can dedicate to reviewing it. Can you please resolve conflicts and we'll try to find some time in the next few weeks?

need I remove setting consent_string_means_in_scope or metrics to reduce codes in this PR?