From 3e142e9c081b3ce2af32d53a82347d48580c0d8e Mon Sep 17 00:00:00 2001 From: Michael Jeffrey Date: Wed, 17 Jul 2024 09:50:15 -0700 Subject: [PATCH 1/4] HIP-119: Location Trust Score from maximum asserted distance difference (#840) * Calculate Location Trust Score from asserted distance in heartbeat HIP-119 introduces a new set of tables for location trust scores based on `(radio_type, distance_to_asserted)`. This also allows for a new minimum trust score multiplier of `0x`. * Remove use of max_distance_to_asserted This value is now contained within the function `asserted_distance_to_trust_multiplier` in the coverage point calculator. * Update test for increased allowable location trust multipliers All location trust scores used to be 1.0 or 0.25. HIP-119 adds 0.00 as a multiplier possibility based on distance. * typo in test name * Use lowest possible location trust multiplier value for bad case scenarios * Pass location trust to determine boost eligibility Location Trust Scores are no longer reduced based on the presence of a boosted hex. However, having an average distance from an asserted location past 50m can cause a radio to be ineligible for boosted rewards. * namespace location consts to provide more context Both constants have to do with service provider boosting in regards to a radios location trust scores. Namespacing allows for not needing to shove all possible context into a top level name. * make service provider boosting module It didn't feel quite correct to have half the service provider boosting code in lib.rs and the other in location.rs. The constants have to with location trust scores, but they do not get used there. * Remove trust score tests that expect score modification from boosting The distance to asserted no longer changes trust scores in the presence of a boosted hex. Service provider boosting eligibility is determined from the distance to asserted, that is tested at the top level of this crate. * Test Boosting does not apply when too far away * consolidate seeding heartbeats v1 and v3 The only difference was the location distance from asserted and assigned location trust multipliers. Refactoring heartbeat seeding further is left as an exercise to the next person who has a need to change these tests. * Test being too far from asserted location removes service provider boosting HIP-119 removes the part of calculating coverage points that degrades a trust score when a radio covers a boosted hex and is more than 50m away from their asserted location. Now, being +50m away from an asserted location makes a radio not eligible for receiving service provider boosted rewards. They continue to receive the full force of their location trust score. Being an Indoor radio 100m away from an asserted location is enough to keep a location trust multiplier of 1.0x, but not receive boosted rewards. * add hip-125 mentions in the docs * Try to link to relevant hips when possible Adding links to multiple places so you don't have to know a secret location where they exist. * Active boosting ineligibility takes precendence over passive ineligibility * Remove answered questions Both were answered yes * New location scores apply to Wifi only CBRS is always trusted for location. The location trust score is intercepted earlier when validating heartbeats. If for some reason CBRS radios do make it to this function, they will receive a good trust score anyways. * ServiceProvider -> SP for brevity --- coverage_point_calculator/src/lib.rs | 185 +++++++--- coverage_point_calculator/src/location.rs | 143 +++----- .../src/service_provider_boosting.rs | 37 ++ .../tests/coverage_point_calculator.rs | 14 +- mobile_verifier/src/heartbeats/cbrs.rs | 5 - mobile_verifier/src/heartbeats/mod.rs | 31 +- mobile_verifier/src/heartbeats/wifi.rs | 5 - mobile_verifier/src/reward_shares.rs | 8 +- .../src/rewarder/boosted_hex_eligibility.rs | 28 +- mobile_verifier/src/settings.rs | 9 - .../tests/integrations/boosting_oracles.rs | 1 - .../tests/integrations/hex_boosting.rs | 332 ++++++++++++------ .../tests/integrations/last_location.rs | 14 +- .../tests/integrations/modeled_coverage.rs | 127 +++---- 14 files changed, 543 insertions(+), 396 deletions(-) create mode 100644 coverage_point_calculator/src/service_provider_boosting.rs diff --git a/coverage_point_calculator/src/lib.rs b/coverage_point_calculator/src/lib.rs index 2e23ee181..8e891d737 100644 --- a/coverage_point_calculator/src/lib.rs +++ b/coverage_point_calculator/src/lib.rs @@ -24,6 +24,7 @@ //! - [HIP-98][qos-score] //! - states 30m requirement for boosted hexes [HIP-107][prevent-gaming] //! - increase Boosted hex restriction, 30m -> 50m [Pull Request][boosted-hex-restriction] +//! - Maximum Asserted Distance Difference [HIP-119][location-gaming] //! //! - [CoveragePoints::speedtest_multiplier] //! - [HIP-74][modeled-coverage] @@ -32,7 +33,7 @@ //! //! ## Notable Conditions: //! - [LocationTrust] -//! - If a Radio covers any boosted hexes, [LocationTrust] scores must meet distance requirements, or be degraded. +//! - The average distance to asserted must be <=50m to be eligible for boosted rewards. //! - CBRS Radio's location is always trusted because of GPS. //! //! - [Speedtest] @@ -43,6 +44,10 @@ //! - If a Radio is not [BoostedHexStatus::Eligible], boost values are removed before calculations. //! - If a Hex is boosted by a Provider, the Oracle Assignment multiplier is automatically 1x. //! +//! - [ServiceProviderBoostedRewardEligibility] +//! - Radio must pass at least 1mb of data from 3 unique phones [HIP-84][provider-boosting] +//! - Service Provider can invalidate boosted rewards of a hotspot [HIP-125][provider-banning] +//! //! [modeled-coverage]: https://github.com/helium/HIP/blob/main/0074-mobile-poc-modeled-coverage-rewards.md#outdoor-radios //! [provider-boosting]: https://github.com/helium/HIP/blob/main/0084-service-provider-hex-boosting.md //! [wifi-aps]: https://github.com/helium/HIP/blob/main/0093-addition-of-wifi-aps-to-mobile-subdao.md @@ -53,18 +58,23 @@ //! [cbrs-experimental]: https://github.com/helium/HIP/blob/main/0113-reward-cbrs-as-experimental.md //! [mobile-poc-blog]: https://docs.helium.com/mobile/proof-of-coverage //! [boosted-hex-restriction]: https://github.com/helium/oracles/pull/808 +//! [location-gaming]: https://github.com/helium/HIP/blob/main/0119-closing-gaming-loopholes-within-the-mobile-network.md +//! [provider-banning]: https://github.com/helium/HIP/blob/main/0125-temporary-anti-gaming-measures-for-boosted-hexes.md //! pub use crate::{ hexes::{CoveredHex, HexPoints}, - location::LocationTrust, + location::{asserted_distance_to_trust_multiplier, LocationTrust}, + service_provider_boosting::SPBoostedRewardEligibility, speedtest::{BytesPs, Speedtest, SpeedtestTier}, }; use coverage_map::SignalLevel; use rust_decimal::Decimal; use rust_decimal_macros::dec; +use service_provider_boosting::{MAX_AVERAGE_DISTANCE, MIN_WIFI_TRUST_MULTIPLIER}; mod hexes; mod location; +mod service_provider_boosting; mod speedtest; pub type Result = std::result::Result; @@ -122,7 +132,7 @@ pub struct CoveragePoints { /// Input Radio Type pub radio_type: RadioType, /// Input ServiceProviderBoostedRewardEligibility - pub service_provider_boosted_reward_eligibility: ServiceProviderBoostedRewardEligibility, + pub service_provider_boosted_reward_eligibility: SPBoostedRewardEligibility, /// Derived Eligibility for Boosted Hex Rewards pub boosted_hex_eligibility: BoostedHexStatus, /// Speedtests used in calculcation @@ -136,18 +146,18 @@ pub struct CoveragePoints { impl CoveragePoints { pub fn new( radio_type: RadioType, - service_provider_boosted_reward_eligibility: ServiceProviderBoostedRewardEligibility, + service_provider_boosted_reward_eligibility: SPBoostedRewardEligibility, speedtests: Vec, - trust_scores: Vec, + location_trust_scores: Vec, ranked_coverage: Vec, ) -> Result { - let location_trust_scores = location::clean_trust_scores(trust_scores, &ranked_coverage); let location_trust_multiplier = location::multiplier(radio_type, &location_trust_scores); let boost_eligibility = BoostedHexStatus::new( - &radio_type, + radio_type, location_trust_multiplier, - &service_provider_boosted_reward_eligibility, + &location_trust_scores, + service_provider_boosted_reward_eligibility, ); let covered_hexes = @@ -181,7 +191,7 @@ impl CoveragePoints { /// value referred to as "shares". /// /// Ref: - /// https://github.com/helium/proto/blob/master/src/service/poc_mobile.proto + /// /// `message radio_reward` pub fn coverage_points_v1(&self) -> Decimal { let total_coverage_points = self.coverage_points.base + self.boosted_points(); @@ -209,39 +219,48 @@ impl CoveragePoints { match self.boosted_hex_eligibility { BoostedHexStatus::Eligible => self.coverage_points.boosted, BoostedHexStatus::WifiLocationScoreBelowThreshold(_) => dec!(0), + BoostedHexStatus::AverageAssertedDistanceOverLimit(_) => dec!(0), BoostedHexStatus::RadioThresholdNotMet => dec!(0), BoostedHexStatus::ServiceProviderBanned => dec!(0), } } } -#[derive(Debug, Clone, Copy)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum BoostedHexStatus { Eligible, WifiLocationScoreBelowThreshold(Decimal), + AverageAssertedDistanceOverLimit(Decimal), RadioThresholdNotMet, ServiceProviderBanned, } impl BoostedHexStatus { fn new( - radio_type: &RadioType, - location_trust_score: Decimal, - service_provider_boosted_reward_eligibility: &ServiceProviderBoostedRewardEligibility, + radio_type: RadioType, + location_trust_multiplier: Decimal, + location_trust_scores: &[LocationTrust], + service_provider_boosted_reward_eligibility: SPBoostedRewardEligibility, ) -> Self { - // hip-93: if radio is wifi & location_trust score multiplier < 0.75, no boosting - if radio_type.is_wifi() && location_trust_score < dec!(0.75) { - return Self::WifiLocationScoreBelowThreshold(location_trust_score); - } - - // hip-84: if radio has not met minimum data and subscriber thresholds, no boosting match service_provider_boosted_reward_eligibility { - ServiceProviderBoostedRewardEligibility::Eligible => Self::Eligible, - ServiceProviderBoostedRewardEligibility::ServiceProviderBanned => { - Self::ServiceProviderBanned - } - ServiceProviderBoostedRewardEligibility::RadioThresholdNotMet => { - Self::RadioThresholdNotMet + // hip-125: if radio has been banned by service provider, no boosting + SPBoostedRewardEligibility::ServiceProviderBanned => Self::ServiceProviderBanned, + // hip-84: if radio has not met minimum data and subscriber thresholds, no boosting + SPBoostedRewardEligibility::RadioThresholdNotMet => Self::RadioThresholdNotMet, + SPBoostedRewardEligibility::Eligible => { + // hip-93: if radio is wifi & location_trust score multiplier < 0.75, no boosting + if radio_type.is_wifi() && location_trust_multiplier < MIN_WIFI_TRUST_MULTIPLIER { + return Self::WifiLocationScoreBelowThreshold(location_trust_multiplier); + } + + // hip-119: if the average distance to asserted is beyond 50m, no boosting + let average_distance = + location::average_distance(radio_type, location_trust_scores); + if average_distance > MAX_AVERAGE_DISTANCE { + return Self::AverageAssertedDistanceOverLimit(average_distance); + } + + Self::Eligible } } } @@ -315,13 +334,6 @@ impl RadioType { } } -#[derive(Debug, Clone, Copy, PartialEq)] -pub enum ServiceProviderBoostedRewardEligibility { - Eligible, - ServiceProviderBanned, - RadioThresholdNotMet, -} - #[cfg(test)] mod tests { @@ -346,7 +358,7 @@ mod tests { ) { let wifi = CoveragePoints::new( RadioType::IndoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -368,7 +380,7 @@ mod tests { #[test] fn hip_84_radio_meets_minimum_subscriber_threshold_for_boosted_hexes() { - let calculate_wifi = |eligibility: ServiceProviderBoostedRewardEligibility| { + let calculate_wifi = |eligibility: SPBoostedRewardEligibility| { CoveragePoints::new( RadioType::IndoorWifi, eligibility, @@ -393,13 +405,12 @@ mod tests { // Radio meeting the threshold is eligible for boosted hexes. // Boosted hex provides radio with more than base_points. - let verified_wifi = calculate_wifi(ServiceProviderBoostedRewardEligibility::Eligible); + let verified_wifi = calculate_wifi(SPBoostedRewardEligibility::Eligible); assert_eq!(base_points * dec!(5), verified_wifi.coverage_points_v1()); // Radio not meeting the threshold is not eligible for boosted hexes. // Boost from hex is not applied, radio receives base points. - let unverified_wifi = - calculate_wifi(ServiceProviderBoostedRewardEligibility::RadioThresholdNotMet); + let unverified_wifi = calculate_wifi(SPBoostedRewardEligibility::RadioThresholdNotMet); assert_eq!(base_points, unverified_wifi.coverage_points_v1()); } @@ -408,7 +419,7 @@ mod tests { let calculate_wifi = |location_trust_scores: Vec| { CoveragePoints::new( RadioType::IndoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_scores, vec![RankedCoverage { @@ -441,12 +452,48 @@ mod tests { assert!(untrusted_wifi.coverage_points_v1() < base_points); } + #[test] + fn hip_119_radio_with_past_50m_from_asserted_receives_no_boosted_hexes() { + let calculate_wifi = |location_trust_scores: Vec| { + CoveragePoints::new( + RadioType::IndoorWifi, + SPBoostedRewardEligibility::Eligible, + speedtest_maximum(), + location_trust_scores, + vec![RankedCoverage { + hotspot_key: pubkey(), + cbsd_id: None, + hex: hex_location(), + rank: 1, + signal_level: SignalLevel::High, + assignments: assignments_maximum(), + boosted: NonZeroU32::new(5), + }], + ) + .expect("indoor wifi with location scores") + }; + + let base_points = RadioType::IndoorWifi + .base_coverage_points(&SignalLevel::High) + .unwrap(); + + // Radio with distance to asserted under the limit is eligible for boosted hexes. + // Boosted hex provides radio with more than base_points. + let trusted_wifi = calculate_wifi(location_trust_with_asserted_distance(&[0, 49])); + assert!(trusted_wifi.total_shares() > base_points); + + // Radio with distance to asserted over the limit is not eligible for boosted hexes. + // Boost from hex is not applied. + let untrusted_wifi = calculate_wifi(location_trust_with_asserted_distance(&[50, 51])); + assert_eq!(untrusted_wifi.total_shares(), base_points); + } + #[test] fn speedtests_effect_reward_shares() { let calculate_indoor_cbrs = |speedtests: Vec| { CoveragePoints::new( RadioType::IndoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtests, location_trust_maximum(), vec![RankedCoverage { @@ -534,7 +581,7 @@ mod tests { use Assignment::*; let indoor_cbrs = CoveragePoints::new( RadioType::IndoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![ @@ -591,7 +638,7 @@ mod tests { ) { let outdoor_wifi = CoveragePoints::new( radio_type, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -620,7 +667,7 @@ mod tests { ) { let indoor_wifi = CoveragePoints::new( radio_type, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![ @@ -663,7 +710,7 @@ mod tests { // Location scores are averaged together let indoor_wifi = CoveragePoints::new( RadioType::IndoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_with_scores(&[dec!(0.1), dec!(0.2), dec!(0.3), dec!(0.4)]), vec![RankedCoverage { @@ -707,7 +754,7 @@ mod tests { ]; let indoor_wifi = CoveragePoints::new( RadioType::IndoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), covered_hexes.clone(), @@ -730,7 +777,7 @@ mod tests { ) { let outdoor_cbrs = CoveragePoints::new( RadioType::OutdoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -757,7 +804,7 @@ mod tests { ) { let indoor_cbrs = CoveragePoints::new( RadioType::IndoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -786,7 +833,7 @@ mod tests { ) { let outdoor_wifi = CoveragePoints::new( RadioType::OutdoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -813,7 +860,7 @@ mod tests { ) { let indoor_wifi = CoveragePoints::new( RadioType::IndoorWifi, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtest_maximum(), location_trust_maximum(), vec![RankedCoverage { @@ -831,6 +878,36 @@ mod tests { assert_eq!(expected, indoor_wifi.coverage_points_v1()); } + #[test] + fn wifi_with_bad_location_boosted_hex_status_prioritizes_service_provider_statuses() { + let bad_location = vec![LocationTrust { + meters_to_asserted: 100, + trust_score: dec!(0.0), + }]; + + let wifi_bad_trust_score = |sp_status: SPBoostedRewardEligibility| { + BoostedHexStatus::new( + RadioType::IndoorWifi, + location::multiplier(RadioType::IndoorWifi, &bad_location), + &bad_location, + sp_status, + ) + }; + + assert_eq!( + wifi_bad_trust_score(SPBoostedRewardEligibility::Eligible), + BoostedHexStatus::WifiLocationScoreBelowThreshold(dec!(0)), + ); + assert_eq!( + wifi_bad_trust_score(SPBoostedRewardEligibility::ServiceProviderBanned), + BoostedHexStatus::ServiceProviderBanned + ); + assert_eq!( + wifi_bad_trust_score(SPBoostedRewardEligibility::RadioThresholdNotMet), + BoostedHexStatus::RadioThresholdNotMet + ); + } + fn hex_location() -> hextree::Cell { hextree::Cell::from_raw(0x8c2681a3064edff).unwrap() } @@ -896,6 +973,18 @@ mod tests { .collect() } + fn location_trust_with_asserted_distance(distances_to_asserted: &[u32]) -> Vec { + distances_to_asserted + .to_owned() + .iter() + .copied() + .map(|meters_to_asserted| LocationTrust { + meters_to_asserted, + trust_score: dec!(1.0), + }) + .collect() + } + fn pubkey() -> Vec { vec![1] } diff --git a/coverage_point_calculator/src/location.rs b/coverage_point_calculator/src/location.rs index a75ef88eb..ad4ac179d 100644 --- a/coverage_point_calculator/src/location.rs +++ b/coverage_point_calculator/src/location.rs @@ -1,14 +1,8 @@ -use coverage_map::RankedCoverage; use rust_decimal::Decimal; use rust_decimal_macros::dec; use crate::RadioType; -/// When a Radio is covering any boosted hexes, it's trust score location must -/// be within this distance to it's asserted location. Otherwise the trust_score -/// will be capped at 0.25x. -const RESTRICTIVE_MAX_DISTANCE: Meters = 50; - type Meters = u32; #[derive(Debug, Clone, PartialEq)] @@ -17,20 +11,44 @@ pub struct LocationTrust { pub trust_score: Decimal, } -pub(crate) fn clean_trust_scores( - trust_scores: Vec, - ranked_coverage: &[RankedCoverage], -) -> Vec { - let any_boosted_hexes = ranked_coverage.iter().any(|hex| hex.boosted.is_some()); - - if any_boosted_hexes { - trust_scores - .into_iter() - .map(LocationTrust::into_boosted) - .collect() - } else { - trust_scores +/// Returns the trust multiplier for a given radio type and distance to it's asserted location. +/// +/// [HIP-119: Gaming Loopholes][gaming-loopholes] +/// +/// [gaming-loopholes]: https://github.com/helium/HIP/blob/main/0119-closing-gaming-loopholes-within-the-mobile-network.md#maximum-asserted-distance-difference +pub fn asserted_distance_to_trust_multiplier( + radio_type: RadioType, + meters_to_asserted: Meters, +) -> Decimal { + match radio_type { + RadioType::IndoorWifi => match meters_to_asserted { + 0..=200 => dec!(1.00), + 201..=300 => dec!(0.25), + _ => dec!(0.00), + }, + RadioType::OutdoorWifi => match meters_to_asserted { + 0..=75 => dec!(1.00), + 76..=100 => dec!(0.25), + _ => dec!(0.00), + }, + RadioType::IndoorCbrs => dec!(1.0), + RadioType::OutdoorCbrs => dec!(1.0), + } +} + +pub(crate) fn average_distance(radio_type: RadioType, trust_scores: &[LocationTrust]) -> Decimal { + // CBRS radios are always trusted because they have internal GPS + if radio_type.is_cbrs() { + return dec!(0); } + + let count = Decimal::from(trust_scores.len()); + let sum: Decimal = trust_scores + .iter() + .map(|l| Decimal::from(l.meters_to_asserted)) + .sum(); + + sum / count } pub(crate) fn multiplier(radio_type: RadioType, trust_scores: &[LocationTrust]) -> Decimal { @@ -45,35 +63,18 @@ pub(crate) fn multiplier(radio_type: RadioType, trust_scores: &[LocationTrust]) scores / count } -impl LocationTrust { - fn into_boosted(self) -> Self { - // Cap multipliers to 0.25x when a radio covers _any_ boosted hex - // and it's distance to asserted is above the threshold. - let trust_score = if self.meters_to_asserted > RESTRICTIVE_MAX_DISTANCE { - dec!(0.25).min(self.trust_score) - } else { - self.trust_score - }; - - LocationTrust { - trust_score, - meters_to_asserted: self.meters_to_asserted, - } - } -} - #[cfg(test)] mod tests { - use std::num::NonZeroU32; - - use coverage_map::SignalLevel; - use hex_assignments::{assignment::HexAssignments, Assignment}; use super::*; #[test] - fn all_locations_within_max_boosted_distance() { + fn distance_does_not_effect_multiplier() { let trust_scores = vec![ + LocationTrust { + meters_to_asserted: 0, + trust_score: dec!(0.5), + }, LocationTrust { meters_to_asserted: 49, trust_score: dec!(0.5), @@ -82,17 +83,6 @@ mod tests { meters_to_asserted: 50, trust_score: dec!(0.5), }, - ]; - let boosted = clean_trust_scores(trust_scores.clone(), &boosted_ranked_coverage()); - let unboosted = clean_trust_scores(trust_scores, &[]); - - assert_eq!(dec!(0.5), multiplier(RadioType::IndoorWifi, &boosted)); - assert_eq!(dec!(0.5), multiplier(RadioType::IndoorWifi, &unboosted)); - } - - #[test] - fn all_locations_past_max_boosted_distance() { - let trust_scores = vec![ LocationTrust { meters_to_asserted: 51, trust_score: dec!(0.5), @@ -101,36 +91,13 @@ mod tests { meters_to_asserted: 100, trust_score: dec!(0.5), }, - ]; - - let boosted = clean_trust_scores(trust_scores.clone(), &boosted_ranked_coverage()); - let unboosted = clean_trust_scores(trust_scores, &[]); - - assert_eq!(dec!(0.25), multiplier(RadioType::IndoorWifi, &boosted)); - assert_eq!(dec!(0.5), multiplier(RadioType::IndoorWifi, &unboosted)); - } - - #[test] - fn locations_around_max_boosted_distance() { - let trust_scores = vec![ LocationTrust { - meters_to_asserted: 50, - trust_score: dec!(0.5), - }, - LocationTrust { - meters_to_asserted: 51, + meters_to_asserted: 99999, trust_score: dec!(0.5), }, ]; - let boosted = clean_trust_scores(trust_scores.clone(), &boosted_ranked_coverage()); - let unboosted = clean_trust_scores(trust_scores, &[]); - - // location past distance limit trust score is degraded - let degraded_mult = (dec!(0.5) + dec!(0.25)) / dec!(2); - assert_eq!(degraded_mult, multiplier(RadioType::IndoorWifi, &boosted)); - // location past distance limit trust score is untouched - assert_eq!(dec!(0.5), multiplier(RadioType::IndoorWifi, &unboosted)); + assert_eq!(dec!(0.5), multiplier(RadioType::IndoorWifi, &trust_scores)); } #[test] @@ -143,26 +110,6 @@ mod tests { trust_score: dec!(0), }]; - let boosted = clean_trust_scores(trust_scores.clone(), &boosted_ranked_coverage()); - let unboosted = clean_trust_scores(trust_scores, &[]); - - assert_eq!(dec!(1), multiplier(RadioType::IndoorCbrs, &boosted)); - assert_eq!(dec!(1), multiplier(RadioType::IndoorCbrs, &unboosted)); - } - - fn boosted_ranked_coverage() -> Vec { - vec![RankedCoverage { - hex: hextree::Cell::from_raw(0x8c2681a3064edff).unwrap(), - rank: 1, - hotspot_key: vec![], - cbsd_id: None, - signal_level: SignalLevel::High, - assignments: HexAssignments { - footfall: Assignment::A, - landtype: Assignment::A, - urbanized: Assignment::A, - }, - boosted: NonZeroU32::new(5), - }] + assert_eq!(dec!(1), multiplier(RadioType::IndoorCbrs, &trust_scores)); } } diff --git a/coverage_point_calculator/src/service_provider_boosting.rs b/coverage_point_calculator/src/service_provider_boosting.rs new file mode 100644 index 000000000..af31b524a --- /dev/null +++ b/coverage_point_calculator/src/service_provider_boosting.rs @@ -0,0 +1,37 @@ +use rust_decimal::Decimal; +use rust_decimal_macros::dec; + +// In order for the Wi-Fi access point to be eligible for boosted hex rewards +// as described in HIP84 the location trust score needs to be 0.75 or higher. +// +// [HIP-93: Add Wifi to Mobile Dao][add-wifi-aps] +// +// [add-wifi-aps]: https://github.com/helium/HIP/blob/main/0093-addition-of-wifi-aps-to-mobile-subdao.md#341-indoor-access-points-rewards +pub(crate) const MIN_WIFI_TRUST_MULTIPLIER: Decimal = dec!(0.75); + +// In order for access points to be eligible for boosted Service Provider +// rewards defined in HIP-84, the asserted distances must be 50 meters or +// less than the reported location from external services for both indoor +// and outdoor Access Points. +// +// [HIP-119: Gaming Loopholes][gaming-loopholes] +// +// [gaming-loopholes]: https://github.com/helium/HIP/blob/main/0119-closing-gaming-loopholes-within-the-mobile-network.md#maximum-asserted-distance-for-boosted-hexes +pub(crate) const MAX_AVERAGE_DISTANCE: Decimal = dec!(50); + +#[derive(Debug, Clone, Copy, PartialEq)] +pub enum SPBoostedRewardEligibility { + Eligible, + /// Service Provider can invalidate boosted rewards of a hotspot + /// + /// [HIP-125: Anti gaming measures][anti-gaming] + /// + /// [anti-gaming]: https://github.com/helium/HIP/blob/main/0125-temporary-anti-gaming-measures-for-boosted-hexes.md + ServiceProviderBanned, + /// Radio must pass at least 1mb of data from 3 unique phones. + /// + /// [HIP-84: Provider Hex Boosting][provider-boosting] + /// + /// [provider-boosting]: https://github.com/helium/HIP/blob/main/0084-service-provider-hex-boosting.md + RadioThresholdNotMet, +} diff --git a/coverage_point_calculator/tests/coverage_point_calculator.rs b/coverage_point_calculator/tests/coverage_point_calculator.rs index aba3323ed..f8de92fc9 100644 --- a/coverage_point_calculator/tests/coverage_point_calculator.rs +++ b/coverage_point_calculator/tests/coverage_point_calculator.rs @@ -3,8 +3,8 @@ use std::num::NonZeroU32; use chrono::Utc; use coverage_map::{BoostedHexMap, RankedCoverage, SignalLevel, UnrankedCoverage}; use coverage_point_calculator::{ - BytesPs, CoveragePoints, LocationTrust, RadioType, Result, - ServiceProviderBoostedRewardEligibility, Speedtest, SpeedtestTier, + BytesPs, CoveragePoints, LocationTrust, RadioType, Result, SPBoostedRewardEligibility, + Speedtest, SpeedtestTier, }; use hex_assignments::{assignment::HexAssignments, Assignment}; use rust_decimal_macros::dec; @@ -52,7 +52,7 @@ fn base_radio_coverage_points() { ] { let coverage_points = CoveragePoints::new( radio_type, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, speedtests.clone(), location_trust_scores.clone(), hexes.clone(), @@ -113,7 +113,7 @@ fn radios_with_coverage() { ] { let coverage_points = CoveragePoints::new( radio_type, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, default_speedtests.clone(), default_location_trust_scores.clone(), base_hex_iter.clone().take(num_hexes).collect(), @@ -240,7 +240,7 @@ fn cbrs_outdoor_with_mixed_signal_level_coverage() -> Result { let radio = CoveragePoints::new( RadioType::OutdoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, Speedtest::mock(SpeedtestTier::Good), vec![], // Location Trust is ignored for Cbrs vec![ @@ -372,7 +372,7 @@ fn indoor_cbrs_radio( ) -> Result { CoveragePoints::new( RadioType::IndoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, Speedtest::mock(speedtest_tier), vec![], coverage.to_owned(), @@ -385,7 +385,7 @@ fn outdoor_cbrs_radio( ) -> Result { CoveragePoints::new( RadioType::OutdoorCbrs, - ServiceProviderBoostedRewardEligibility::Eligible, + SPBoostedRewardEligibility::Eligible, Speedtest::mock(speedtest_tier), vec![], coverage.to_owned(), diff --git a/mobile_verifier/src/heartbeats/cbrs.rs b/mobile_verifier/src/heartbeats/cbrs.rs index fa3cdf3a6..2d7e8f0a1 100644 --- a/mobile_verifier/src/heartbeats/cbrs.rs +++ b/mobile_verifier/src/heartbeats/cbrs.rs @@ -28,7 +28,6 @@ pub struct CbrsHeartbeatDaemon { pool: sqlx::Pool, gateway_info_resolver: GIR, heartbeats: Receiver>, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, heartbeat_sink: FileSinkClient, seniority_sink: FileSinkClient, @@ -65,7 +64,6 @@ where pool, gateway_resolver, cbrs_heartbeats, - settings.max_asserted_distance_deviation, settings.max_distance_from_coverage, valid_heartbeats, seniority_updates, @@ -83,7 +81,6 @@ where pool: sqlx::Pool, gateway_info_resolver: GIR, heartbeats: Receiver>, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, heartbeat_sink: FileSinkClient, seniority_sink: FileSinkClient, @@ -93,7 +90,6 @@ where pool, gateway_info_resolver, heartbeats, - max_distance_to_asserted, max_distance_to_coverage, heartbeat_sink, seniority_sink, @@ -170,7 +166,6 @@ where &self.gateway_info_resolver, coverage_object_cache, location_cache, - self.max_distance_to_asserted, self.max_distance_to_coverage, &epoch, &self.geofence, diff --git a/mobile_verifier/src/heartbeats/mod.rs b/mobile_verifier/src/heartbeats/mod.rs index 2e21a68a9..985fcae52 100644 --- a/mobile_verifier/src/heartbeats/mod.rs +++ b/mobile_verifier/src/heartbeats/mod.rs @@ -373,7 +373,6 @@ impl ValidatedHeartbeat { gateway_info_resolver: &impl GatewayResolver, coverage_object_cache: &CoverageObjectCache, last_location_cache: &LocationCache, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, epoch: &Range>, geofence: &impl GeofenceValidator, @@ -538,17 +537,29 @@ impl ValidatedHeartbeat { true } }; + let distance_to_asserted = asserted_latlng.distance_m(hb_latlng).round() as i64; - let location_trust_score_multiplier = if is_valid - // The heartbeat location to asserted location must be less than the max_distance_to_asserted value: - && distance_to_asserted <= max_distance_to_asserted as i64 - // The heartbeat location to every associated coverage hex must be less than max_distance_to_coverage: - && coverage_object.max_distance_m(hb_latlng).round() as u32 <= max_distance_to_coverage - { - dec!(1.0) + let max_distance = coverage_object.max_distance_m(hb_latlng).round() as u32; + + let location_trust_score_multiplier = if !is_valid { + dec!(0) + } else if max_distance >= max_distance_to_coverage { + // Furthest hex in Heartbeat exceeds allowed coverage distance + dec!(0) } else { - dec!(0.25) + // HIP-119 maximum asserted distance check + use coverage_point_calculator::{ + asserted_distance_to_trust_multiplier, RadioType, + }; + let radio_type = match (heartbeat.hb_type, coverage_object.meta.indoor) { + (HbType::Cbrs, true) => RadioType::IndoorCbrs, + (HbType::Cbrs, false) => RadioType::OutdoorCbrs, + (HbType::Wifi, true) => RadioType::IndoorWifi, + (HbType::Wifi, false) => RadioType::OutdoorWifi, + }; + asserted_distance_to_trust_multiplier(radio_type, distance_to_asserted as u32) }; + Ok(Self::new( heartbeat, cell_type, @@ -575,7 +586,6 @@ impl ValidatedHeartbeat { gateway_info_resolver: &'a impl GatewayResolver, coverage_object_cache: &'a CoverageObjectCache, last_location_cache: &'a LocationCache, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, epoch: &'a Range>, geofence: &'a impl GeofenceValidator, @@ -586,7 +596,6 @@ impl ValidatedHeartbeat { gateway_info_resolver, coverage_object_cache, last_location_cache, - max_distance_to_asserted, max_distance_to_coverage, epoch, geofence, diff --git a/mobile_verifier/src/heartbeats/wifi.rs b/mobile_verifier/src/heartbeats/wifi.rs index 7456af788..f40da6e24 100644 --- a/mobile_verifier/src/heartbeats/wifi.rs +++ b/mobile_verifier/src/heartbeats/wifi.rs @@ -27,7 +27,6 @@ pub struct WifiHeartbeatDaemon { pool: sqlx::Pool, gateway_info_resolver: GIR, heartbeats: Receiver>, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, heartbeat_sink: FileSinkClient, seniority_sink: FileSinkClient, @@ -63,7 +62,6 @@ where pool, gateway_resolver, wifi_heartbeats, - settings.max_asserted_distance_deviation, settings.max_distance_from_coverage, valid_heartbeats, seniority_updates, @@ -81,7 +79,6 @@ where pool: sqlx::Pool, gateway_info_resolver: GIR, heartbeats: Receiver>, - max_distance_to_asserted: u32, max_distance_to_coverage: u32, heartbeat_sink: FileSinkClient, seniority_sink: FileSinkClient, @@ -91,7 +88,6 @@ where pool, gateway_info_resolver, heartbeats, - max_distance_to_asserted, max_distance_to_coverage, heartbeat_sink, seniority_sink, @@ -161,7 +157,6 @@ where &self.gateway_info_resolver, coverage_object_cache, location_cache, - self.max_distance_to_asserted, self.max_distance_to_coverage, &epoch, &self.geofence, diff --git a/mobile_verifier/src/reward_shares.rs b/mobile_verifier/src/reward_shares.rs index 7496acbb7..5a35cdfbc 100644 --- a/mobile_verifier/src/reward_shares.rs +++ b/mobile_verifier/src/reward_shares.rs @@ -8,7 +8,7 @@ use crate::{ subscriber_location::SubscriberValidatedLocations, }; use chrono::{DateTime, Duration, Utc}; -use coverage_point_calculator::ServiceProviderBoostedRewardEligibility; +use coverage_point_calculator::SPBoostedRewardEligibility; use file_store::traits::TimestampEncode; use futures::{Stream, StreamExt}; use helium_crypto::PublicKeyBinary; @@ -427,7 +427,7 @@ struct RadioInfo { coverage_obj_uuid: Uuid, seniority: Seniority, trust_scores: Vec, - sp_boosted_reward_eligibility: ServiceProviderBoostedRewardEligibility, + sp_boosted_reward_eligibility: SPBoostedRewardEligibility, speedtests: Vec, } @@ -2035,7 +2035,7 @@ mod test { inserted_at: now, update_reason: 0, }, - sp_boosted_reward_eligibility: ServiceProviderBoostedRewardEligibility::Eligible, + sp_boosted_reward_eligibility: SPBoostedRewardEligibility::Eligible, speedtests: vec![ coverage_point_calculator::Speedtest { upload_speed: coverage_point_calculator::BytesPs::new(100_000_000), @@ -2068,7 +2068,7 @@ mod test { inserted_at: now, update_reason: 0, }, - sp_boosted_reward_eligibility: ServiceProviderBoostedRewardEligibility::Eligible, + sp_boosted_reward_eligibility: SPBoostedRewardEligibility::Eligible, speedtests: vec![], }, ); diff --git a/mobile_verifier/src/rewarder/boosted_hex_eligibility.rs b/mobile_verifier/src/rewarder/boosted_hex_eligibility.rs index 21f327fea..3db3ea19e 100644 --- a/mobile_verifier/src/rewarder/boosted_hex_eligibility.rs +++ b/mobile_verifier/src/rewarder/boosted_hex_eligibility.rs @@ -1,4 +1,4 @@ -use coverage_point_calculator::ServiceProviderBoostedRewardEligibility; +use coverage_point_calculator::SPBoostedRewardEligibility; use helium_crypto::PublicKeyBinary; use crate::{radio_threshold::VerifiedRadioThresholds, sp_boosted_rewards_bans::BannedRadios}; @@ -21,13 +21,13 @@ impl BoostedHexEligibility { &self, key: PublicKeyBinary, cbsd_id_opt: Option, - ) -> ServiceProviderBoostedRewardEligibility { + ) -> SPBoostedRewardEligibility { if self.banned_radios.contains(&key, cbsd_id_opt.as_deref()) { - ServiceProviderBoostedRewardEligibility::ServiceProviderBanned + SPBoostedRewardEligibility::ServiceProviderBanned } else if self.radio_thresholds.is_verified(key, cbsd_id_opt) { - ServiceProviderBoostedRewardEligibility::Eligible + SPBoostedRewardEligibility::Eligible } else { - ServiceProviderBoostedRewardEligibility::RadioThresholdNotMet + SPBoostedRewardEligibility::RadioThresholdNotMet } } } @@ -56,14 +56,14 @@ mod tests { let eligibility = boosted_hex_eligibility.eligibility(pub_key.clone(), None); assert_eq!( - ServiceProviderBoostedRewardEligibility::ServiceProviderBanned, + SPBoostedRewardEligibility::ServiceProviderBanned, eligibility ); let eligibility = boosted_hex_eligibility.eligibility(pub_key, Some(cbsd_id)); assert_eq!( - ServiceProviderBoostedRewardEligibility::ServiceProviderBanned, + SPBoostedRewardEligibility::ServiceProviderBanned, eligibility ); } @@ -81,14 +81,14 @@ mod tests { let eligibility = boosted_hex_eligibility.eligibility(pub_key.clone(), None); assert_eq!( - ServiceProviderBoostedRewardEligibility::RadioThresholdNotMet, + SPBoostedRewardEligibility::RadioThresholdNotMet, eligibility ); let eligibility = boosted_hex_eligibility.eligibility(pub_key, Some(cbsd_id)); assert_eq!( - ServiceProviderBoostedRewardEligibility::RadioThresholdNotMet, + SPBoostedRewardEligibility::RadioThresholdNotMet, eligibility ); } @@ -109,17 +109,11 @@ mod tests { let eligibility = boosted_hex_eligibility.eligibility(pub_key.clone(), None); - assert_eq!( - ServiceProviderBoostedRewardEligibility::Eligible, - eligibility - ); + assert_eq!(SPBoostedRewardEligibility::Eligible, eligibility); let eligibility = boosted_hex_eligibility.eligibility(pub_key, Some(cbsd_id)); - assert_eq!( - ServiceProviderBoostedRewardEligibility::Eligible, - eligibility - ); + assert_eq!(SPBoostedRewardEligibility::Eligible, eligibility); } fn generate_keypair() -> Keypair { diff --git a/mobile_verifier/src/settings.rs b/mobile_verifier/src/settings.rs index 1a3949ebd..54a31d9dd 100644 --- a/mobile_verifier/src/settings.rs +++ b/mobile_verifier/src/settings.rs @@ -38,11 +38,6 @@ pub struct Settings { /// its respective coverage object #[serde(default = "default_max_distance_from_coverage")] pub max_distance_from_coverage: u32, - /// Max distance in meters between the asserted location of a WIFI hotspot - /// and the lat/lng defined in a heartbeat - /// beyond which its location weight will be reduced - #[serde(default = "default_max_asserted_distance_deviation")] - pub max_asserted_distance_deviation: u32, /// Directory in which new oracle boosting data sets are downloaded into pub data_sets_directory: PathBuf, /// Poll duration for new data sets @@ -66,10 +61,6 @@ fn default_max_distance_from_coverage() -> u32 { 2000 } -fn default_max_asserted_distance_deviation() -> u32 { - 100 -} - fn default_log() -> String { "mobile_verifier=debug,poc_store=info".to_string() } diff --git a/mobile_verifier/tests/integrations/boosting_oracles.rs b/mobile_verifier/tests/integrations/boosting_oracles.rs index 0cf60efeb..d5d00d844 100644 --- a/mobile_verifier/tests/integrations/boosting_oracles.rs +++ b/mobile_verifier/tests/integrations/boosting_oracles.rs @@ -362,7 +362,6 @@ async fn test_footfall_and_urbanization_and_landtype(pool: PgPool) -> anyhow::Re &coverage_objects, &location_cache, 2000, - 2000, &epoch, &MockGeofence, )); diff --git a/mobile_verifier/tests/integrations/hex_boosting.rs b/mobile_verifier/tests/integrations/hex_boosting.rs index a0058ec75..a6de082e5 100644 --- a/mobile_verifier/tests/integrations/hex_boosting.rs +++ b/mobile_verifier/tests/integrations/hex_boosting.rs @@ -724,7 +724,23 @@ async fn test_reduced_location_score_with_boosted_hexes(pool: PgPool) -> anyhow: // seed all the things let mut txn = pool.clone().begin().await?; - seed_heartbeats_v3(epoch.start, &mut txn).await?; + seed_heartbeats_with_location_trust( + epoch.start, + &mut txn, + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + HotspotLocationTrust { + meters: 300, + multiplier: dec!(0.25), + }, + ) + .await?; seed_speedtests(epoch.end, &mut txn).await?; seed_radio_thresholds(epoch.start, &mut txn).await?; txn.commit().await?; @@ -871,6 +887,185 @@ async fn test_reduced_location_score_with_boosted_hexes(pool: PgPool) -> anyhow: Ok(()) } +#[sqlx::test] +async fn test_distance_from_asserted_removes_boosting_but_not_location_trust( + pool: PgPool, +) -> anyhow::Result<()> { + let (mobile_rewards_client, mut mobile_rewards) = common::create_file_sink(); + let (speedtest_avg_client, _speedtest_avg_server) = common::create_file_sink(); + let now = Utc::now(); + let epoch = (now - ChronoDuration::hours(24))..now; + let epoch_duration = epoch.end - epoch.start; + let boost_period_length = Duration::days(30); + + // seed all the things + let mut txn = pool.clone().begin().await?; + seed_heartbeats_with_location_trust( + epoch.start, + &mut txn, + // hotspot 1 can receive boosting + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + // hotspot 2 can receive boosting but has no boosted hexes + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + // hotspot 3 is too far for boosting + HotspotLocationTrust { + meters: 100, + multiplier: dec!(1.0), + }, + ) + .await?; + seed_speedtests(epoch.end, &mut txn).await?; + seed_radio_thresholds(epoch.start, &mut txn).await?; + txn.commit().await?; + update_assignments(&pool).await?; + + // setup boosted hex where reward start time is in the second period length + let multipliers1 = vec![NonZeroU32::new(2).unwrap()]; + let start_ts_1 = epoch.start; + let end_ts_1 = start_ts_1 + (boost_period_length * multipliers1.len() as i32); + + // setup boosted hex where no start or end time is set + let multipliers2 = vec![NonZeroU32::new(2).unwrap()]; + + let boosted_hexes = vec![ + BoostedHexInfo { + // hotspot 1's location + location: Cell::from_raw(0x8a1fb466d2dffff_u64)?, + start_ts: Some(start_ts_1), + end_ts: Some(end_ts_1), + period_length: boost_period_length, + multipliers: multipliers1, + boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), + boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), + version: 0, + }, + BoostedHexInfo { + // hotspot 3's location + location: Cell::from_raw(0x8c2681a306607ff_u64)?, + start_ts: None, + end_ts: None, + period_length: boost_period_length, + multipliers: multipliers2, + boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), + boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), + version: 0, + }, + ]; + + let hex_boosting_client = MockHexBoostingClient::new(boosted_hexes); + let total_poc_emissions = reward_shares::get_scheduled_tokens_for_poc(epoch_duration) + .to_u64() + .unwrap(); + + let (_, rewards) = tokio::join!( + // run rewards for poc and dc + rewarder::reward_poc_and_dc( + &pool, + &hex_boosting_client, + &mobile_rewards_client, + &speedtest_avg_client, + &epoch, + dec!(0.0001) + ), + receive_expected_rewards_maybe_unallocated( + &mut mobile_rewards, + ExpectUnallocated::NoWhenValue(total_poc_emissions) + ) + ); + + let Ok((poc_rewards, unallocated_reward)) = rewards else { + panic!("no rewards received"); + }; + + let mut poc_rewards = poc_rewards.iter(); + let hotspot_2 = poc_rewards.next().unwrap(); // full location trust NO boosts + let hotspot_1 = poc_rewards.next().unwrap(); // full location trust 1 boost + let hotspot_3 = poc_rewards.next().unwrap(); // reduced location trust 1 boost + assert_eq!( + None, + poc_rewards.next(), + "Received more hotspots than expected in rewards" + ); + assert_eq!( + HOTSPOT_1.to_string(), + PublicKeyBinary::from(hotspot_1.hotspot_key.clone()).to_string() + ); + assert_eq!( + HOTSPOT_2.to_string(), + PublicKeyBinary::from(hotspot_2.hotspot_key.clone()).to_string() + ); + assert_eq!( + HOTSPOT_3.to_string(), + PublicKeyBinary::from(hotspot_3.hotspot_key.clone()).to_string() + ); + + // Calculating expected rewards + let (regular_poc, boosted_poc) = get_poc_allocation_buckets(epoch_duration); + + // Here's how we get the regular shares per coverage points + // | base coverage point | speedtest | location | total | + // |---------------------|-----------|----------|-------| + // | 400 | 0.75 | 1.00 | 300 | + // | 400 | 0.75 | 1.00 | 300 | + // | 400 | 0.75 | 1.00 | 300 | + // |---------------------|-----------|----------|-------| + // | 900 | + let regular_share = regular_poc / dec!(900); + + // Boosted hexes are 2x, only one radio qualifies based on the location trust + // 300 * 1 == 300 + // To get points _only_ from boosting. + let boosted_share = boosted_poc / dec!(300); + + let exp_reward_1 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(1)); + let exp_reward_2 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); + let exp_reward_3 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); + + assert_eq!(exp_reward_1, hotspot_1.poc_reward); + assert_eq!(exp_reward_2, hotspot_2.poc_reward); + assert_eq!(exp_reward_3, hotspot_3.poc_reward); + + // assert the number of boosted hexes for each radio + //hotspot 1 has one boosted hex + assert_eq!(1, hotspot_1.boosted_hexes.len()); + //hotspot 2 has no boosted hexes + assert_eq!(0, hotspot_2.boosted_hexes.len()); + // hotspot 3 has a boosted location but as its location trust score + // is reduced the boost does not get applied + assert_eq!(0, hotspot_3.boosted_hexes.len()); + + // assert the hex boost multiplier values + // assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); + assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); + assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.boosted_hexes[0].location); + + // confirm the total rewards allocated matches expectations + let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; + let total = poc_sum + unallocated_reward.amount; + + let expected_sum = reward_shares::get_scheduled_tokens_for_poc(epoch.end - epoch.start) + .to_u64() + .unwrap(); + assert_eq!(expected_sum, total); + + // confirm the rewarded percentage amount matches expectations + let daily_total = reward_shares::get_total_scheduled_tokens(epoch.end - epoch.start); + let percent = (Decimal::from(total) / daily_total) + .round_dp_with_strategy(2, RoundingStrategy::MidpointNearestEven); + assert_eq!(percent, dec!(0.6)); + + Ok(()) +} + #[sqlx::test] async fn test_poc_with_cbrs_and_multi_coverage_boosted_hexes(pool: PgPool) -> anyhow::Result<()> { let (mobile_rewards_client, mut mobile_rewards) = common::create_file_sink(); @@ -1143,100 +1338,24 @@ async fn seed_heartbeats_v1( ts: DateTime, txn: &mut Transaction<'_, Postgres>, ) -> anyhow::Result<()> { - for n in 0..24 { - let hotspot_key1: PublicKeyBinary = HOTSPOT_1.to_string().parse().unwrap(); - let cov_obj_1 = create_coverage_object( - ts + ChronoDuration::hours(n), - None, - hotspot_key1.clone(), - 0x8a1fb466d2dffff_u64, - true, - ); - let wifi_heartbeat1 = ValidatedHeartbeat { - heartbeat: Heartbeat { - hb_type: HbType::Wifi, - hotspot_key: hotspot_key1, - cbsd_id: None, - operation_mode: true, - lat: 0.0, - lon: 0.0, - coverage_object: Some(cov_obj_1.coverage_object.uuid), - location_validation_timestamp: Some(ts - ChronoDuration::hours(24)), - timestamp: ts + ChronoDuration::hours(n), - }, - cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(10), - coverage_meta: None, - location_trust_score_multiplier: dec!(1.0), - validity: HeartbeatValidity::Valid, - }; - - let hotspot_key2: PublicKeyBinary = HOTSPOT_2.to_string().parse().unwrap(); - let cov_obj_2 = create_coverage_object( - ts + ChronoDuration::hours(n), - None, - hotspot_key2.clone(), - 0x8a1fb49642dffff_u64, - true, - ); - let wifi_heartbeat2 = ValidatedHeartbeat { - heartbeat: Heartbeat { - hb_type: HbType::Wifi, - hotspot_key: hotspot_key2, - cbsd_id: None, - operation_mode: true, - lat: 0.0, - lon: 0.0, - coverage_object: Some(cov_obj_2.coverage_object.uuid), - location_validation_timestamp: Some(ts - ChronoDuration::hours(24)), - timestamp: ts + ChronoDuration::hours(n), - }, - cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(10), - coverage_meta: None, - location_trust_score_multiplier: dec!(1.0), - validity: HeartbeatValidity::Valid, - }; - - let hotspot_key3: PublicKeyBinary = HOTSPOT_3.to_string().parse().unwrap(); - let cov_obj_3 = create_coverage_object( - ts + ChronoDuration::hours(n), - None, - hotspot_key3.clone(), - 0x8c2681a306607ff_u64, - true, - ); - let wifi_heartbeat3 = ValidatedHeartbeat { - heartbeat: Heartbeat { - hb_type: HbType::Wifi, - hotspot_key: hotspot_key3, - cbsd_id: None, - operation_mode: true, - lat: 0.0, - lon: 0.0, - coverage_object: Some(cov_obj_3.coverage_object.uuid), - location_validation_timestamp: Some(ts - ChronoDuration::hours(24)), - timestamp: ts + ChronoDuration::hours(n), - }, - cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(10), - coverage_meta: None, - location_trust_score_multiplier: dec!(1.0), - validity: HeartbeatValidity::Valid, - }; - - save_seniority_object(ts + ChronoDuration::hours(n), &wifi_heartbeat1, txn).await?; - save_seniority_object(ts + ChronoDuration::hours(n), &wifi_heartbeat2, txn).await?; - save_seniority_object(ts + ChronoDuration::hours(n), &wifi_heartbeat3, txn).await?; - - wifi_heartbeat1.save(txn).await?; - wifi_heartbeat2.save(txn).await?; - wifi_heartbeat3.save(txn).await?; + seed_heartbeats_with_location_trust( + ts, + txn, + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + ) + .await?; - cov_obj_1.save(txn).await?; - cov_obj_2.save(txn).await?; - cov_obj_3.save(txn).await?; - } Ok(()) } @@ -1342,13 +1461,18 @@ async fn seed_heartbeats_v2( Ok(()) } -async fn seed_heartbeats_v3( +struct HotspotLocationTrust { + meters: i64, + multiplier: Decimal, +} + +async fn seed_heartbeats_with_location_trust( ts: DateTime, txn: &mut Transaction<'_, Postgres>, + hs_1_location: HotspotLocationTrust, + hs_2_location: HotspotLocationTrust, + hs_3_location: HotspotLocationTrust, ) -> anyhow::Result<()> { - // HOTSPOT 1 has full location trust score - // HOTSPOT 2 has full location trust score - // HOTSPOT 3 has reduced location trust score for n in 0..24 { let hotspot_key1: PublicKeyBinary = HOTSPOT_1.to_string().parse().unwrap(); let cov_obj_1 = create_coverage_object( @@ -1371,9 +1495,9 @@ async fn seed_heartbeats_v3( timestamp: ts + ChronoDuration::hours(n), }, cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(10), + distance_to_asserted: Some(hs_1_location.meters), coverage_meta: None, - location_trust_score_multiplier: dec!(1.0), + location_trust_score_multiplier: hs_1_location.multiplier, validity: HeartbeatValidity::Valid, }; @@ -1398,9 +1522,9 @@ async fn seed_heartbeats_v3( timestamp: ts + ChronoDuration::hours(n), }, cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(10), + distance_to_asserted: Some(hs_2_location.meters), coverage_meta: None, - location_trust_score_multiplier: dec!(1.0), + location_trust_score_multiplier: hs_2_location.multiplier, validity: HeartbeatValidity::Valid, }; @@ -1425,9 +1549,9 @@ async fn seed_heartbeats_v3( timestamp: ts + ChronoDuration::hours(n), }, cell_type: CellType::NovaGenericWifiIndoor, - distance_to_asserted: Some(300), + distance_to_asserted: Some(hs_3_location.meters), coverage_meta: None, - location_trust_score_multiplier: dec!(0.25), + location_trust_score_multiplier: hs_3_location.multiplier, validity: HeartbeatValidity::Valid, }; diff --git a/mobile_verifier/tests/integrations/last_location.rs b/mobile_verifier/tests/integrations/last_location.rs index c5ba6a150..5ba87e932 100644 --- a/mobile_verifier/tests/integrations/last_location.rs +++ b/mobile_verifier/tests/integrations/last_location.rs @@ -42,7 +42,7 @@ impl GatewayResolver for AllOwnersValid { } #[sqlx::test] -async fn heatbeat_uses_last_good_location_when_invalid_location( +async fn heartbeat_uses_last_good_location_when_invalid_location( pool: PgPool, ) -> anyhow::Result<()> { let hotspot = PublicKeyBinary::from_str(PUB_KEY)?; @@ -63,7 +63,6 @@ async fn heatbeat_uses_last_good_location_when_invalid_location( &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -82,7 +81,6 @@ async fn heatbeat_uses_last_good_location_when_invalid_location( &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -108,7 +106,7 @@ async fn heatbeat_uses_last_good_location_when_invalid_location( } #[sqlx::test] -async fn heatbeat_will_use_last_good_location_from_db(pool: PgPool) -> anyhow::Result<()> { +async fn heartbeat_will_use_last_good_location_from_db(pool: PgPool) -> anyhow::Result<()> { let hotspot = PublicKeyBinary::from_str(PUB_KEY)?; let epoch_start = Utc::now() - Duration::days(1); let epoch_end = epoch_start + Duration::days(2); @@ -127,7 +125,6 @@ async fn heatbeat_will_use_last_good_location_from_db(pool: PgPool) -> anyhow::R &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -151,7 +148,6 @@ async fn heatbeat_will_use_last_good_location_from_db(pool: PgPool) -> anyhow::R &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -177,7 +173,7 @@ async fn heatbeat_will_use_last_good_location_from_db(pool: PgPool) -> anyhow::R } #[sqlx::test] -async fn heatbeat_does_not_use_last_good_location_when_more_than_12_hours( +async fn heartbeat_does_not_use_last_good_location_when_more_than_12_hours( pool: PgPool, ) -> anyhow::Result<()> { let hotspot = PublicKeyBinary::from_str(PUB_KEY)?; @@ -199,7 +195,6 @@ async fn heatbeat_does_not_use_last_good_location_when_more_than_12_hours( &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -218,7 +213,6 @@ async fn heatbeat_does_not_use_last_good_location_when_more_than_12_hours( &AllOwnersValid, &coverage_objects, &location_cache, - 1, u32::MAX, &(epoch_start..epoch_end), &MockGeofence, @@ -227,7 +221,7 @@ async fn heatbeat_does_not_use_last_good_location_when_more_than_12_hours( assert_eq!( validated_heartbeat_2.location_trust_score_multiplier, - dec!(0.25) + dec!(0.00) ); Ok(()) diff --git a/mobile_verifier/tests/integrations/modeled_coverage.rs b/mobile_verifier/tests/integrations/modeled_coverage.rs index 4d5c64432..0a66fb322 100644 --- a/mobile_verifier/tests/integrations/modeled_coverage.rs +++ b/mobile_verifier/tests/integrations/modeled_coverage.rs @@ -6,6 +6,7 @@ use file_store::{ wifi_heartbeat::{WifiHeartbeat, WifiHeartbeatIngestReport}, }; use futures::stream::{self, StreamExt}; +use h3o::{CellIndex, LatLng}; use helium_crypto::PublicKeyBinary; use helium_proto::services::{ mobile_config::NetworkKeyRole, @@ -416,7 +417,6 @@ async fn process_input( &coverage_objects, &location_cache, 2000, - 2000, epoch, &MockGeofence, )); @@ -1382,26 +1382,15 @@ async fn ensure_lower_trust_score_for_distant_heartbeats(pool: PgPool) -> anyhow coverage_object.save(&mut transaction).await?; transaction.commit().await?; - let hb_1 = WifiHeartbeatIngestReport { - report: WifiHeartbeat { - pubkey: owner_1.clone(), - lon: -105.2715848904, - lat: 40.0194278140, - timestamp: DateTime::::MIN_UTC, - location_validation_timestamp: Some(DateTime::::MIN_UTC), - operation_mode: true, - coverage_object: Vec::from(coverage_object_uuid.into_bytes()), - }, - received_timestamp: Utc::now(), - }; - - let hb_1: Heartbeat = hb_1.into(); + let max_covered_distance = 5_000; + let coverage_object_cache = CoverageObjectCache::new(&pool); + let location_cache = LocationCache::new(&pool); - let hb_2 = WifiHeartbeatIngestReport { + let mk_heartbeat = |latlng: LatLng| WifiHeartbeatIngestReport { report: WifiHeartbeat { pubkey: owner_1.clone(), - lon: -105.2344693282443, - lat: 40.033526907035935, + lon: latlng.lng(), + lat: latlng.lat(), timestamp: DateTime::::MIN_UTC, location_validation_timestamp: Some(DateTime::::MIN_UTC), operation_mode: true, @@ -1410,70 +1399,54 @@ async fn ensure_lower_trust_score_for_distant_heartbeats(pool: PgPool) -> anyhow received_timestamp: Utc::now(), }; - let hb_2: Heartbeat = hb_2.into(); - - let coverage_object_cache = CoverageObjectCache::new(&pool); - let location_cache = LocationCache::new(&pool); - - let validated_hb_1 = ValidatedHeartbeat::validate( - hb_1, - &AllOwnersValid, - &coverage_object_cache, - &location_cache, - 2000, - 2000, - &(DateTime::::MIN_UTC..DateTime::::MAX_UTC), - &MockGeofence, - ) - .await - .unwrap(); - - assert_eq!(validated_hb_1.location_trust_score_multiplier, dec!(1.0)); - - let validated_hb_2 = ValidatedHeartbeat::validate( - hb_2.clone(), - &AllOwnersValid, - &coverage_object_cache, - &location_cache, - 1000000, - 2000, - &(DateTime::::MIN_UTC..DateTime::::MAX_UTC), - &MockGeofence, - ) - .await - .unwrap(); + let validate = |latlng: LatLng| { + ValidatedHeartbeat::validate( + mk_heartbeat(latlng).into(), + &AllOwnersValid, + &coverage_object_cache, + &location_cache, + max_covered_distance, + &(DateTime::::MIN_UTC..DateTime::::MAX_UTC), + &MockGeofence, + ) + }; - assert_eq!(validated_hb_2.location_trust_score_multiplier, dec!(0.25)); + let covered_cell_index: CellIndex = "8c2681a3064d9ff".parse()?; + let covered_latlng = LatLng::from(covered_cell_index); - let validated_hb_2 = ValidatedHeartbeat::validate( - hb_2.clone(), - &AllOwnersValid, - &coverage_object_cache, - &location_cache, - 2000, - 1000000, - &(DateTime::::MIN_UTC..DateTime::::MAX_UTC), - &MockGeofence, - ) - .await - .unwrap(); + // Constrain distances by only moving vertically + let near_latlng = LatLng::new(40.0194278140, -105.272)?; // 35m + let med_latlng = LatLng::new(40.0194278140, -105.274)?; // 205m + let far_latlng = LatLng::new(40.0194278140, -105.3)?; // 2,419m + let past_latlng = LatLng::new(40.0194278140, 105.2715848904)?; // 10,591,975m - assert_eq!(validated_hb_2.location_trust_score_multiplier, dec!(0.25)); + // It's easy to gloss over floats, let make sure the distances are within the ranges we expect. + assert!((0.0..=200.0).contains(&covered_latlng.distance_m(near_latlng))); // Indoor low distance <= 200 + assert!((200.0..=300.0).contains(&covered_latlng.distance_m(med_latlng))); // Indoor Medium distance <= 300 + assert!(covered_latlng.distance_m(far_latlng) > 300.0); // Indoor Over Distance => 300 + assert!(covered_latlng.distance_m(past_latlng) > max_covered_distance as f64); // Indoor past max distance => max_distance - let validated_hb_2 = ValidatedHeartbeat::validate( - hb_2.clone(), - &AllOwnersValid, - &coverage_object_cache, - &location_cache, - 1000000, - 1000000, - &(DateTime::::MIN_UTC..DateTime::::MAX_UTC), - &MockGeofence, - ) - .await - .unwrap(); + let low_dist_validated = validate(near_latlng).await?; + let med_dist_validated = validate(med_latlng).await?; + let far_dist_validated = validate(far_latlng).await?; + let past_dist_validated = validate(past_latlng).await?; - assert_eq!(validated_hb_2.location_trust_score_multiplier, dec!(1.0)); + assert_eq!( + low_dist_validated.location_trust_score_multiplier, + dec!(1.0) + ); + assert_eq!( + med_dist_validated.location_trust_score_multiplier, + dec!(0.25) + ); + assert_eq!( + far_dist_validated.location_trust_score_multiplier, + dec!(0.00) + ); + assert_eq!( + past_dist_validated.location_trust_score_multiplier, + dec!(0.00) + ); Ok(()) } From d24c9f69dfad588042d37d2bcba5f775aa984392 Mon Sep 17 00:00:00 2001 From: Michael Jeffrey Date: Wed, 17 Jul 2024 09:57:56 -0700 Subject: [PATCH 2/4] Integrate mobile rewards v2 (#843) * Calculate Location Trust Score from asserted distance in heartbeat HIP-119 introduces a new set of tables for location trust scores based on `(radio_type, distance_to_asserted)`. This also allows for a new minimum trust score multiplier of `0x`. * Remove use of max_distance_to_asserted This value is now contained within the function `asserted_distance_to_trust_multiplier` in the coverage point calculator. * Update test for increased allowable location trust multipliers All location trust scores used to be 1.0 or 0.25. HIP-119 adds 0.00 as a multiplier possibility based on distance. * typo in test name * Use lowest possible location trust multiplier value for bad case scenarios * Pass location trust to determine boost eligibility Location Trust Scores are no longer reduced based on the presence of a boosted hex. However, having an average distance from an asserted location past 50m can cause a radio to be ineligible for boosted rewards. * namespace location consts to provide more context Both constants have to do with service provider boosting in regards to a radios location trust scores. Namespacing allows for not needing to shove all possible context into a top level name. * make service provider boosting module It didn't feel quite correct to have half the service provider boosting code in lib.rs and the other in location.rs. The constants have to with location trust scores, but they do not get used there. * Remove trust score tests that expect score modification from boosting The distance to asserted no longer changes trust scores in the presence of a boosted hex. Service provider boosting eligibility is determined from the distance to asserted, that is tested at the top level of this crate. * Test Boosting does not apply when too far away * consolidate seeding heartbeats v1 and v3 The only difference was the location distance from asserted and assigned location trust multipliers. Refactoring heartbeat seeding further is left as an exercise to the next person who has a need to change these tests. * Test being too far from asserted location removes service provider boosting HIP-119 removes the part of calculating coverage points that degrades a trust score when a radio covers a boosted hex and is more than 50m away from their asserted location. Now, being +50m away from an asserted location makes a radio not eligible for receiving service provider boosted rewards. They continue to receive the full force of their location trust score. Being an Indoor radio 100m away from an asserted location is enough to keep a location trust multiplier of 1.0x, but not receive boosted rewards. * add hip-125 mentions in the docs * Try to link to relevant hips when possible Adding links to multiple places so you don't have to know a secret location where they exist. * Active boosting ineligibility takes precendence over passive ineligibility * Remove answered questions Both were answered yes * New location scores apply to Wifi only CBRS is always trusted for location. The location trust score is intercepted earlier when validating heartbeats. If for some reason CBRS radios do make it to this function, they will receive a good trust score anyways. * ServiceProvider -> SP for brevity * Fix updated Assignment enum -> OracleBoostingAssignment Assignment used to be a nested message inside the oracle_boosting_hex_assignment message. It's being referenced in mobile_rewards_v2, so it's been pulled out. * Add basic support for radio_reward_v2 in dumping mobile rewards * Add a getter for bytes per second use `as_bps()` instead of `into_inner()` for the symmetry with `as_mbps()` and to prevent compulsory use of `into_inner()` and wanting a different type. * Add helper crate for CoveragePoints -> radio_reward_v2 proto Put the cumbersome transformations in a helper crate to help keep transforming from a calculator CoveragePoints to proto a little neater. * Construct RadioReward and RadioRewardV2 for mobile rewards Both will live in the proto for a while until we fully move over to v2. To still be done is using the v2 reward for test assertions before we can fully move over. * apply auto-formatting to Cargo.toml * Use RadioRewardV2 in integration tests Until RadioReward v1 stop being output, we can compare the poc_reward fields of both version, they should be the same always. (maybe off by a bone, but that would be a special case indeed.) * Use RadioRewardV2 in unit tests * Move beacon closer to helium-proto Because `beacon` is in the helium-proto repo, when you want to test a different branch you need to update both entries. It is a convenience that they are moved closer together. * Update proto to branch with v2 rewards * Add notes about testing proto changes locally * Name mobile reward proto variables similarly and consistently A blank v2 looked confusing in the diff. * spell out var name, remove println * Update helium-proto to include hip-119 boosting ineligible reason HIP-119 added average distance to asserted further than threshold reason for a radio to not receive boosted rewards. * Prevent indexer from breaking when it encounters mobile_reward_v2 Both `mobile_reward` and `mobile_reward_v2` will be written out for 90 days. Within that 90 days, Indexer will switch to using `mobile_reward_v2` as the source of reward values, then `mobile_reward`s will stop being written. If both messages were used radios would be double rewarded. * proto back to master --- Cargo.lock | 8 +- Cargo.toml | 14 +- coverage_point_calculator/src/speedtest.rs | 4 + file_store/src/cli/dump_mobile_rewards.rs | 10 + hex_assignments/src/assignment.rs | 2 +- mobile_verifier/src/cli/reward_from_db.rs | 2 +- mobile_verifier/src/reward_shares.rs | 167 ++++++--- .../src/reward_shares/radio_reward_v2.rs | 87 +++++ mobile_verifier/src/rewarder.rs | 11 +- .../tests/integrations/common/mod.rs | 85 ++++- .../tests/integrations/hex_boosting.rs | 332 ++++++++++++++---- .../tests/integrations/rewarder_poc_dc.rs | 20 +- reward_index/src/indexer.rs | 46 ++- 13 files changed, 612 insertions(+), 176 deletions(-) create mode 100644 mobile_verifier/src/reward_shares/radio_reward_v2.rs diff --git a/Cargo.lock b/Cargo.lock index b37ef565e..5c7390197 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1617,7 +1617,7 @@ checksum = "8c3c1a368f70d6cf7302d78f8f7093da241fb8e8807c05cc9e51a125895a6d5b" [[package]] name = "beacon" version = "0.1.0" -source = "git+https://github.com/helium/proto?branch=master#04ef234d5cff695fa88cd115d5fbecf03ce97bf8" +source = "git+https://github.com/helium/proto?branch=master#62e22b2a4fb7de6f2ba7d67f933d77d4ad52a882" dependencies = [ "base64 0.21.7", "byteorder", @@ -1627,7 +1627,7 @@ dependencies = [ "rand_chacha 0.3.0", "rust_decimal", "serde", - "sha2 0.10.8", + "sha2 0.9.9", "thiserror", ] @@ -3801,7 +3801,7 @@ dependencies = [ [[package]] name = "helium-proto" version = "0.1.0" -source = "git+https://github.com/helium/proto?branch=master#04ef234d5cff695fa88cd115d5fbecf03ce97bf8" +source = "git+https://github.com/helium/proto?branch=master#62e22b2a4fb7de6f2ba7d67f933d77d4ad52a882" dependencies = [ "bytes", "prost", @@ -9851,7 +9851,7 @@ dependencies = [ "rand 0.8.5", "serde", "serde_json", - "sha2 0.10.8", + "sha2 0.9.9", "thiserror", "twox-hash", "xorf", diff --git a/Cargo.toml b/Cargo.toml index 870fd79e0..8144a56df 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,7 +25,7 @@ members = [ "reward_scheduler", "solana", "task_manager", - "hex_assignments" + "hex_assignments", ] resolver = "2" @@ -73,6 +73,7 @@ hextree = { git = "https://github.com/jaykickliter/HexTree", branch = "main", fe helium-proto = { git = "https://github.com/helium/proto", branch = "master", features = [ "services", ] } +beacon = { git = "https://github.com/helium/proto", branch = "master" } solana-client = "1.18" solana-sdk = "1.18" solana-program = "1.18" @@ -82,7 +83,6 @@ reqwest = { version = "0", default-features = false, features = [ "json", "rustls-tls", ] } -beacon = { git = "https://github.com/helium/proto", branch = "master" } humantime = "2" humantime-serde = "1" metrics = ">=0.22" @@ -123,3 +123,13 @@ derive_builder = "0" [patch.crates-io] sqlx = { git = "https://github.com/helium/sqlx.git", rev = "92a2268f02e0cac6fccb34d3e926347071dbb88d" } + +# When attempting to test proto changes without needing to push a branch you can +# patch the github url to point to your local proto repo. +# +# Patching for beacon must point directly to the crate, it will not look in the +# repo for sibling crates. +# +# [patch.'https://github.com/helium/proto'] +# helium-proto = { path = "../proto" } +# beacon = { path = "../proto/beacon" } diff --git a/coverage_point_calculator/src/speedtest.rs b/coverage_point_calculator/src/speedtest.rs index 000623e68..dea741595 100644 --- a/coverage_point_calculator/src/speedtest.rs +++ b/coverage_point_calculator/src/speedtest.rs @@ -25,6 +25,10 @@ impl BytesPs { fn as_mbps(&self) -> u64 { self.0 / Self::BYTES_PER_MEGABYTE } + + pub fn as_bps(&self) -> u64 { + self.0 + } } pub(crate) fn clean_speedtests(speedtests: Vec) -> Vec { diff --git a/file_store/src/cli/dump_mobile_rewards.rs b/file_store/src/cli/dump_mobile_rewards.rs index 56ba0fccc..34e605002 100644 --- a/file_store/src/cli/dump_mobile_rewards.rs +++ b/file_store/src/cli/dump_mobile_rewards.rs @@ -18,6 +18,7 @@ impl Cmd { let mut file_stream = file_source::source([&self.path]); let mut radio_reward = vec![]; + let mut radio_reward_v2 = vec![]; let mut gateway_reward = vec![]; let mut subscriber_reward = vec![]; let mut service_provider_reward = vec![]; @@ -34,6 +35,14 @@ impl Cmd { "poc_reward": reward.poc_reward, "boosted_hexes": reward.boosted_hexes, })), + RadioRewardV2(reward) => radio_reward_v2.push(json!({ + "hotspot_key": PublicKey::try_from(reward.hotspot_key)?.to_string(), + "cbsd_id": reward.cbsd_id, + "base_poc_reward": reward.base_poc_reward, + "boosted_poc_reward" : reward.boosted_poc_reward, + "total_poc_reward": reward.base_poc_reward + reward.boosted_poc_reward, + "covered_hexes": reward.covered_hexes + })), GatewayReward(reward) => gateway_reward.push(json!({ "hotspot_key": PublicKey::try_from(reward.hotspot_key)?.to_string(), "dc_transfer_reward": reward.dc_transfer_reward, @@ -57,6 +66,7 @@ impl Cmd { print_json(&json!({ "radio_reward": radio_reward, + "radio_reward": radio_reward_v2, "gateway_reward": gateway_reward, "subscriber_reward": subscriber_reward, "service_provider_reward": service_provider_reward, diff --git a/hex_assignments/src/assignment.rs b/hex_assignments/src/assignment.rs index 90a99d990..732df1672 100644 --- a/hex_assignments/src/assignment.rs +++ b/hex_assignments/src/assignment.rs @@ -1,5 +1,5 @@ use anyhow::Result; -use helium_proto::services::poc_mobile::oracle_boosting_hex_assignment::Assignment as ProtoAssignment; +use helium_proto::services::poc_mobile::OracleBoostingAssignment as ProtoAssignment; use rust_decimal::Decimal; use rust_decimal_macros::dec; use std::fmt; diff --git a/mobile_verifier/src/cli/reward_from_db.rs b/mobile_verifier/src/cli/reward_from_db.rs index d064225f7..7de6ce945 100644 --- a/mobile_verifier/src/cli/reward_from_db.rs +++ b/mobile_verifier/src/cli/reward_from_db.rs @@ -61,7 +61,7 @@ impl Cmd { ) .ok_or(anyhow::anyhow!("no rewardable events"))? .1; - for (_reward_amount, reward) in radio_rewards { + for (_reward_amount, reward, _v2) in radio_rewards { if let Some(proto::mobile_reward_share::Reward::RadioReward(proto::RadioReward { hotspot_key, poc_reward, diff --git a/mobile_verifier/src/reward_shares.rs b/mobile_verifier/src/reward_shares.rs index 5a35cdfbc..71c1c02ec 100644 --- a/mobile_verifier/src/reward_shares.rs +++ b/mobile_verifier/src/reward_shares.rs @@ -25,11 +25,14 @@ use mobile_config::{ boosted_hex_info::BoostedHexes, client::{carrier_service_client::CarrierServiceVerifier, ClientError}, }; +use radio_reward_v2::{RadioRewardV2Ext, ToProtoDecimal}; use rust_decimal::prelude::*; use rust_decimal_macros::dec; use std::{collections::HashMap, ops::Range}; use uuid::Uuid; +mod radio_reward_v2; + /// Total tokens emissions pool per 365 days or 366 days for a leap year const TOTAL_EMISSIONS_POOL: Decimal = dec!(30_000_000_000_000_000); @@ -373,9 +376,10 @@ pub fn coverage_point_to_mobile_reward_share( reward_epoch: &Range>, radio_id: &RadioId, poc_reward: u64, + rewards_per_share: CalculatedPocRewardShares, seniority_timestamp: DateTime, coverage_object_uuid: Uuid, -) -> proto::MobileRewardShare { +) -> (proto::MobileRewardShare, proto::MobileRewardShare) { let (hotspot_key, cbsd_id) = radio_id.clone(); let boosted_hexes = coverage_points @@ -392,31 +396,63 @@ pub fn coverage_point_to_mobile_reward_share( let location_trust_score_multiplier = to_proto_value(coverage_points.location_trust_multiplier); let speedtest_multiplier = to_proto_value(coverage_points.speedtest_multiplier); - let coverage_points = coverage_points + let coverage_points_v1 = coverage_points .coverage_points_v1() .to_u64() .unwrap_or_default(); let coverage_object = Vec::from(coverage_object_uuid.into_bytes()); - proto::MobileRewardShare { + let radio_reward_v1 = proto::mobile_reward_share::Reward::RadioReward(proto::RadioReward { + hotspot_key: hotspot_key.clone().into(), + cbsd_id: cbsd_id.clone().unwrap_or_default(), + poc_reward, + coverage_points: coverage_points_v1, + seniority_timestamp: seniority_timestamp.encode_timestamp(), + coverage_object: coverage_object.clone(), + location_trust_score_multiplier, + speedtest_multiplier, + boosted_hexes, + ..Default::default() + }); + + let radio_reward_v2 = proto::mobile_reward_share::Reward::RadioRewardV2(proto::RadioRewardV2 { + hotspot_key: hotspot_key.into(), + cbsd_id: cbsd_id.unwrap_or_default(), + base_coverage_points_sum: Some(coverage_points.coverage_points.base.proto_decimal()), + boosted_coverage_points_sum: Some(coverage_points.coverage_points.boosted.proto_decimal()), + base_reward_shares: Some(coverage_points.total_base_shares().proto_decimal()), + boosted_reward_shares: Some(coverage_points.total_boosted_shares().proto_decimal()), + base_poc_reward: rewards_per_share.base_poc_reward(&coverage_points), + boosted_poc_reward: rewards_per_share.boosted_poc_reward(&coverage_points), + seniority_timestamp: seniority_timestamp.encode_timestamp(), + coverage_object, + location_trust_scores: coverage_points.proto_location_trust_scores(), + location_trust_score_multiplier: Some( + coverage_points.location_trust_multiplier.proto_decimal(), + ), + speedtests: coverage_points.proto_speedtests(), + speedtest_multiplier: Some(coverage_points.speedtest_multiplier.proto_decimal()), + boosted_hex_status: coverage_points.proto_boosted_hex_status().into(), + covered_hexes: coverage_points.proto_covered_hexes(), + }); + + let base = proto::MobileRewardShare { start_period: reward_epoch.start.encode_timestamp(), end_period: reward_epoch.end.encode_timestamp(), - reward: Some(proto::mobile_reward_share::Reward::RadioReward( - proto::RadioReward { - hotspot_key: hotspot_key.into(), - cbsd_id: cbsd_id.unwrap_or_default(), - poc_reward, - coverage_points, - seniority_timestamp: seniority_timestamp.encode_timestamp(), - coverage_object, - location_trust_score_multiplier, - speedtest_multiplier, - boosted_hexes, - ..Default::default() - }, - )), - } + reward: None, + }; + + ( + proto::MobileRewardShare { + reward: Some(radio_reward_v1), + ..base.clone() + }, + proto::MobileRewardShare { + reward: Some(radio_reward_v2), + ..base + }, + ) } type RadioId = (PublicKeyBinary, Option); @@ -579,7 +615,7 @@ impl CoverageShares { epoch: &'_ Range>, ) -> Option<( CalculatedPocRewardShares, - impl Iterator + '_, + impl Iterator + '_, )> { struct ProcessedRadio { radio_id: RadioId, @@ -631,17 +667,19 @@ impl CoverageShares { } = radio; let poc_reward = rewards_per_share.poc_reward(&points); - let mobile_reward_share = coverage_point_to_mobile_reward_share( - points, - epoch, - &radio_id, - poc_reward, - seniority.seniority_ts, - coverage_obj_uuid, - ); - (poc_reward, mobile_reward_share) + let (mobile_reward_v1, mobile_reward_v2) = + coverage_point_to_mobile_reward_share( + points, + epoch, + &radio_id, + poc_reward, + rewards_per_share, + seniority.seniority_ts, + coverage_obj_uuid, + ); + (poc_reward, mobile_reward_v1, mobile_reward_v2) }) - .filter(|(poc_reward, _mobile_reward)| *poc_reward > 0), + .filter(|(poc_reward, _mobile_reward_v1, _mobile_reward_v2)| *poc_reward > 0), )) } @@ -748,15 +786,20 @@ impl CalculatedPocRewardShares { } } - fn poc_reward(&self, points: &coverage_point_calculator::CoveragePoints) -> u64 { - let base_reward = (self.normal * points.total_base_shares()) + fn base_poc_reward(&self, points: &coverage_point_calculator::CoveragePoints) -> u64 { + (self.normal * points.total_base_shares()) .to_u64() - .unwrap_or_default(); - let boosted_reward = (self.boost * points.total_boosted_shares()) + .unwrap_or_default() + } + + fn boosted_poc_reward(&self, points: &coverage_point_calculator::CoveragePoints) -> u64 { + (self.boost * points.total_boosted_shares()) .to_u64() - .unwrap_or_default(); + .unwrap_or_default() + } - base_reward + boosted_reward + fn poc_reward(&self, points: &coverage_point_calculator::CoveragePoints) -> u64 { + self.base_poc_reward(points) + self.boosted_poc_reward(points) } } @@ -1491,7 +1534,7 @@ mod test { let mut allocated_poc_rewards = 0_u64; let epoch = (now - Duration::hours(1))..now; - for (reward_amount, mobile_reward) in CoverageShares::new( + for (reward_amount, _mobile_reward_v1, mobile_reward_v2) in CoverageShares::new( &hex_coverage, stream::iter(heartbeat_rewards), &speedtest_avgs, @@ -1505,17 +1548,22 @@ mod test { .unwrap() .1 { - let radio_reward = match mobile_reward.reward { - Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward, + let radio_reward = match mobile_reward_v2.reward { + Some(MobileReward::RadioRewardV2(radio_reward)) => radio_reward, _ => unreachable!(), }; let owner = owners .get(&PublicKeyBinary::from(radio_reward.hotspot_key)) .expect("Could not find owner") .clone(); - assert_eq!(reward_amount, radio_reward.poc_reward); + + let base = radio_reward.base_poc_reward; + let boosted = radio_reward.boosted_poc_reward; + let poc_reward = base + boosted; + assert_eq!(reward_amount, poc_reward); + allocated_poc_rewards += reward_amount; - *owner_rewards.entry(owner).or_default() += radio_reward.poc_reward; + *owner_rewards.entry(owner).or_default() += poc_reward; } assert_eq!( @@ -1666,7 +1714,7 @@ mod test { let reward_shares = DataTransferAndPocAllocatedRewardBuckets::new_poc_only(&epoch); - for (_reward_amount, mobile_reward) in CoverageShares::new( + for (_reward_amount, _mobile_reward_v1, mobile_reward_v2) in CoverageShares::new( &hex_coverage, stream::iter(heartbeat_rewards), &speedtest_avgs, @@ -1680,8 +1728,8 @@ mod test { .unwrap() .1 { - let radio_reward = match mobile_reward.reward { - Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward, + let radio_reward = match mobile_reward_v2.reward { + Some(MobileReward::RadioRewardV2(radio_reward)) => radio_reward, _ => unreachable!(), }; let owner = owners @@ -1689,9 +1737,10 @@ mod test { .expect("Could not find owner") .clone(); - *owner_rewards.entry(owner).or_default() += radio_reward.poc_reward; + let base = radio_reward.base_poc_reward; + let boosted = radio_reward.boosted_poc_reward; + *owner_rewards.entry(owner).or_default() += base + boosted; } - println!("owner rewards {:?}", owner_rewards); // wifi let owner1_reward = *owner_rewards @@ -1798,7 +1847,7 @@ mod test { let epoch = (now - duration)..now; let reward_shares = DataTransferAndPocAllocatedRewardBuckets::new_poc_only(&epoch); - for (_reward_amount, mobile_reward) in CoverageShares::new( + for (_reward_amount, _mobile_reward_v1, mobile_reward_v2) in CoverageShares::new( &hex_coverage, stream::iter(heartbeat_rewards), &speedtest_avgs, @@ -1812,8 +1861,8 @@ mod test { .unwrap() .1 { - let radio_reward = match mobile_reward.reward { - Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward, + let radio_reward = match mobile_reward_v2.reward { + Some(MobileReward::RadioRewardV2(radio_reward)) => radio_reward, _ => unreachable!(), }; let owner = owners @@ -1821,7 +1870,9 @@ mod test { .expect("Could not find owner") .clone(); - *owner_rewards.entry(owner).or_default() += radio_reward.poc_reward; + let base = radio_reward.base_poc_reward; + let boosted = radio_reward.boosted_poc_reward; + *owner_rewards.entry(owner).or_default() += base + boosted; } // wifi @@ -1930,7 +1981,7 @@ mod test { let epoch = (now - duration)..now; let reward_shares = DataTransferAndPocAllocatedRewardBuckets::new_poc_only(&epoch); - for (_reward_amount, mobile_reward) in CoverageShares::new( + for (_reward_amount, _mobile_reward_v1, mobile_reward_v2) in CoverageShares::new( &hex_coverage, stream::iter(heartbeat_rewards), &speedtest_avgs, @@ -1944,8 +1995,8 @@ mod test { .unwrap() .1 { - let radio_reward = match mobile_reward.reward { - Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward, + let radio_reward = match mobile_reward_v2.reward { + Some(MobileReward::RadioRewardV2(radio_reward)) => radio_reward, _ => unreachable!(), }; let owner = owners @@ -1953,11 +2004,11 @@ mod test { .expect("Could not find owner") .clone(); - *owner_rewards.entry(owner).or_default() += radio_reward.poc_reward; + let base = radio_reward.base_poc_reward; + let boosted = radio_reward.boosted_poc_reward; + *owner_rewards.entry(owner).or_default() += base + boosted; } - // These were different, now they are the same: - println!("owner rewards {:?}", owner_rewards); // wifi let owner1_reward = *owner_rewards .get(&owner1) @@ -2081,13 +2132,13 @@ mod test { let reward_shares = DataTransferAndPocAllocatedRewardBuckets::new_poc_only(&epoch); // gw2 does not have enough speedtests for a mulitplier let expected_hotspot = gw1; - for (_reward_amount, mobile_reward) in coverage_shares + for (_reward_amount, _mobile_reward_v1, mobile_reward_v2) in coverage_shares .into_rewards(reward_shares, &epoch) .expect("rewards output") .1 { - let radio_reward = match mobile_reward.reward { - Some(proto::mobile_reward_share::Reward::RadioReward(radio_reward)) => radio_reward, + let radio_reward = match mobile_reward_v2.reward { + Some(MobileReward::RadioRewardV2(radio_reward)) => radio_reward, _ => unreachable!(), }; let actual_hotspot = PublicKeyBinary::from(radio_reward.hotspot_key); diff --git a/mobile_verifier/src/reward_shares/radio_reward_v2.rs b/mobile_verifier/src/reward_shares/radio_reward_v2.rs new file mode 100644 index 000000000..f22a7ec6f --- /dev/null +++ b/mobile_verifier/src/reward_shares/radio_reward_v2.rs @@ -0,0 +1,87 @@ +use file_store::traits::TimestampEncode; +use helium_proto::services::poc_mobile::{ + radio_reward_v2::{CoveredHex, LocationTrustScore}, + BoostedHexStatus, Speedtest, +}; +use rust_decimal::prelude::ToPrimitive; + +pub trait ToProtoDecimal { + fn proto_decimal(&self) -> helium_proto::Decimal; +} + +impl ToProtoDecimal for rust_decimal::Decimal { + fn proto_decimal(&self) -> helium_proto::Decimal { + helium_proto::Decimal { + value: self.to_string(), + } + } +} +pub trait RadioRewardV2Ext { + fn proto_location_trust_scores(&self) -> Vec; + fn proto_speedtests(&self) -> Vec; + fn proto_boosted_hex_status(&self) -> BoostedHexStatus; + fn proto_covered_hexes(&self) -> Vec; +} + +impl RadioRewardV2Ext for coverage_point_calculator::CoveragePoints { + fn proto_location_trust_scores(&self) -> Vec { + self.location_trust_scores + .iter() + .map(|lt| LocationTrustScore { + meters_to_asserted: lt.meters_to_asserted.into(), + trust_score: Some(lt.trust_score.proto_decimal()), + }) + .collect() + } + + fn proto_speedtests(&self) -> Vec { + self.speedtests + .iter() + .map(|st| helium_proto::services::poc_mobile::Speedtest { + upload_speed_bps: st.upload_speed.as_bps(), + download_speed_bps: st.download_speed.as_bps(), + latency_ms: st.latency_millis, + timestamp: st.timestamp.encode_timestamp(), + }) + .collect() + } + + fn proto_boosted_hex_status(&self) -> BoostedHexStatus { + match self.boosted_hex_eligibility { + coverage_point_calculator::BoostedHexStatus::Eligible => BoostedHexStatus::Eligible, + coverage_point_calculator::BoostedHexStatus::WifiLocationScoreBelowThreshold(_) => { + BoostedHexStatus::LocationScoreBelowThreshold + } + coverage_point_calculator::BoostedHexStatus::RadioThresholdNotMet => { + BoostedHexStatus::RadioThresholdNotMet + } + coverage_point_calculator::BoostedHexStatus::ServiceProviderBanned => { + BoostedHexStatus::ServiceProviderBan + } + coverage_point_calculator::BoostedHexStatus::AverageAssertedDistanceOverLimit(_) => { + BoostedHexStatus::AverageAssertedDistanceOverLimit + } + } + } + + fn proto_covered_hexes(&self) -> Vec { + self.covered_hexes + .iter() + .map(|covered_hex| CoveredHex { + location: covered_hex.hex.into_raw(), + base_coverage_points: Some(covered_hex.points.base.proto_decimal()), + boosted_coverage_points: Some(covered_hex.points.boosted.proto_decimal()), + urbanized: covered_hex.assignments.urbanized.into(), + footfall: covered_hex.assignments.footfall.into(), + landtype: covered_hex.assignments.landtype.into(), + assignment_multiplier: Some(covered_hex.assignment_multiplier.proto_decimal()), + rank: covered_hex.rank as u32, + rank_multiplier: Some(covered_hex.rank_multiplier.proto_decimal()), + boosted_multiplier: covered_hex + .boosted_multiplier + .and_then(|x| x.to_u32()) + .unwrap_or_default(), + }) + .collect() + } +} diff --git a/mobile_verifier/src/rewarder.rs b/mobile_verifier/src/rewarder.rs index fbc53258c..638ec8c06 100644 --- a/mobile_verifier/src/rewarder.rs +++ b/mobile_verifier/src/rewarder.rs @@ -450,13 +450,20 @@ async fn reward_poc( { // handle poc reward outputs let mut allocated_poc_rewards = 0_u64; - for (poc_reward_amount, mobile_reward_share) in mobile_reward_shares { + for (poc_reward_amount, mobile_reward_share_v1, mobile_reward_share_v2) in + mobile_reward_shares + { allocated_poc_rewards += poc_reward_amount; mobile_rewards - .write(mobile_reward_share, []) + .write(mobile_reward_share_v1, []) .await? // Await the returned one shot to ensure that we wrote the file .await??; + mobile_rewards + .write(mobile_reward_share_v2, []) + .await? + // Await the returned one shot ot ensure that we wrote the file + .await??; } // calculate any unallocated poc reward ( diff --git a/mobile_verifier/tests/integrations/common/mod.rs b/mobile_verifier/tests/integrations/common/mod.rs index fc2465b74..7fb27d442 100644 --- a/mobile_verifier/tests/integrations/common/mod.rs +++ b/mobile_verifier/tests/integrations/common/mod.rs @@ -6,9 +6,9 @@ use file_store::{ use futures::{stream, StreamExt}; use helium_proto::{ services::poc_mobile::{ - mobile_reward_share::Reward as MobileReward, GatewayReward, MobileRewardShare, - OracleBoostingHexAssignment, OracleBoostingReportV1, RadioReward, ServiceProviderReward, - SpeedtestAvg, SubscriberReward, UnallocatedReward, + mobile_reward_share::Reward as MobileReward, radio_reward_v2, GatewayReward, + MobileRewardShare, OracleBoostingHexAssignment, OracleBoostingReportV1, RadioReward, + RadioRewardV2, ServiceProviderReward, SpeedtestAvg, SubscriberReward, UnallocatedReward, }, Message, }; @@ -19,10 +19,10 @@ use mobile_config::{ }; use mobile_verifier::boosting_oracles::AssignedCoverageObjects; -use rust_decimal::prelude::ToPrimitive; +use rust_decimal::{prelude::ToPrimitive, Decimal}; use rust_decimal_macros::dec; use sqlx::PgPool; -use std::collections::HashMap; +use std::{collections::HashMap, str::FromStr}; use tokio::{sync::mpsc::error::TryRecvError, time::timeout}; #[derive(Debug, Clone)] @@ -98,8 +98,8 @@ impl MockFileSinkReceiver { .collect() } - pub async fn receive_radio_reward(&mut self) -> RadioReward { - match self.receive("receive_radio_reward").await { + pub async fn receive_radio_reward_v1(&mut self) -> RadioReward { + match self.receive("receive_radio_reward_v1").await { Some(bytes) => { let mobile_reward = MobileRewardShare::decode(bytes.as_slice()) .expect("failed to decode expected radio reward"); @@ -113,6 +113,30 @@ impl MockFileSinkReceiver { } } + pub async fn receive_radio_reward(&mut self) -> RadioRewardV2 { + // NOTE(mj): When v1 rewards stop being written, remove this receiver + // and the comparison. + let radio_reward_v1 = self.receive_radio_reward_v1().await; + match self.receive("receive_radio_reward").await { + Some(bytes) => { + let mobile_reward = MobileRewardShare::decode(bytes.as_slice()) + .expect("failed to decode expected radio reward v2"); + match mobile_reward.reward { + Some(MobileReward::RadioRewardV2(reward)) => { + assert_eq!( + reward.total_poc_reward(), + radio_reward_v1.poc_reward, + "mismatch in poc rewards between v1 and v2" + ); + reward + } + _ => panic!("failed to get radio reward"), + } + } + None => panic!("failed to receive radio reward"), + } + } + pub async fn receive_gateway_reward(&mut self) -> GatewayReward { match self.receive("receive_gateway_reward").await { Some(bytes) => { @@ -185,6 +209,53 @@ pub fn create_file_sink() -> (FileSinkClient, MockFileSinkReceiver) { ) } +pub trait RadioRewardV2Ext { + fn boosted_hexes(&self) -> Vec; + fn nth_boosted_hex(&self, index: usize) -> radio_reward_v2::CoveredHex; + fn boosted_hexes_len(&self) -> usize; + fn total_poc_reward(&self) -> u64; + fn total_coverage_points(&self) -> u64; +} + +impl RadioRewardV2Ext for RadioRewardV2 { + fn boosted_hexes(&self) -> Vec { + self.covered_hexes.to_vec() + } + + fn boosted_hexes_len(&self) -> usize { + self.covered_hexes + .iter() + .filter(|hex| hex.boosted_multiplier > 0) + .collect::>() + .len() + } + + fn nth_boosted_hex(&self, index: usize) -> radio_reward_v2::CoveredHex { + self.covered_hexes + .iter() + .filter(|hex| hex.boosted_multiplier > 0) + .cloned() + .collect::>() + .get(index) + .unwrap_or_else(|| panic!("expected {index} in boosted_hexes")) + .clone() + } + + fn total_poc_reward(&self) -> u64 { + self.base_poc_reward + self.boosted_poc_reward + } + + fn total_coverage_points(&self) -> u64 { + let base = self.base_coverage_points_sum.clone().unwrap_or_default(); + let boosted = self.boosted_coverage_points_sum.clone().unwrap_or_default(); + + let base = Decimal::from_str(&base.value).expect("decoding base cp"); + let boosted = Decimal::from_str(&boosted.value).expect("decoding boosted cp"); + + (base + boosted).to_u64().unwrap() + } +} + pub fn seconds(s: u64) -> std::time::Duration { std::time::Duration::from_secs(s) } diff --git a/mobile_verifier/tests/integrations/hex_boosting.rs b/mobile_verifier/tests/integrations/hex_boosting.rs index a6de082e5..424cf9554 100644 --- a/mobile_verifier/tests/integrations/hex_boosting.rs +++ b/mobile_verifier/tests/integrations/hex_boosting.rs @@ -1,4 +1,4 @@ -use crate::common::{self, MockFileSinkReceiver, MockHexBoostingClient}; +use crate::common::{self, MockFileSinkReceiver, MockHexBoostingClient, RadioRewardV2Ext}; use chrono::{DateTime, Duration as ChronoDuration, Duration, Utc}; use file_store::{ coverage::{CoverageObject as FSCoverageObject, KeyType, RadioHexSignalLevel}, @@ -9,8 +9,8 @@ use helium_crypto::PublicKeyBinary; use helium_proto::services::{ poc_lora::UnallocatedRewardType, poc_mobile::{ - CoverageObjectValidity, HeartbeatValidity, RadioReward, SeniorityUpdateReason, SignalLevel, - UnallocatedReward, + CoverageObjectValidity, HeartbeatValidity, RadioRewardV2, SeniorityUpdateReason, + SignalLevel, UnallocatedReward, }, }; use hextree::Cell; @@ -199,28 +199,30 @@ async fn test_poc_with_boosted_hexes(pool: PgPool) -> anyhow::Result<()> { let exp_reward_3 = rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); - assert_eq!(exp_reward_1, hotspot_2.poc_reward); // 20x boost - assert_eq!(exp_reward_2, hotspot_1.poc_reward); // 10x boost - assert_eq!(exp_reward_3, hotspot_3.poc_reward); // no boost + assert_eq!(exp_reward_1, hotspot_2.total_poc_reward()); // 20x boost + assert_eq!(exp_reward_2, hotspot_1.total_poc_reward()); // 10x boost + assert_eq!(exp_reward_3, hotspot_3.total_poc_reward()); // no boost // assert the boosted hexes in the radio rewards // assert the number of boosted hexes for each radio - assert_eq!(1, hotspot_2.boosted_hexes.len()); - assert_eq!(1, hotspot_1.boosted_hexes.len()); - // hotspot 3 has no boosted hexes as all its hex boosts are 1x multiplier - // and those get filtered out as they dont affect points - assert_eq!(0, hotspot_3.boosted_hexes.len()); + assert_eq!(1, hotspot_2.boosted_hexes_len()); + assert_eq!(1, hotspot_1.boosted_hexes_len()); + // hotspot 3 has 1 boosted hex at 1x, it does not effect rewards, but all + // covered hexes are reported with their corresponding boost values. + assert_eq!(1, hotspot_3.boosted_hexes_len()); // assert the hex boost multiplier values - assert_eq!(20, hotspot_2.boosted_hexes[0].multiplier); - assert_eq!(10, hotspot_1.boosted_hexes[0].multiplier); + assert_eq!(20, hotspot_2.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(10, hotspot_1.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(1, hotspot_3.nth_boosted_hex(0).boosted_multiplier); // assert the hex boost location values - assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.boosted_hexes[0].location); - assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.boosted_hexes[0].location); + assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.nth_boosted_hex(0).location); + assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.nth_boosted_hex(0).location); // confirm the total rewards allocated matches expectations - let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; + let poc_sum = + hotspot_1.total_poc_reward() + hotspot_2.total_poc_reward() + hotspot_3.total_poc_reward(); let total = poc_sum + unallocated_reward.amount; assert_eq!(total_poc_emissions, total); @@ -337,29 +339,29 @@ async fn test_poc_boosted_hexes_thresholds_not_met(pool: PgPool) -> anyhow::Resu let exp_reward_2 = 16393442622950; let exp_reward_3 = 16393442622950; - assert_eq!(exp_reward_1, poc_rewards[0].poc_reward); + assert_eq!(exp_reward_1, poc_rewards[0].total_poc_reward()); assert_eq!( HOTSPOT_2.to_string(), PublicKeyBinary::from(poc_rewards[0].hotspot_key.clone()).to_string() ); - assert_eq!(exp_reward_2, poc_rewards[1].poc_reward); + assert_eq!(exp_reward_2, poc_rewards[1].total_poc_reward()); assert_eq!( HOTSPOT_1.to_string(), PublicKeyBinary::from(poc_rewards[1].hotspot_key.clone()).to_string() ); - assert_eq!(exp_reward_3, poc_rewards[2].poc_reward); + assert_eq!(exp_reward_3, poc_rewards[2].total_poc_reward()); assert_eq!( HOTSPOT_3.to_string(), PublicKeyBinary::from(poc_rewards[2].hotspot_key.clone()).to_string() ); // assert the number of boosted hexes for each radio - assert_eq!(0, poc_rewards[0].boosted_hexes.len()); - assert_eq!(0, poc_rewards[1].boosted_hexes.len()); - assert_eq!(0, poc_rewards[2].boosted_hexes.len()); + assert_eq!(0, poc_rewards[0].boosted_hexes_len()); + assert_eq!(0, poc_rewards[1].boosted_hexes_len()); + assert_eq!(0, poc_rewards[2].boosted_hexes_len()); // confirm the total rewards allocated matches expectations - let poc_sum: u64 = poc_rewards.iter().map(|r| r.poc_reward).sum(); + let poc_sum: u64 = poc_rewards.iter().map(|r| r.total_poc_reward()).sum(); let unallocated_sum: u64 = unallocated_reward.amount; let total = poc_sum + unallocated_sum; @@ -547,38 +549,42 @@ async fn test_poc_with_multi_coverage_boosted_hexes(pool: PgPool) -> anyhow::Res let exp_reward_2 = rounded(hex_coverage(1)) + rounded(boost_coverage(19)); let exp_reward_3 = rounded(hex_coverage(1)) + rounded(boost_coverage(0)); - assert_eq!(exp_reward_1, hotspot_1.poc_reward); // 2 at 10x boost - assert_eq!(exp_reward_2, hotspot_2.poc_reward); // 1 at 20x boost - assert_eq!(exp_reward_3, hotspot_3.poc_reward); // 1 at no boost + assert_eq!(exp_reward_1, hotspot_1.total_poc_reward()); // 2 at 10x boost + assert_eq!(exp_reward_2, hotspot_2.total_poc_reward()); // 1 at 20x boost + assert_eq!(exp_reward_3, hotspot_3.total_poc_reward()); // 1 at no boost // hotspot 1 and 2 should have the same coverage points, but different poc rewards. - assert_eq!(hotspot_1.coverage_points, hotspot_2.coverage_points); - assert_ne!(hotspot_1.poc_reward, hotspot_2.poc_reward); + assert_eq!( + hotspot_1.total_coverage_points(), + hotspot_2.total_coverage_points() + ); + assert_ne!(hotspot_1.total_poc_reward(), hotspot_2.total_poc_reward()); // assert the number of boosted hexes for each radio - assert_eq!(1, hotspot_2.boosted_hexes.len()); - assert_eq!(2, hotspot_1.boosted_hexes.len()); - // hotspot 3 has no boosted hexes as all its hex boosts are 1x multiplier - // and those get filtered out as they dont affect points - assert_eq!(0, hotspot_3.boosted_hexes.len()); + assert_eq!(1, hotspot_2.boosted_hexes_len()); + assert_eq!(2, hotspot_1.boosted_hexes_len()); + // hotspot 3 has 1 boosted hex at 1x, it does not effect rewards, but all + // covered hexes are reported with their corresponding boost values. + assert_eq!(1, hotspot_3.boosted_hexes_len()); // assert the hex boost multiplier values // as hotspot 3 has 2 covered hexes, it should have 2 boosted hexes // sort order in the vec for these is not guaranteed, so sort them - let mut hotspot_1_boosted_hexes = hotspot_1.boosted_hexes.clone(); + let mut hotspot_1_boosted_hexes = hotspot_1.boosted_hexes(); hotspot_1_boosted_hexes.sort_by(|a, b| b.location.cmp(&a.location)); - assert_eq!(20, hotspot_2.boosted_hexes[0].multiplier); - assert_eq!(10, hotspot_1_boosted_hexes[1].multiplier); - assert_eq!(10, hotspot_1_boosted_hexes[1].multiplier); + assert_eq!(20, hotspot_2.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(10, hotspot_1_boosted_hexes[0].boosted_multiplier); + assert_eq!(10, hotspot_1_boosted_hexes[1].boosted_multiplier); // assert the hex boost location values assert_eq!(0x8a1fb46622dffff_u64, hotspot_1_boosted_hexes[0].location); assert_eq!(0x8a1fb46622d7fff_u64, hotspot_1_boosted_hexes[1].location); - assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.boosted_hexes[0].location); + assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.nth_boosted_hex(0).location); // confirm the total rewards allocated matches expectations - let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; + let poc_sum = + hotspot_1.total_poc_reward() + hotspot_2.total_poc_reward() + hotspot_3.total_poc_reward(); let total = poc_sum + unallocated_reward.amount; assert_eq!(total_poc_emissions, total); @@ -670,17 +676,17 @@ async fn test_expired_boosted_hex(pool: PgPool) -> anyhow::Result<()> { let exp_reward_2 = 16_393_442_622_950; let exp_reward_3 = 16_393_442_622_950; - assert_eq!(exp_reward_1, poc_rewards[0].poc_reward); + assert_eq!(exp_reward_1, poc_rewards[0].total_poc_reward()); assert_eq!( HOTSPOT_2.to_string(), PublicKeyBinary::from(poc_rewards[0].hotspot_key.clone()).to_string() ); - assert_eq!(exp_reward_2, poc_rewards[1].poc_reward); + assert_eq!(exp_reward_2, poc_rewards[1].total_poc_reward()); assert_eq!( HOTSPOT_1.to_string(), PublicKeyBinary::from(poc_rewards[1].hotspot_key.clone()).to_string() ); - assert_eq!(exp_reward_3, poc_rewards[2].poc_reward); + assert_eq!(exp_reward_3, poc_rewards[2].total_poc_reward()); assert_eq!( HOTSPOT_3.to_string(), PublicKeyBinary::from(poc_rewards[2].hotspot_key.clone()).to_string() @@ -688,12 +694,12 @@ async fn test_expired_boosted_hex(pool: PgPool) -> anyhow::Result<()> { // assert the number of boosted hexes for each radio // all will be zero as the boost period has expired for the single boosted hex - assert_eq!(0, poc_rewards[0].boosted_hexes.len()); - assert_eq!(0, poc_rewards[1].boosted_hexes.len()); - assert_eq!(0, poc_rewards[2].boosted_hexes.len()); + assert_eq!(0, poc_rewards[0].boosted_hexes_len()); + assert_eq!(0, poc_rewards[1].boosted_hexes_len()); + assert_eq!(0, poc_rewards[2].boosted_hexes_len()); // confirm the total rewards allocated matches expectations - let poc_sum: u64 = poc_rewards.iter().map(|r| r.poc_reward).sum(); + let poc_sum: u64 = poc_rewards.iter().map(|r| r.total_poc_reward()).sum(); let unallocated_sum: u64 = unallocated_reward.amount; let total = poc_sum + unallocated_sum; @@ -851,26 +857,207 @@ async fn test_reduced_location_score_with_boosted_hexes(pool: PgPool) -> anyhow: let exp_reward_3 = rounded(regular_share * dec!(75)) + rounded(boosted_share * dec!(75) * dec!(0)); - assert_eq!(exp_reward_1, hotspot_1.poc_reward); - assert_eq!(exp_reward_2, hotspot_2.poc_reward); - assert_eq!(exp_reward_3, hotspot_3.poc_reward); + assert_eq!(exp_reward_1, hotspot_1.total_poc_reward()); + assert_eq!(exp_reward_2, hotspot_2.total_poc_reward()); + assert_eq!(exp_reward_3, hotspot_3.total_poc_reward()); // assert the number of boosted hexes for each radio //hotspot 1 has one boosted hex - assert_eq!(1, hotspot_1.boosted_hexes.len()); + assert_eq!(1, hotspot_1.boosted_hexes_len()); //hotspot 2 has no boosted hexes - assert_eq!(0, hotspot_2.boosted_hexes.len()); + assert_eq!(0, hotspot_2.boosted_hexes_len()); // hotspot 3 has a boosted location but as its location trust score // is reduced the boost does not get applied - assert_eq!(0, hotspot_3.boosted_hexes.len()); + assert_eq!(0, hotspot_3.boosted_hexes_len()); // assert the hex boost multiplier values // assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); - assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); - assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.boosted_hexes[0].location); + assert_eq!(2, hotspot_1.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.nth_boosted_hex(0).location); // confirm the total rewards allocated matches expectations - let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; + let poc_sum = + hotspot_1.total_poc_reward() + hotspot_2.total_poc_reward() + hotspot_3.total_poc_reward(); + let total = poc_sum + unallocated_reward.amount; + + let expected_sum = reward_shares::get_scheduled_tokens_for_poc(epoch.end - epoch.start) + .to_u64() + .unwrap(); + assert_eq!(expected_sum, total); + + // confirm the rewarded percentage amount matches expectations + let daily_total = reward_shares::get_total_scheduled_tokens(epoch.end - epoch.start); + let percent = (Decimal::from(total) / daily_total) + .round_dp_with_strategy(2, RoundingStrategy::MidpointNearestEven); + assert_eq!(percent, dec!(0.6)); + + Ok(()) +} + +#[sqlx::test] +async fn test_distance_from_asserted_removes_boosting_but_not_location_trust( + pool: PgPool, +) -> anyhow::Result<()> { + let (mobile_rewards_client, mut mobile_rewards) = common::create_file_sink(); + let (speedtest_avg_client, _speedtest_avg_server) = common::create_file_sink(); + let now = Utc::now(); + let epoch = (now - ChronoDuration::hours(24))..now; + let epoch_duration = epoch.end - epoch.start; + let boost_period_length = Duration::days(30); + + // seed all the things + let mut txn = pool.clone().begin().await?; + seed_heartbeats_with_location_trust( + epoch.start, + &mut txn, + // hotspot 1 can receive boosting + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + // hotspot 2 can receive boosting but has no boosted hexes + HotspotLocationTrust { + meters: 10, + multiplier: dec!(1.0), + }, + // hotspot 3 is too far for boosting + HotspotLocationTrust { + meters: 100, + multiplier: dec!(1.0), + }, + ) + .await?; + seed_speedtests(epoch.end, &mut txn).await?; + seed_radio_thresholds(epoch.start, &mut txn).await?; + txn.commit().await?; + update_assignments(&pool).await?; + + // setup boosted hex where reward start time is in the second period length + let multipliers1 = vec![NonZeroU32::new(2).unwrap()]; + let start_ts_1 = epoch.start; + let end_ts_1 = start_ts_1 + (boost_period_length * multipliers1.len() as i32); + + // setup boosted hex where no start or end time is set + let multipliers2 = vec![NonZeroU32::new(2).unwrap()]; + + let boosted_hexes = vec![ + BoostedHexInfo { + // hotspot 1's location + location: Cell::from_raw(0x8a1fb466d2dffff_u64)?, + start_ts: Some(start_ts_1), + end_ts: Some(end_ts_1), + period_length: boost_period_length, + multipliers: multipliers1, + boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), + boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), + version: 0, + }, + BoostedHexInfo { + // hotspot 3's location + location: Cell::from_raw(0x8c2681a306607ff_u64)?, + start_ts: None, + end_ts: None, + period_length: boost_period_length, + multipliers: multipliers2, + boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), + boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), + version: 0, + }, + ]; + + let hex_boosting_client = MockHexBoostingClient::new(boosted_hexes); + let total_poc_emissions = reward_shares::get_scheduled_tokens_for_poc(epoch_duration) + .to_u64() + .unwrap(); + + let (_, rewards) = tokio::join!( + // run rewards for poc and dc + rewarder::reward_poc_and_dc( + &pool, + &hex_boosting_client, + &mobile_rewards_client, + &speedtest_avg_client, + &epoch, + dec!(0.0001) + ), + receive_expected_rewards_maybe_unallocated( + &mut mobile_rewards, + ExpectUnallocated::NoWhenValue(total_poc_emissions) + ) + ); + + let Ok((poc_rewards, unallocated_reward)) = rewards else { + panic!("no rewards received"); + }; + + let mut poc_rewards = poc_rewards.iter(); + let hotspot_2 = poc_rewards.next().unwrap(); // full location trust NO boosts + let hotspot_1 = poc_rewards.next().unwrap(); // full location trust 1 boost + let hotspot_3 = poc_rewards.next().unwrap(); // reduced location trust 1 boost + assert_eq!( + None, + poc_rewards.next(), + "Received more hotspots than expected in rewards" + ); + assert_eq!( + HOTSPOT_1.to_string(), + PublicKeyBinary::from(hotspot_1.hotspot_key.clone()).to_string() + ); + assert_eq!( + HOTSPOT_2.to_string(), + PublicKeyBinary::from(hotspot_2.hotspot_key.clone()).to_string() + ); + assert_eq!( + HOTSPOT_3.to_string(), + PublicKeyBinary::from(hotspot_3.hotspot_key.clone()).to_string() + ); + + // Calculating expected rewards + let (regular_poc, boosted_poc) = get_poc_allocation_buckets(epoch_duration); + + // Here's how we get the regular shares per coverage points + // | base coverage point | speedtest | location | total | + // |---------------------|-----------|----------|-------| + // | 400 | 0.75 | 1.00 | 300 | + // | 400 | 0.75 | 1.00 | 300 | + // | 400 | 0.75 | 1.00 | 300 | + // |---------------------|-----------|----------|-------| + // | 900 | + let regular_share = regular_poc / dec!(900); + + // Boosted hexes are 2x, only one radio qualifies based on the location trust + // 300 * 1 == 300 + // To get points _only_ from boosting. + let boosted_share = boosted_poc / dec!(300); + + let exp_reward_1 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(1)); + let exp_reward_2 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); + let exp_reward_3 = + rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); + + assert_eq!(exp_reward_1, hotspot_1.total_poc_reward()); + assert_eq!(exp_reward_2, hotspot_2.total_poc_reward()); + assert_eq!(exp_reward_3, hotspot_3.total_poc_reward()); + + // assert the number of boosted hexes for each radio + //hotspot 1 has one boosted hex + assert_eq!(1, hotspot_1.boosted_hexes_len()); + //hotspot 2 has no boosted hexes + assert_eq!(0, hotspot_2.boosted_hexes_len()); + // hotspot 3 has a boosted location but as its location trust score + // is reduced the boost does not get applied + assert_eq!(0, hotspot_3.boosted_hexes_len()); + + // assert the hex boost multiplier values + // assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); + assert_eq!(2, hotspot_1.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.nth_boosted_hex(0).location); + + // confirm the total rewards allocated matches expectations + let poc_sum = + hotspot_1.total_poc_reward() + hotspot_2.total_poc_reward() + hotspot_3.total_poc_reward(); let total = poc_sum + unallocated_reward.amount; let expected_sum = reward_shares::get_scheduled_tokens_for_poc(epoch.end - epoch.start) @@ -1234,34 +1421,35 @@ async fn test_poc_with_cbrs_and_multi_coverage_boosted_hexes(pool: PgPool) -> an let exp_reward_3 = rounded(regular_share * dec!(75) * dec!(1)) + rounded(boosted_share * dec!(75) * dec!(0)); - assert_eq!(exp_reward_1, hotspot_1.poc_reward); - assert_eq!(exp_reward_2, hotspot_2.poc_reward); - assert_eq!(exp_reward_3, hotspot_3.poc_reward); + assert_eq!(exp_reward_1, hotspot_1.total_poc_reward()); + assert_eq!(exp_reward_2, hotspot_2.total_poc_reward()); + assert_eq!(exp_reward_3, hotspot_3.total_poc_reward()); // assert the number of boosted hexes for each radio - assert_eq!(1, hotspot_2.boosted_hexes.len()); - assert_eq!(2, hotspot_1.boosted_hexes.len()); - // hotspot 3 has no boosted hexes as all its hex boosts are 1x multiplier - // and those get filtered out as they dont affect points - assert_eq!(0, hotspot_3.boosted_hexes.len()); + assert_eq!(1, hotspot_2.boosted_hexes_len()); + assert_eq!(2, hotspot_1.boosted_hexes_len()); + // hotspot 3 has 1 boosted hex at 1x, it does not effect rewards, but all + // covered hexes are reported with their corresponding boost values. + assert_eq!(1, hotspot_3.boosted_hexes_len()); // assert the hex boost multiplier values // as hotspot 3 has 2 covered hexes, it should have 2 boosted hexes // sort order in the vec for these is not guaranteed, so sort them - let mut hotspot_1_boosted_hexes = hotspot_1.boosted_hexes.clone(); + let mut hotspot_1_boosted_hexes = hotspot_1.boosted_hexes(); hotspot_1_boosted_hexes.sort_by(|a, b| b.location.cmp(&a.location)); - assert_eq!(20, hotspot_2.boosted_hexes[0].multiplier); - assert_eq!(10, hotspot_1_boosted_hexes[1].multiplier); - assert_eq!(10, hotspot_1_boosted_hexes[1].multiplier); + assert_eq!(20, hotspot_2.nth_boosted_hex(0).boosted_multiplier); + assert_eq!(10, hotspot_1_boosted_hexes[1].boosted_multiplier); + assert_eq!(10, hotspot_1_boosted_hexes[1].boosted_multiplier); // assert the hex boost location values assert_eq!(0x8a1fb46622dffff_u64, hotspot_1_boosted_hexes[0].location); assert_eq!(0x8a1fb46622d7fff_u64, hotspot_1_boosted_hexes[1].location); - assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.boosted_hexes[0].location); + assert_eq!(0x8a1fb49642dffff_u64, hotspot_2.nth_boosted_hex(0).location); // confirm the total rewards allocated matches expectations - let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; + let poc_sum = + hotspot_1.total_poc_reward() + hotspot_2.total_poc_reward() + hotspot_3.total_poc_reward(); let total = poc_sum + unallocated_reward.amount; let expected_sum = reward_shares::get_scheduled_tokens_for_poc(epoch.end - epoch.start) @@ -1284,7 +1472,7 @@ fn rounded(num: Decimal) -> u64 { async fn receive_expected_rewards( mobile_rewards: &mut MockFileSinkReceiver, -) -> anyhow::Result<(Vec, UnallocatedReward)> { +) -> anyhow::Result<(Vec, UnallocatedReward)> { receive_expected_rewards_maybe_unallocated(mobile_rewards, ExpectUnallocated::Yes).await } @@ -1296,7 +1484,7 @@ enum ExpectUnallocated { async fn receive_expected_rewards_maybe_unallocated( mobile_rewards: &mut MockFileSinkReceiver, expect_unallocated: ExpectUnallocated, -) -> anyhow::Result<(Vec, UnallocatedReward)> { +) -> anyhow::Result<(Vec, UnallocatedReward)> { // get the filestore outputs from rewards run let radio_reward1 = mobile_rewards.receive_radio_reward().await; let radio_reward2 = mobile_rewards.receive_radio_reward().await; @@ -1309,7 +1497,7 @@ async fn receive_expected_rewards_maybe_unallocated( let unallocated_poc_reward = match expect_unallocated { ExpectUnallocated::Yes => mobile_rewards.receive_unallocated_reward().await, ExpectUnallocated::NoWhenValue(max_emission) => { - let total: u64 = poc_rewards.iter().map(|p| p.poc_reward).sum(); + let total: u64 = poc_rewards.iter().map(|p| p.total_poc_reward()).sum(); let emitted_is_total = total == max_emission; tracing::info!( emitted_is_total, diff --git a/mobile_verifier/tests/integrations/rewarder_poc_dc.rs b/mobile_verifier/tests/integrations/rewarder_poc_dc.rs index f3eaf1e4a..f018f31c9 100644 --- a/mobile_verifier/tests/integrations/rewarder_poc_dc.rs +++ b/mobile_verifier/tests/integrations/rewarder_poc_dc.rs @@ -1,4 +1,4 @@ -use crate::common::{self, MockFileSinkReceiver, MockHexBoostingClient}; +use crate::common::{self, MockFileSinkReceiver, MockHexBoostingClient, RadioRewardV2Ext}; use chrono::{DateTime, Duration as ChronoDuration, Utc}; use file_store::{ coverage::{CoverageObject as FSCoverageObject, KeyType, RadioHexSignalLevel}, @@ -6,7 +6,7 @@ use file_store::{ }; use helium_crypto::PublicKeyBinary; use helium_proto::services::poc_mobile::{ - CoverageObjectValidity, GatewayReward, HeartbeatValidity, RadioReward, SeniorityUpdateReason, + CoverageObjectValidity, GatewayReward, HeartbeatValidity, RadioRewardV2, SeniorityUpdateReason, SignalLevel, UnallocatedReward, UnallocatedRewardType, }; use mobile_verifier::{ @@ -62,17 +62,17 @@ async fn test_poc_and_dc_rewards(pool: PgPool) -> anyhow::Result<()> { let hotspot_1_reward = 9_758_001_263_661; let hotspot_2_reward = 39_032_005_054_644; let hotspot_3_reward = 390_320_050_546; - assert_eq!(hotspot_1_reward, poc_rewards[0].poc_reward); + assert_eq!(hotspot_1_reward, poc_rewards[0].total_poc_reward()); assert_eq!( HOTSPOT_1.to_string(), PublicKeyBinary::from(poc_rewards[0].hotspot_key.clone()).to_string() ); - assert_eq!(hotspot_2_reward, poc_rewards[1].poc_reward); + assert_eq!(hotspot_2_reward, poc_rewards[1].total_poc_reward()); assert_eq!( HOTSPOT_3.to_string(), PublicKeyBinary::from(poc_rewards[1].hotspot_key.clone()).to_string() ); - assert_eq!(hotspot_3_reward, poc_rewards[2].poc_reward); + assert_eq!(hotspot_3_reward, poc_rewards[2].total_poc_reward()); assert_eq!( HOTSPOT_2.to_string(), PublicKeyBinary::from(poc_rewards[2].hotspot_key.clone()).to_string() @@ -81,9 +81,9 @@ async fn test_poc_and_dc_rewards(pool: PgPool) -> anyhow::Result<()> { // assert the boosted hexes in the radio rewards // boosted hexes will contain the used multiplier for each boosted hex // in this test there are no boosted hexes - assert_eq!(0, poc_rewards[0].boosted_hexes.len()); - assert_eq!(0, poc_rewards[1].boosted_hexes.len()); - assert_eq!(0, poc_rewards[2].boosted_hexes.len()); + assert_eq!(0, poc_rewards[0].boosted_hexes_len()); + assert_eq!(0, poc_rewards[1].boosted_hexes_len()); + assert_eq!(0, poc_rewards[2].boosted_hexes_len()); // assert unallocated amount assert_eq!( @@ -110,7 +110,7 @@ async fn test_poc_and_dc_rewards(pool: PgPool) -> anyhow::Result<()> { ); // confirm the total rewards allocated matches expectations - let poc_sum: u64 = poc_rewards.iter().map(|r| r.poc_reward).sum(); + let poc_sum: u64 = poc_rewards.iter().map(|r| r.total_poc_reward()).sum(); let dc_sum: u64 = dc_rewards.iter().map(|r| r.dc_transfer_reward).sum(); let unallocated_sum: u64 = unallocated_poc_reward.amount; let total = poc_sum + dc_sum + unallocated_sum; @@ -133,7 +133,7 @@ async fn test_poc_and_dc_rewards(pool: PgPool) -> anyhow::Result<()> { async fn receive_expected_rewards( mobile_rewards: &mut MockFileSinkReceiver, -) -> anyhow::Result<(Vec, Vec, UnallocatedReward)> { +) -> anyhow::Result<(Vec, Vec, UnallocatedReward)> { // get the filestore outputs from rewards run // expect 3 gateway rewards for dc transfer diff --git a/reward_index/src/indexer.rs b/reward_index/src/indexer.rs index c30e1438b..e4f6fb847 100644 --- a/reward_index/src/indexer.rs +++ b/reward_index/src/indexer.rs @@ -113,8 +113,9 @@ impl Indexer { let mut hotspot_rewards: HashMap = HashMap::new(); while let Some(msg) = reward_shares.try_next().await? { - let (key, amount) = self.extract_reward_share(&msg)?; - *hotspot_rewards.entry(key).or_default() += amount; + if let Some((key, amount)) = self.extract_reward_share(&msg)? { + *hotspot_rewards.entry(key).or_default() += amount; + }; } for (reward_key, amount) in hotspot_rewards { @@ -131,79 +132,86 @@ impl Indexer { Ok(()) } - fn extract_reward_share(&self, msg: &[u8]) -> Result<(RewardKey, u64)> { + fn extract_reward_share(&self, msg: &[u8]) -> Result> { match self.mode { settings::Mode::Mobile => { let share = MobileRewardShare::decode(msg)?; match share.reward { - Some(MobileReward::RadioReward(r)) => Ok(( + Some(MobileReward::RadioReward(r)) => Ok(Some(( RewardKey { key: PublicKeyBinary::from(r.hotspot_key).to_string(), reward_type: RewardType::MobileGateway, }, r.poc_reward, - )), - Some(MobileReward::GatewayReward(r)) => Ok(( + ))), + Some(MobileReward::RadioRewardV2(_)) => { + // NOTE: Eventually radio_reward_v2 will replace + // radio_reward as the source of rewards, then + // radio_reward will cease to be written. + Ok(None) + } + + Some(MobileReward::GatewayReward(r)) => Ok(Some(( RewardKey { key: PublicKeyBinary::from(r.hotspot_key).to_string(), reward_type: RewardType::MobileGateway, }, r.dc_transfer_reward, - )), - Some(MobileReward::SubscriberReward(r)) => Ok(( + ))), + Some(MobileReward::SubscriberReward(r)) => Ok(Some(( RewardKey { key: bs58::encode(&r.subscriber_id).into_string(), reward_type: RewardType::MobileSubscriber, }, r.discovery_location_amount, - )), + ))), Some(MobileReward::ServiceProviderReward(r)) => { ServiceProvider::try_from(r.service_provider_id) .map(|sp| { - Ok(( + Ok(Some(( RewardKey { key: sp.to_string(), reward_type: RewardType::MobileServiceProvider, }, r.amount, - )) + ))) }) .map_err(|_| anyhow!("failed to decode service provider"))? } - Some(MobileReward::UnallocatedReward(r)) => Ok(( + Some(MobileReward::UnallocatedReward(r)) => Ok(Some(( RewardKey { key: self.unallocated_reward_key.clone(), reward_type: RewardType::MobileUnallocated, }, r.amount, - )), + ))), _ => bail!("got an invalid reward share"), } } settings::Mode::Iot => { let share = IotRewardShare::decode(msg)?; match share.reward { - Some(IotReward::GatewayReward(r)) => Ok(( + Some(IotReward::GatewayReward(r)) => Ok(Some(( RewardKey { key: PublicKeyBinary::from(r.hotspot_key).to_string(), reward_type: RewardType::IotGateway, }, r.witness_amount + r.beacon_amount + r.dc_transfer_amount, - )), - Some(IotReward::OperationalReward(r)) => Ok(( + ))), + Some(IotReward::OperationalReward(r)) => Ok(Some(( RewardKey { key: self.op_fund_key.clone(), reward_type: RewardType::IotOperational, }, r.amount, - )), - Some(IotReward::UnallocatedReward(r)) => Ok(( + ))), + Some(IotReward::UnallocatedReward(r)) => Ok(Some(( RewardKey { key: self.unallocated_reward_key.clone(), reward_type: RewardType::IotUnallocated, }, r.amount, - )), + ))), _ => bail!("got an invalid iot reward share"), } } From 632b995677ebc197b06490db6bedcd77934fb624 Mon Sep 17 00:00:00 2001 From: Michael Jeffrey Date: Wed, 17 Jul 2024 10:31:55 -0700 Subject: [PATCH 3/4] fix bad test file merge after merging unrebased branch (#845) mobile-rewards-v2 integration branch was based off the hip-119 branch. The hip-119 branch was merged, then the v2 inteagration branch was merged directly after without rebasing it onto master. Somehow, this resulted in a duplicate test. --- .../tests/integrations/hex_boosting.rs | 179 ------------------ 1 file changed, 179 deletions(-) diff --git a/mobile_verifier/tests/integrations/hex_boosting.rs b/mobile_verifier/tests/integrations/hex_boosting.rs index 424cf9554..7001bdc2c 100644 --- a/mobile_verifier/tests/integrations/hex_boosting.rs +++ b/mobile_verifier/tests/integrations/hex_boosting.rs @@ -1074,185 +1074,6 @@ async fn test_distance_from_asserted_removes_boosting_but_not_location_trust( Ok(()) } -#[sqlx::test] -async fn test_distance_from_asserted_removes_boosting_but_not_location_trust( - pool: PgPool, -) -> anyhow::Result<()> { - let (mobile_rewards_client, mut mobile_rewards) = common::create_file_sink(); - let (speedtest_avg_client, _speedtest_avg_server) = common::create_file_sink(); - let now = Utc::now(); - let epoch = (now - ChronoDuration::hours(24))..now; - let epoch_duration = epoch.end - epoch.start; - let boost_period_length = Duration::days(30); - - // seed all the things - let mut txn = pool.clone().begin().await?; - seed_heartbeats_with_location_trust( - epoch.start, - &mut txn, - // hotspot 1 can receive boosting - HotspotLocationTrust { - meters: 10, - multiplier: dec!(1.0), - }, - // hotspot 2 can receive boosting but has no boosted hexes - HotspotLocationTrust { - meters: 10, - multiplier: dec!(1.0), - }, - // hotspot 3 is too far for boosting - HotspotLocationTrust { - meters: 100, - multiplier: dec!(1.0), - }, - ) - .await?; - seed_speedtests(epoch.end, &mut txn).await?; - seed_radio_thresholds(epoch.start, &mut txn).await?; - txn.commit().await?; - update_assignments(&pool).await?; - - // setup boosted hex where reward start time is in the second period length - let multipliers1 = vec![NonZeroU32::new(2).unwrap()]; - let start_ts_1 = epoch.start; - let end_ts_1 = start_ts_1 + (boost_period_length * multipliers1.len() as i32); - - // setup boosted hex where no start or end time is set - let multipliers2 = vec![NonZeroU32::new(2).unwrap()]; - - let boosted_hexes = vec![ - BoostedHexInfo { - // hotspot 1's location - location: Cell::from_raw(0x8a1fb466d2dffff_u64)?, - start_ts: Some(start_ts_1), - end_ts: Some(end_ts_1), - period_length: boost_period_length, - multipliers: multipliers1, - boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), - boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), - version: 0, - }, - BoostedHexInfo { - // hotspot 3's location - location: Cell::from_raw(0x8c2681a306607ff_u64)?, - start_ts: None, - end_ts: None, - period_length: boost_period_length, - multipliers: multipliers2, - boosted_hex_pubkey: Pubkey::from_str(BOOST_HEX_PUBKEY).unwrap(), - boost_config_pubkey: Pubkey::from_str(BOOST_CONFIG_PUBKEY).unwrap(), - version: 0, - }, - ]; - - let hex_boosting_client = MockHexBoostingClient::new(boosted_hexes); - let total_poc_emissions = reward_shares::get_scheduled_tokens_for_poc(epoch_duration) - .to_u64() - .unwrap(); - - let (_, rewards) = tokio::join!( - // run rewards for poc and dc - rewarder::reward_poc_and_dc( - &pool, - &hex_boosting_client, - &mobile_rewards_client, - &speedtest_avg_client, - &epoch, - dec!(0.0001) - ), - receive_expected_rewards_maybe_unallocated( - &mut mobile_rewards, - ExpectUnallocated::NoWhenValue(total_poc_emissions) - ) - ); - - let Ok((poc_rewards, unallocated_reward)) = rewards else { - panic!("no rewards received"); - }; - - let mut poc_rewards = poc_rewards.iter(); - let hotspot_2 = poc_rewards.next().unwrap(); // full location trust NO boosts - let hotspot_1 = poc_rewards.next().unwrap(); // full location trust 1 boost - let hotspot_3 = poc_rewards.next().unwrap(); // reduced location trust 1 boost - assert_eq!( - None, - poc_rewards.next(), - "Received more hotspots than expected in rewards" - ); - assert_eq!( - HOTSPOT_1.to_string(), - PublicKeyBinary::from(hotspot_1.hotspot_key.clone()).to_string() - ); - assert_eq!( - HOTSPOT_2.to_string(), - PublicKeyBinary::from(hotspot_2.hotspot_key.clone()).to_string() - ); - assert_eq!( - HOTSPOT_3.to_string(), - PublicKeyBinary::from(hotspot_3.hotspot_key.clone()).to_string() - ); - - // Calculating expected rewards - let (regular_poc, boosted_poc) = get_poc_allocation_buckets(epoch_duration); - - // Here's how we get the regular shares per coverage points - // | base coverage point | speedtest | location | total | - // |---------------------|-----------|----------|-------| - // | 400 | 0.75 | 1.00 | 300 | - // | 400 | 0.75 | 1.00 | 300 | - // | 400 | 0.75 | 1.00 | 300 | - // |---------------------|-----------|----------|-------| - // | 900 | - let regular_share = regular_poc / dec!(900); - - // Boosted hexes are 2x, only one radio qualifies based on the location trust - // 300 * 1 == 300 - // To get points _only_ from boosting. - let boosted_share = boosted_poc / dec!(300); - - let exp_reward_1 = - rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(1)); - let exp_reward_2 = - rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); - let exp_reward_3 = - rounded(regular_share * dec!(300)) + rounded(boosted_share * dec!(300) * dec!(0)); - - assert_eq!(exp_reward_1, hotspot_1.poc_reward); - assert_eq!(exp_reward_2, hotspot_2.poc_reward); - assert_eq!(exp_reward_3, hotspot_3.poc_reward); - - // assert the number of boosted hexes for each radio - //hotspot 1 has one boosted hex - assert_eq!(1, hotspot_1.boosted_hexes.len()); - //hotspot 2 has no boosted hexes - assert_eq!(0, hotspot_2.boosted_hexes.len()); - // hotspot 3 has a boosted location but as its location trust score - // is reduced the boost does not get applied - assert_eq!(0, hotspot_3.boosted_hexes.len()); - - // assert the hex boost multiplier values - // assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); - assert_eq!(2, hotspot_1.boosted_hexes[0].multiplier); - assert_eq!(0x8a1fb466d2dffff_u64, hotspot_1.boosted_hexes[0].location); - - // confirm the total rewards allocated matches expectations - let poc_sum = hotspot_1.poc_reward + hotspot_2.poc_reward + hotspot_3.poc_reward; - let total = poc_sum + unallocated_reward.amount; - - let expected_sum = reward_shares::get_scheduled_tokens_for_poc(epoch.end - epoch.start) - .to_u64() - .unwrap(); - assert_eq!(expected_sum, total); - - // confirm the rewarded percentage amount matches expectations - let daily_total = reward_shares::get_total_scheduled_tokens(epoch.end - epoch.start); - let percent = (Decimal::from(total) / daily_total) - .round_dp_with_strategy(2, RoundingStrategy::MidpointNearestEven); - assert_eq!(percent, dec!(0.6)); - - Ok(()) -} - #[sqlx::test] async fn test_poc_with_cbrs_and_multi_coverage_boosted_hexes(pool: PgPool) -> anyhow::Result<()> { let (mobile_rewards_client, mut mobile_rewards) = common::create_file_sink(); From 89d3820adc36589b925ee60728a2769c64f1d976 Mon Sep 17 00:00:00 2001 From: Matthew Plant Date: Sun, 21 Jul 2024 23:54:33 -0400 Subject: [PATCH 4/4] Make `modeled` field in `HexPoints` public (#842) --- coverage_point_calculator/src/hexes.rs | 2 +- file_store/src/reward_manifest.rs | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/coverage_point_calculator/src/hexes.rs b/coverage_point_calculator/src/hexes.rs index a68da1ccc..c784390f7 100644 --- a/coverage_point_calculator/src/hexes.rs +++ b/coverage_point_calculator/src/hexes.rs @@ -29,7 +29,7 @@ pub struct HexPoints { /// /// This is a convenience field for debugging, hexes can reach similar /// values through different means, it helps to know the starting value. - modeled: Decimal, + pub modeled: Decimal, /// Points including Coverage affected multipliers /// /// modeled + (Rank * Assignment) diff --git a/file_store/src/reward_manifest.rs b/file_store/src/reward_manifest.rs index ac8dab666..6942da221 100644 --- a/file_store/src/reward_manifest.rs +++ b/file_store/src/reward_manifest.rs @@ -14,8 +14,8 @@ pub struct RewardManifest { #[derive(Clone, Debug)] pub enum RewardData { MobileRewardData { - poc_bones_per_coverage_point: Decimal, - boosted_poc_bones_per_coverage_point: Decimal, + poc_bones_per_reward_share: Decimal, + boosted_poc_bones_per_reward_share: Decimal, }, IotRewardData { poc_bones_per_beacon_reward_share: Decimal, @@ -49,16 +49,16 @@ impl TryFrom for RewardManifest { reward_data: match value.reward_data { Some(proto::reward_manifest::RewardData::MobileRewardData(reward_data)) => { Some(RewardData::MobileRewardData { - poc_bones_per_coverage_point: reward_data + poc_bones_per_reward_share: reward_data .poc_bones_per_reward_share - .ok_or(DecodeError::empty_field("poc_bones_per_coverage_point"))? + .ok_or(DecodeError::empty_field("poc_bones_per_reward_share"))? .value .parse() .map_err(DecodeError::from)?, - boosted_poc_bones_per_coverage_point: reward_data + boosted_poc_bones_per_reward_share: reward_data .boosted_poc_bones_per_reward_share .ok_or(DecodeError::empty_field( - "boosted_poc_bones_per_coverage_point", + "boosted_poc_bones_per_reward_share", ))? .value .parse()