Update pallet-nis to support Block Number Provider #6764

Doordashcon · 2024-12-04T21:57:21Z

Part of #6297

Review Notes

Default Bound Requirement:ReceiptRecord & SummaryRecord types used in the Receipts and Summary storage items respectively, now adopts T::BlockNumberProvider with the condition that concrete types implement Default

Doordashcon · 2024-12-08T16:35:51Z

Hello @gui1117, please review

substrate/frame/nis/src/lib.rs

bkchr · 2024-12-08T20:31:25Z

substrate/frame/nis/src/lib.rs

@@ -693,7 +702,7 @@ pub mod pallet {
 			let (owner, mut on_hold) = receipt.owner.ok_or(Error::<T>::AlreadyCommunal)?;
 			ensure!(owner == who, Error::<T>::NotOwner);

-			let now = frame_system::Pallet::<T>::block_number();
+			let now = T::BlockNumberProvider::current_block_number();


Misses a migration for the Receipts to change the expiry to the "new clock". (Not required when switched to System as block number provider.

Most of those PR is for pallet in the relay chain to go to a parachain without requiring migration.

But in the wider scope I can see parachain could also make use of the relay chain block number provider when they are going more agile and skip some blocks.
In this case a migration would help indeed. We can just provide a function helper to do this migration, no need for storage versioning. Because parachain could change the configuration of the block number whenever they want, so unrelated to the version of the pallet.

@kianenigma do you think we should provide the block number migration function for all such pallets? I think some approved one don't have it.
Or should it be done on-demand when people ask for it?

I would not include a migration for the time being, assuming the PR and code-doc are super clear under which circumstances you do, and do not need a migration.

But in the wider scope I can see parachain could also make use of the relay chain block number provider when they are going more agile and skip some blocks.

This is the entire point of this exercise. Otherwise it is quite useless. Not sure why we approve prs without a migration. Also requiring everyone to write their own migration is a little bit stupid. Especially as by just "reading the docs" it will not be that easy. Block numbers are stored internally in some structures and may not be that obvious for others to find them etc. This will just end up in a mess, if people are forced to write their own migrations.

Gone ahead writing a migration

gui1117 · 2024-12-09T01:09:45Z

substrate/frame/nis/src/mock.rs

+		);
+		Nis::on_initialize(
+			<Test as pallet_nis::Config>::BlockNumberProvider::current_block_number(),
+		);


on_finalize/on_initialize is taking system block number. Here is works because both pallet_nis block number and system block number are the same.
I think the code is cleaner if we use system block number same as before in this test no?

Thanks for the review

gui1117 · 2025-01-25T06:10:05Z

substrate/frame/nis/src/lib.rs

+	pub type BlockNumberFor<T> =
+		<<T as Config>::BlockNumberProvider as BlockNumberProvider>::BlockNumber;
+	pub type BalanceOf<T> =


I don't mind reexporting types, but those names are so common I am afraid it will crate unecessary conflicts and bad IDE suggestions.

Suggested change

pub type BlockNumberFor<T> =

<<T as Config>::BlockNumberProvider as BlockNumberProvider>::BlockNumber;

pub type BalanceOf<T> =

pub(crate) type BlockNumberFor<T> =

<<T as Config>::BlockNumberProvider as BlockNumberProvider>::BlockNumber;

pub(crate) type BalanceOf<T> =

gui1117 · 2025-01-25T06:11:02Z

substrate/frame/nis/src/lib.rs

 		<<T as Config>::Currency as FunInspect<<T as frame_system::Config>::AccountId>>::Balance;
 	type DebtOf<T> =
 		fungible::Debt<<T as frame_system::Config>::AccountId, <T as Config>::Currency>;
-	type ReceiptRecordOf<T> =
+	pub type ReceiptRecordOf<T> =


Suggested change

pub type ReceiptRecordOf<T> =

pub(crate) type ReceiptRecordOf<T> =

gui1117 · 2025-01-25T06:17:59Z

substrate/frame/nis/src/migration.rs

+		<T as frame_system::Config>::AccountId,
+		SystemBlockNumberFor<T>,
+		BalanceOf<T>,
+	>;


we would also need to migrate Summary storage.

gui1117 · 2025-01-25T06:22:25Z

substrate/frame/nis/src/migration.rs

+			// Iterate over v0::Receipts and update the expiry in Receipts
+			v0::Receipts::<T>::iter().for_each(|(index, v0_receipt)| {
+				// Use mutate to modify the value directly in the storage
+				Receipts::<T>::mutate(index, |maybe_receipt| {


Doesn't look good, Here we are decoding the value with the old type. Note that system block number and relay block number can be different types like u32 and u64.

We should only write the new value.

... if StorageVersion::get::<Pallet<T>>() == 0 { Receipts::<T>::translate(|index, old_receipt: v0::OldReceiptRecordOf<T>| { call_count.saturating_inc(); log::info!(target: TARGET, "migrating reciept record expiry #{:?}", &index); let new_expiry = BlockConversion::convert_block_number_to_relay_height(old_receipt.expiry); Some(ReceiptRecord { proportion: old_receipt.proportion, owner: old_receipt.owner, expiry: new_expiry, }) }); .... } ...

muharem · 2025-02-03T06:06:34Z

@Doordashcon please check this comment - #6297 (comment)

muharem · 2025-02-07T07:30:39Z

substrate/frame/nis/src/lib.rs

+		/// # Example: Using Relay Chain Block Numbers
+		/// ```rust,ignore
+		/// impl Config for Runtime {
+		///     type BlockNumberProvider = RelayChainBlockNumber;


Suggested change

/// type BlockNumberProvider = RelayChainBlockNumber;

/// type BlockNumberProvider = frame_system::Pallet<Runtime>;

muharem · 2025-02-07T07:30:49Z

substrate/frame/nis/src/lib.rs

+		///
+		/// # Warning
+		/// Only implement custom providers if you need alternative timekeeping - most
+		/// pallets should use the system block number through `frame_system::Pallet<T>`.


Suggested change

/// pallets should use the system block number through `frame_system::Pallet<T>`.

/// pallets should use the system block number through `frame_system::Pallet<Runtime>`.

muharem · 2025-02-07T07:37:01Z

substrate/frame/nis/src/lib.rs

 			let mut weight_counter =
 				WeightCounter { used: Weight::zero(), limit: T::MaxIntakeWeight::get() };
-			if T::IntakePeriod::get().is_zero() || (n % T::IntakePeriod::get()).is_zero() {
+			if T::IntakePeriod::get().is_zero() || (block_number % T::IntakePeriod::get()).is_zero()


This wont work. I mentioned in the issue

Ensure pallet hooks like on_initialize(n) are compatible with the BlockNumberProvider and can handle cases when the hook isn’t called on every tick (e.g., adapt conditions that rely on exact matches like compilation_time == BlockNumberProvider::current_block_number()).

proposed fix @muharem

If IntakePeriod is not at every block then process_queues will execute not long after the IntakePeriod.

fn on_initialize(_n: SystemBlockNumberFor<T>) -> Weight { let block_number = T::BlockNumberProvider::current_block_number(); let mut weight_counter = WeightCounter { used: Weight::zero(), limit: T::MaxIntakeWeight::get() }; let last_processed = LastProcessedBlock::<T>::get(); let intake_period = T::IntakePeriod::get(); if intake_period.is_zero() || block_number.saturating_sub(last_processed) >= intake_period { if weight_counter.check_accrue(T::WeightInfo::process_queues()) { LastProcessedBlock::<T>::put(block_number); weight_counter.check_accrue(T::DbWeight::get().writes(1)); Self::process_queues( T::Target::get(), T::QueueCount::get(), u32::max_value(), &mut weight_counter, ) } } weight_counter.used }

muharem · 2025-02-07T07:42:44Z

substrate/frame/nis/src/migration.rs

+}
+
+/// Migrate the pallet storage from `0` to `1`.
+pub type MigrateV0ToV1<T, BlockConversion> = frame_support::migrations::VersionedMigration<


so everyone who uses this pallet should migrate to RC block numbers? why? this PR switches storage version to 1, from 0. this means this migration should be applied by all setups? what if my setup stays on Relay Chain? what if my setup stays on Parachain with system block provider?

Working on a fix

please review @muharem

muharem · 2025-02-07T07:48:11Z

@Doordashcon please let me know if you feel that we cannot get this merged by 17 Feb

Doordashcon · 2025-02-17T16:52:05Z

please review @gui1117 @muharem @bkchr

bkchr · 2025-02-17T17:11:47Z

substrate/frame/nis/src/lib.rs

+		/// - Custom timekeeping mechanisms (e.g. mock timelines for testing)
+		/// - Composite block numbers (e.g. parachain blocks vs relay chain blocks)
+		///
+		/// # Example: Using Relay Chain Block Numbers


Suggested change

/// # Example: Using Relay Chain Block Numbers

/// # Example: Using the local chain block numbers

bkchr · 2025-02-17T17:12:26Z

substrate/frame/nis/src/migration.rs

@@ -0,0 +1,261 @@
+use crate::*;


Missing license header

bkchr · 2025-02-17T17:13:10Z

substrate/frame/nis/src/lib.rs

+		///
+		/// # Warning
+		/// Only implement custom providers if you need alternative timekeeping - most
+		/// pallets should use the system block number through `frame_system::Pallet<Runtime>`.


Suggested change

///

/// # Warning

/// Only implement custom providers if you need alternative timekeeping - most

/// pallets should use the system block number through `frame_system::Pallet<Runtime>`.

Not sure this is that useful.

bkchr · 2025-02-17T17:13:37Z

substrate/frame/nis/src/lib.rs

@@ -407,6 +435,9 @@ pub mod pallet {
 	pub type Receipts<T> =
 		StorageMap<_, Blake2_128Concat, ReceiptIndex, ReceiptRecordOf<T>, OptionQuery>;

+	#[pallet::storage]
+	pub type LastProcessedBlock<T> = StorageValue<_, BlockNumberFor<T>, ValueQuery>;


Missing docs

bkchr · 2025-02-17T17:15:33Z

substrate/frame/nis/src/lib.rs

-			if T::IntakePeriod::get().is_zero() || (n % T::IntakePeriod::get()).is_zero() {
+			let last_processed = LastProcessedBlock::<T>::get();
+			let intake_period = T::IntakePeriod::get();
+			if intake_period.is_zero() ||


Suggested change

if intake_period.is_zero() ||

if

saturating_sub will be always at least zero and thus, we don't need this check for zero.

bkchr · 2025-02-17T17:17:28Z

substrate/frame/nis/src/migration.rs

+			"running migration with onchain storage version {:?}", on_chain_version
+		);
+
+		if on_chain_version == 0 {


Does not needs to be bound to the storage version. Also later you could do a change of the block number source.

bkchr · 2025-02-17T17:19:28Z

substrate/frame/nis/src/migration.rs

+	/// The log target.
+	const TARGET: &'static str = "runtime::nis::migration::change_block_number_provider";
+
+	pub trait BlockNumberConversion<Old, New> {


Could be just the generic convert trait.

Doordashcon added 5 commits December 4, 2024 13:55

include BlockNumberProvider in config

7aab0eb

BlockNumberProvider: Default

f2bfa0c

add prdoc & benchmark helper function

f947458

remove BlockNumberFor import benchmarks

edc68f2

Merge branch 'master' into BNP-pallet-nis

f73ae91

Doordashcon marked this pull request as ready for review December 8, 2024 16:34

Doordashcon requested a review from a team as a code owner December 8, 2024 16:34

bkchr reviewed Dec 8, 2024

View reviewed changes

gui1117 reviewed Dec 9, 2024

View reviewed changes

add migration file

7e38fde

gui1117 added the T2-pallets This PR/Issue is related to a particular pallet. label Dec 18, 2024

WIP migration

64d0bb8

paritytech-review-bot bot requested a review from a team January 7, 2025 03:39

Doordashcon added 2 commits January 7, 2025 19:00

remove convert_block_length_to_relay_length

fafd840

BNP config doc

ef38d00

seadanda mentioned this pull request Jan 15, 2025

[tracking] adopt BlockNumberProvider for the pallets migrating to AH #6297

Open

gui1117 reviewed Jan 25, 2025

View reviewed changes

Doordashcon added 2 commits February 4, 2025 05:48

migrate last_period for Summary storage item also

c2fbdfa

adds documentation for BlockNumberProvider

7fe4577

paritytech-review-bot bot requested a review from a team February 4, 2025 05:29

Doordashcon added 6 commits February 3, 2025 21:29

Merge branch 'master' into BNP-pallet-nis

148c4ab

add generics

f7c02fc

configure weight cal

601b043

use taplo

3ab5a0a

nit

fcf5395

log/std feature

48d3af5

muharem requested changes Feb 7, 2025

View reviewed changes

adjusting migration file..., introduce LastProcessedBlock

2a41a85

paritytech-review-bot bot requested a review from a team February 9, 2025 20:42

Doordashcon added 7 commits February 9, 2025 21:47

fix BlockNumberProvider Doc

b075dec

decode to current type

3fa3627

Merge branch 'master' into BNP-pallet-nis

542c7ca

nit

80f7aff

fmt

f170935

Merge branch 'master' into BNP-pallet-nis

8205841

add try-runtime to tests

70a7e5d

bkchr reviewed Feb 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update pallet-nis to support Block Number Provider #6764

Update pallet-nis to support Block Number Provider #6764

Doordashcon commented Dec 4, 2024 •

edited

Loading

Doordashcon commented Dec 8, 2024

bkchr Dec 8, 2024

gui1117 Dec 9, 2024 •

edited

Loading

kianenigma Dec 9, 2024

bkchr Dec 9, 2024 •

edited

Loading

Doordashcon Jan 7, 2025

gui1117 Dec 9, 2024

Doordashcon Jan 7, 2025

gui1117 Jan 25, 2025

gui1117 Jan 25, 2025

gui1117 Jan 25, 2025

gui1117 Jan 25, 2025

Doordashcon Feb 4, 2025

muharem commented Feb 3, 2025

muharem Feb 7, 2025

muharem Feb 7, 2025

muharem Feb 7, 2025

Doordashcon Feb 9, 2025

muharem Feb 7, 2025

Doordashcon Feb 9, 2025

Doordashcon Feb 11, 2025

muharem commented Feb 7, 2025

Doordashcon commented Feb 17, 2025 •

edited

Loading

bkchr Feb 17, 2025

bkchr Feb 17, 2025

bkchr Feb 17, 2025

bkchr Feb 17, 2025

bkchr Feb 17, 2025

bkchr Feb 17, 2025

bkchr Feb 17, 2025

	pub type ReceiptRecordOf<T> =
	pub(crate) type ReceiptRecordOf<T> =

	/// type BlockNumberProvider = RelayChainBlockNumber;
	/// type BlockNumberProvider = frame_system::Pallet<Runtime>;

	/// pallets should use the system block number through `frame_system::Pallet<T>`.
	/// pallets should use the system block number through `frame_system::Pallet<Runtime>`.

	/// # Example: Using Relay Chain Block Numbers
	/// # Example: Using the local chain block numbers

Update pallet-nis to support Block Number Provider #6764

Are you sure you want to change the base?

Update pallet-nis to support Block Number Provider #6764

Conversation

Doordashcon commented Dec 4, 2024 • edited Loading

Review Notes

Doordashcon commented Dec 8, 2024

Choose a reason for hiding this comment

gui1117 Dec 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bkchr Dec 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

muharem commented Feb 3, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

muharem commented Feb 7, 2025

Doordashcon commented Feb 17, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Doordashcon commented Dec 4, 2024 •

edited

Loading

gui1117 Dec 9, 2024 •

edited

Loading

bkchr Dec 9, 2024 •

edited

Loading

Doordashcon commented Feb 17, 2025 •

edited

Loading