Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Apply Forest root changes to Buckets in Blockchain Service #335

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
7e501b6
fix: :rotating_light: Remove error rust analyzer was complaining abou…
ffarall Jan 3, 2025
f5267b5
chore: :see_no_evil: Gitignore cSpell config file
ffarall Jan 6, 2025
af7b41b
feat: :construction: Emit events of mutations applied always
ffarall Jan 6, 2025
02880e6
feat: :construction: Half way implementing apply forest root changes
ffarall Jan 7, 2025
f3644e7
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 9, 2025
67e175b
chore: :see_no_evil: Typo in cSpell gitignore
ffarall Jan 9, 2025
60048fc
fix: :rotating_light: Fix event fields after merge
ffarall Jan 9, 2025
bdedae5
feat: :construction: Implement applying `TrieMutation`s in reverse
ffarall Jan 9, 2025
1190017
feat: :construction: Implement forest root catch ups for BSPs
ffarall Jan 10, 2025
61ab757
chore: :label: Run `pnpm typegen`
ffarall Jan 10, 2025
4e1d9a2
fix: :white_check_mark: Use new `MutationsAppliedForProvider` event i…
ffarall Jan 10, 2025
c2db1de
fix: :bug: Create tree route with old and new best block in regular n…
ffarall Jan 10, 2025
1d038d0
feat: :loud_sound: Improve logs for apply forest root changes
ffarall Jan 10, 2025
3034eab
feat: :fire: Remove now unused code of applying forest root changes i…
ffarall Jan 10, 2025
3b52ab0
feat: :fire: Remove forest root changing from `bsp_charge_fees` and `…
ffarall Jan 10, 2025
c24fa3c
fix: :white_check_mark: Wait for new log from blockchain service to c…
ffarall Jan 10, 2025
891672c
fix: :white_check_mark: Fix race condition in onboarding by using `wa…
ffarall Jan 10, 2025
ab108f0
test: :white_check_mark: Cleanup reorgs test
ffarall Jan 10, 2025
57c1cde
style: :art: Apply cargo format
ffarall Jan 11, 2025
af21adf
feat: :loud_sound: Improve error logs for failed extrinsics in `sealB…
ffarall Jan 13, 2025
c512c8e
test: :bulb: Improve comment docs in reorg utilities
ffarall Jan 13, 2025
de2e8af
fix: :adhesive_bandage: Remove type parameter to please rust analyzer
ffarall Jan 13, 2025
fd220fb
feat: :sparkles: Add optional block sealing to `wait.bspStored`
ffarall Jan 13, 2025
32872b0
fix: :bug: Fix `Multiaddresses` type in `types-bundle`
ffarall Jan 13, 2025
a458cc0
fix: 🐛 Fix `BackupStorageProvider` type in `types-bundle`
ffarall Jan 13, 2025
be8da1b
chore: :label: Fix wrong `StorageDataType` in types-bundle
ffarall Jan 13, 2025
9dd2f5a
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 13, 2025
abffd18
fix: :label: Fix `StorageDataType` type in `types-bundle`
ffarall Jan 13, 2025
d462936
test: :construction: Add tests for reorgs with forest root changes
ffarall Jan 13, 2025
0c35b0d
feat: :sparkles: Add optional `nonce` to `block.seal` testing functio…
ffarall Jan 14, 2025
b9a7371
test: :construction: Test failing in reorg with file deletion confirm…
ffarall Jan 15, 2025
226d61f
fix: :bug: Fix building of `tree_route` both for reorgs and new best …
ffarall Jan 16, 2025
90382b0
test: :white_check_mark: Test scenario with reorgs with changes retra…
ffarall Jan 16, 2025
ed774e1
fix: :adhesive_bandage: Fix wrong error logging when reverting change…
ffarall Jan 16, 2025
f7245b5
test: :white_check_mark: Remove `only` flag leftover
ffarall Jan 16, 2025
77da01f
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 16, 2025
fdf92d8
chore: :label: Run typegen after merge
ffarall Jan 16, 2025
3f891fb
docs: :memo: Add test diagram for reorgs test
ffarall Jan 16, 2025
f88b6f1
refactor: :truck: Move `enqueue_expiration_item` out of runtime API i…
ffarall Jan 20, 2025
75a4033
feat: :sparkles: Add `query_buckets_for_msp` runtime API
ffarall Jan 20, 2025
9fc2f1d
feat: :sparkles: Add `event_info` field to `generic_apply_mutations` …
ffarall Jan 21, 2025
822e84a
fix: :rotating_light: Add new runtime API to xcm simulator runtime
ffarall Jan 22, 2025
9988de6
feat: :construction: Apply Bucket's Forest root changes in Blockchain…
ffarall Jan 22, 2025
6ff9dc1
feat: :fire: Remove forest root mutation from task MSP
ffarall Jan 22, 2025
44f0643
fix: :fire: Remove file that should have been ignored
ffarall Jan 22, 2025
ad35490
chore: :label: Add new runtime API to `types-bundle`
ffarall Jan 22, 2025
384fb65
chore: :label: Run `pnpm typegen`
ffarall Jan 22, 2025
5b984a6
Merge branch 'main' into feat/forest-root-catch-ups-msp
ffarall Jan 24, 2025
562ffdc
fix: :rotating_light: Fix compiler errors after merge
ffarall Jan 24, 2025
35ccda7
chore: :label: Update `api-augment`
ffarall Jan 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion api-augment/dist/interfaces/lookup.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api-augment/dist/interfaces/lookup.js.map

Large diffs are not rendered by default.

13 changes: 11 additions & 2 deletions api-augment/dist/types/interfaces/augment-api-events.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2125,14 +2125,23 @@ declare module "@polkadot/api-base/types/events" {
* This is the generic version of [`MutationsAppliedForProvider`](Event::MutationsAppliedForProvider)
* when [`generic_apply_delta`](ProofsDealerInterface::generic_apply_delta) is used
* and the root is not necessarily linked to a specific Provider.
*
* Additional information for context on where the mutations were applied can be provided
* by using the `event_info` field.
**/
MutationsApplied: AugmentedEvent<
ApiType,
[mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, oldRoot: H256, newRoot: H256],
[
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>,
oldRoot: H256,
newRoot: H256,
eventInfo: Option<Bytes>
],
{
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
oldRoot: H256;
newRoot: H256;
eventInfo: Option<Bytes>;
}
>;
/**
Expand Down Expand Up @@ -2245,7 +2254,7 @@ declare module "@polkadot/api-base/types/events" {
providers: {
/**
* Event emitted when a provider has been slashed and they have reached a capacity deficit (i.e. the provider's capacity fell below their used capacity)
* signaling the end of the grace period since an automatic top up could not be performed due to insufficient free balance.
* signalling the end of the grace period since an automatic top up could not be performed due to insufficient free balance.
**/
AwaitingTopUp: AugmentedEvent<
ApiType,
Expand Down
21 changes: 21 additions & 0 deletions api-augment/dist/types/interfaces/augment-api-runtime.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,20 +48,23 @@ import type { IExtrinsic, Observable } from "@polkadot/types/types";
import type {
BackupStorageProvider,
BackupStorageProviderId,
BucketId,
ChunkId,
GetBspInfoError,
GetChallengePeriodError,
GetChallengeSeedError,
GetCheckpointChallengesError,
GetNextDeadlineTickError,
GetProofSubmissionRecordError,
GetStakeError,
GetUsersWithDebtOverThresholdError,
IsStorageRequestOpenToVolunteersError,
MainStorageProviderId,
Multiaddresses,
ProviderId,
QueryAvailableStorageCapacityError,
QueryBspConfirmChunksToProveForFileError,
QueryBucketsForMspError,
QueryEarliestChangeCapacityBlockError,
QueryFileEarliestVolunteerBlockError,
QueryMspConfirmChunksToProveForFileError,
Expand Down Expand Up @@ -631,6 +634,15 @@ declare module "@polkadot/api-base/types/calls" {
bspId: BackupStorageProviderId | string | Uint8Array
) => Observable<Result<BackupStorageProvider, GetBspInfoError>>
>;
/**
* Get the stake of a BSP.
**/
getBspStake: AugmentedCall<
ApiType,
(
bspId: BackupStorageProviderId | string | Uint8Array
) => Observable<Result<Balance, GetStakeError>>
>;
/**
* Get the slashable amount corresponding to the configured max file size.
**/
Expand Down Expand Up @@ -658,6 +670,15 @@ declare module "@polkadot/api-base/types/calls" {
providerId: ProviderId | string | Uint8Array
) => Observable<Result<StorageDataUnit, QueryAvailableStorageCapacityError>>
>;
/**
* Get the Buckets that an MSP is storing.
**/
queryBucketsForMsp: AugmentedCall<
ApiType,
(
mspId: MainStorageProviderId | string | Uint8Array
) => Observable<Result<Vec<BucketId>, QueryBucketsForMspError>>
>;
/**
* Query the earliest block number that a BSP can change its capacity.
**/
Expand Down
2 changes: 1 addition & 1 deletion api-augment/dist/types/interfaces/augment-api-tx.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3481,7 +3481,7 @@ declare module "@polkadot/api-base/types/submittable" {
* Notes:
* - This extrinsic could be called by the user itself or by a third party
* - The deposit that the user has to pay to register as a SP is held when the user requests to register as a SP
* - If this extrinsic is successful, it will be free for the caller, to incentive state debloating
* - If this extrinsic is successful, it will be free for the caller, to incentive state de-bloating
**/
confirmSignUp: AugmentedSubmittable<
(
Expand Down
6 changes: 6 additions & 0 deletions api-augment/dist/types/interfaces/augment-types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1320,6 +1320,7 @@ import type { Error } from "@polkadot/types/interfaces/xcmRuntimeApi";
import type {
BackupStorageProvider,
BackupStorageProviderId,
BucketId,
CheckpointChallenge,
ChunkId,
FileMetadata,
Expand All @@ -1330,6 +1331,7 @@ import type {
GetFileFromFileStorageResult,
GetNextDeadlineTickError,
GetProofSubmissionRecordError,
GetStakeError,
GetUsersWithDebtOverThresholdError,
IncompleteFileStatus,
IsStorageRequestOpenToVolunteersError,
Expand All @@ -1340,6 +1342,7 @@ import type {
ProviderId,
QueryAvailableStorageCapacityError,
QueryBspConfirmChunksToProveForFileError,
QueryBucketsForMspError,
QueryConfirmChunksToProveForFileError,
QueryEarliestChangeCapacityBlockError,
QueryFileEarliestVolunteerBlockError,
Expand Down Expand Up @@ -1503,6 +1506,7 @@ declare module "@polkadot/types/types/registry" {
BridgedBlockNumber: BridgedBlockNumber;
BridgedHeader: BridgedHeader;
BridgeMessageId: BridgeMessageId;
BucketId: BucketId;
BufferedSessionChange: BufferedSessionChange;
Bytes: Bytes;
Call: Call;
Expand Down Expand Up @@ -1845,6 +1849,7 @@ declare module "@polkadot/types/types/registry" {
GetFileFromFileStorageResult: GetFileFromFileStorageResult;
GetNextDeadlineTickError: GetNextDeadlineTickError;
GetProofSubmissionRecordError: GetProofSubmissionRecordError;
GetStakeError: GetStakeError;
GetUsersWithDebtOverThresholdError: GetUsersWithDebtOverThresholdError;
GiltBid: GiltBid;
GlobalValidationData: GlobalValidationData;
Expand Down Expand Up @@ -2211,6 +2216,7 @@ declare module "@polkadot/types/types/registry" {
PvfPrepTimeoutKind: PvfPrepTimeoutKind;
QueryAvailableStorageCapacityError: QueryAvailableStorageCapacityError;
QueryBspConfirmChunksToProveForFileError: QueryBspConfirmChunksToProveForFileError;
QueryBucketsForMspError: QueryBucketsForMspError;
QueryConfirmChunksToProveForFileError: QueryConfirmChunksToProveForFileError;
QueryEarliestChangeCapacityBlockError: QueryEarliestChangeCapacityBlockError;
QueryFileEarliestVolunteerBlockError: QueryFileEarliestVolunteerBlockError;
Expand Down
1 change: 1 addition & 0 deletions api-augment/dist/types/interfaces/lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1678,6 +1678,7 @@ declare const _default: {
mutations: string;
oldRoot: string;
newRoot: string;
eventInfo: string;
};
ChallengesTickerSet: {
paused: string;
Expand Down
14 changes: 14 additions & 0 deletions api-augment/dist/types/interfaces/storagehubclient/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ export interface BackupStorageProvider extends Struct {
}
/** @name BackupStorageProviderId */
export interface BackupStorageProviderId extends H256 {}
/** @name BucketId */
export interface BucketId extends H256 {}
/** @name CheckpointChallenge */
export interface CheckpointChallenge extends Struct {
readonly file_key: H256;
Expand Down Expand Up @@ -98,6 +100,12 @@ export interface GetProofSubmissionRecordError extends Enum {
readonly isInternalApiError: boolean;
readonly type: "ProviderNotRegistered" | "ProviderNeverSubmittedProof" | "InternalApiError";
}
/** @name GetStakeError */
export interface GetStakeError extends Enum {
readonly isProviderNotRegistered: boolean;
readonly isInternalError: boolean;
readonly type: "ProviderNotRegistered" | "InternalError";
}
/** @name GetUsersWithDebtOverThresholdError */
export interface GetUsersWithDebtOverThresholdError extends Enum {
readonly isProviderNotRegistered: boolean;
Expand Down Expand Up @@ -155,6 +163,12 @@ export interface QueryBspConfirmChunksToProveForFileError extends Enum {
readonly isInternalError: boolean;
readonly type: "StorageRequestNotFound" | "ConfirmChunks" | "InternalError";
}
/** @name QueryBucketsForMspError */
export interface QueryBucketsForMspError extends Enum {
readonly isProviderNotRegistered: boolean;
readonly isInternalError: boolean;
readonly type: "ProviderNotRegistered" | "InternalError";
}
/** @name QueryConfirmChunksToProveForFileError */
export interface QueryConfirmChunksToProveForFileError extends Enum {
readonly isChallengedChunkToChunkIdError: boolean;
Expand Down
1 change: 1 addition & 0 deletions api-augment/dist/types/interfaces/types-lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2168,6 +2168,7 @@ declare module "@polkadot/types/lookup" {
readonly mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
readonly oldRoot: H256;
readonly newRoot: H256;
readonly eventInfo: Option<Bytes>;
} & Struct;
readonly isChallengesTickerSet: boolean;
readonly asChallengesTickerSet: {
Expand Down
19 changes: 16 additions & 3 deletions api-augment/src/interfaces/augment-api-events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1647,11 +1647,24 @@ declare module "@polkadot/api-base/types/events" {
* This is the generic version of [`MutationsAppliedForProvider`](Event::MutationsAppliedForProvider)
* when [`generic_apply_delta`](ProofsDealerInterface::generic_apply_delta) is used
* and the root is not necessarily linked to a specific Provider.
*
* Additional information for context on where the mutations were applied can be provided
* by using the `event_info` field.
**/
MutationsApplied: AugmentedEvent<
ApiType,
[mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, oldRoot: H256, newRoot: H256],
{ mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>; oldRoot: H256; newRoot: H256 }
[
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>,
oldRoot: H256,
newRoot: H256,
eventInfo: Option<Bytes>
],
{
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
oldRoot: H256;
newRoot: H256;
eventInfo: Option<Bytes>;
}
>;
/**
* A set of mutations has been applied to the Forest of a given Provider.
Expand Down Expand Up @@ -1741,7 +1754,7 @@ declare module "@polkadot/api-base/types/events" {
providers: {
/**
* Event emitted when a provider has been slashed and they have reached a capacity deficit (i.e. the provider's capacity fell below their used capacity)
* signaling the end of the grace period since an automatic top up could not be performed due to insufficient free balance.
* signalling the end of the grace period since an automatic top up could not be performed due to insufficient free balance.
**/
AwaitingTopUp: AugmentedEvent<
ApiType,
Expand Down
21 changes: 21 additions & 0 deletions api-augment/src/interfaces/augment-api-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,20 +54,23 @@ import type { IExtrinsic, Observable } from "@polkadot/types/types";
import type {
BackupStorageProvider,
BackupStorageProviderId,
BucketId,
ChunkId,
GetBspInfoError,
GetChallengePeriodError,
GetChallengeSeedError,
GetCheckpointChallengesError,
GetNextDeadlineTickError,
GetProofSubmissionRecordError,
GetStakeError,
GetUsersWithDebtOverThresholdError,
IsStorageRequestOpenToVolunteersError,
MainStorageProviderId,
Multiaddresses,
ProviderId,
QueryAvailableStorageCapacityError,
QueryBspConfirmChunksToProveForFileError,
QueryBucketsForMspError,
QueryEarliestChangeCapacityBlockError,
QueryFileEarliestVolunteerBlockError,
QueryMspConfirmChunksToProveForFileError,
Expand Down Expand Up @@ -581,6 +584,15 @@ declare module "@polkadot/api-base/types/calls" {
bspId: BackupStorageProviderId | string | Uint8Array
) => Observable<Result<BackupStorageProvider, GetBspInfoError>>
>;
/**
* Get the stake of a BSP.
**/
getBspStake: AugmentedCall<
ApiType,
(
bspId: BackupStorageProviderId | string | Uint8Array
) => Observable<Result<Balance, GetStakeError>>
>;
/**
* Get the slashable amount corresponding to the configured max file size.
**/
Expand Down Expand Up @@ -608,6 +620,15 @@ declare module "@polkadot/api-base/types/calls" {
providerId: ProviderId | string | Uint8Array
) => Observable<Result<StorageDataUnit, QueryAvailableStorageCapacityError>>
>;
/**
* Get the Buckets that an MSP is storing.
**/
queryBucketsForMsp: AugmentedCall<
ApiType,
(
mspId: MainStorageProviderId | string | Uint8Array
) => Observable<Result<Vec<BucketId>, QueryBucketsForMspError>>
>;
/**
* Query the earliest block number that a BSP can change its capacity.
**/
Expand Down
2 changes: 1 addition & 1 deletion api-augment/src/interfaces/augment-api-tx.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2923,7 +2923,7 @@ declare module "@polkadot/api-base/types/submittable" {
* Notes:
* - This extrinsic could be called by the user itself or by a third party
* - The deposit that the user has to pay to register as a SP is held when the user requests to register as a SP
* - If this extrinsic is successful, it will be free for the caller, to incentive state debloating
* - If this extrinsic is successful, it will be free for the caller, to incentive state de-bloating
**/
confirmSignUp: AugmentedSubmittable<
(
Expand Down
6 changes: 6 additions & 0 deletions api-augment/src/interfaces/augment-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1326,6 +1326,7 @@ import type { Error } from "@polkadot/types/interfaces/xcmRuntimeApi";
import type {
BackupStorageProvider,
BackupStorageProviderId,
BucketId,
CheckpointChallenge,
ChunkId,
FileMetadata,
Expand All @@ -1336,6 +1337,7 @@ import type {
GetFileFromFileStorageResult,
GetNextDeadlineTickError,
GetProofSubmissionRecordError,
GetStakeError,
GetUsersWithDebtOverThresholdError,
IncompleteFileStatus,
IsStorageRequestOpenToVolunteersError,
Expand All @@ -1346,6 +1348,7 @@ import type {
ProviderId,
QueryAvailableStorageCapacityError,
QueryBspConfirmChunksToProveForFileError,
QueryBucketsForMspError,
QueryConfirmChunksToProveForFileError,
QueryEarliestChangeCapacityBlockError,
QueryFileEarliestVolunteerBlockError,
Expand Down Expand Up @@ -1510,6 +1513,7 @@ declare module "@polkadot/types/types/registry" {
BridgedBlockNumber: BridgedBlockNumber;
BridgedHeader: BridgedHeader;
BridgeMessageId: BridgeMessageId;
BucketId: BucketId;
BufferedSessionChange: BufferedSessionChange;
Bytes: Bytes;
Call: Call;
Expand Down Expand Up @@ -1852,6 +1856,7 @@ declare module "@polkadot/types/types/registry" {
GetFileFromFileStorageResult: GetFileFromFileStorageResult;
GetNextDeadlineTickError: GetNextDeadlineTickError;
GetProofSubmissionRecordError: GetProofSubmissionRecordError;
GetStakeError: GetStakeError;
GetUsersWithDebtOverThresholdError: GetUsersWithDebtOverThresholdError;
GiltBid: GiltBid;
GlobalValidationData: GlobalValidationData;
Expand Down Expand Up @@ -2218,6 +2223,7 @@ declare module "@polkadot/types/types/registry" {
PvfPrepTimeoutKind: PvfPrepTimeoutKind;
QueryAvailableStorageCapacityError: QueryAvailableStorageCapacityError;
QueryBspConfirmChunksToProveForFileError: QueryBspConfirmChunksToProveForFileError;
QueryBucketsForMspError: QueryBucketsForMspError;
QueryConfirmChunksToProveForFileError: QueryConfirmChunksToProveForFileError;
QueryEarliestChangeCapacityBlockError: QueryEarliestChangeCapacityBlockError;
QueryFileEarliestVolunteerBlockError: QueryFileEarliestVolunteerBlockError;
Expand Down
3 changes: 2 additions & 1 deletion api-augment/src/interfaces/lookup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1699,7 +1699,8 @@ export default {
MutationsApplied: {
mutations: "Vec<(H256,ShpTraitsTrieMutation)>",
oldRoot: "H256",
newRoot: "H256"
newRoot: "H256",
eventInfo: "Option<Bytes>"
},
ChallengesTickerSet: {
paused: "bool"
Expand Down
Loading
Loading