Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing epochs data #142

Merged
merged 8 commits into from
Dec 14, 2023
Merged

Indexing epochs data #142

merged 8 commits into from
Dec 14, 2023

Conversation

kobayurii
Copy link
Member

@kobayurii kobayurii commented Dec 7, 2023

This pull request introduces several significant enhancements and new functionalities to the codebase, focusing on improving the indexer's capabilities and implementing new endpoints for enhanced querying:

1. Epoch-Indexer Implementation for Validators and Protocol Configurations:

  • Introduced the epoch-indexer functionality to efficiently index validators and protocol configurations according to epochs. This implementation ensures a more granular and organized indexing approach, facilitating precise data retrieval based on different epochs.

2. State-Indexer Enhancement for Epochs:

  • Improved the state-indexer to include epochs indexing. This enhancement enriches the indexer's functionality to indexing validators and protocol configs for epochs.

3. Query Function Call Improvement and Bug Fix:

  • The use of indexed epoch data has significantly improved the call_function_query mechanism and eliminated a bug that was causing inconsistent and inconsistent results. This fix ensures reliability and consistency of query results.

4. Validators Endpoint Implementation:

  • Implemented the validators endpoint, offering users the capability to retrieve and query list validators for epochs.

5. Protocol_Config Endpoint Implementation:

  • Added the protocol_config endpoint, allowing users to access and retrieve protocol configuration details for epochs. This endpoint serves as a valuable resource for understanding protocol settings and configurations.

6. Gas_Price Endpoint Implementation:

  • Introduced the gas_price endpoint, enabling users to retrieve gas-price for blocks.

This pull request signifies a significant stride in augmenting the system's capabilities, refining data retrieval mechanisms, and providing users with enhanced access to critical system information.

@kobayurii kobayurii requested a review from khorolets December 7, 2023 15:35
@kobayurii kobayurii self-assigned this Dec 7, 2023
@kobayurii kobayurii force-pushed the epoch_indexer branch 10 times, most recently from 5a05c60 to f9933af Compare December 12, 2023 07:54
@kobayurii kobayurii changed the title add epoch indexer Indexing epochs data Dec 12, 2023
@kobayurii kobayurii marked this pull request as ready for review December 12, 2023 08:29
@kobayurii kobayurii force-pushed the epoch_indexer branch 2 times, most recently from 2a1b764 to 7d83419 Compare December 12, 2023 09:44
Copy link
Member

@khorolets khorolets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start! I have left change requests, questions and suggestions I would like you to address.

database/Cargo.toml Outdated Show resolved Hide resolved
epoch_id: near_indexer_primitives::CryptoHash,
) -> anyhow::Result<readnode_primitives::EpochValidatorsInfo> {
let epoch = crate::models::Validators::get_validators(
Self::get_connection(&self.pg_pool).await?,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like establishing a connection every time instead of passing an established one. Could you clarify why don't you pass a pg_pool to the function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg_pool - Generic object and connection pool. This struct can be cloned and transferred across thread boundaries and uses reference counting for its internal state.
get_connection - Retrieves a connection from pg_pool or waits for one to become available.

epoch-indexer/README.md Outdated Show resolved Hide resolved
epoch-indexer/README.md Outdated Show resolved Hide resolved
rpc-server/src/modules/network/methods.rs Outdated Show resolved Hide resolved
rpc-server/src/modules/network/methods.rs Outdated Show resolved Hide resolved
state-indexer/src/configs.rs Outdated Show resolved Hide resolved
state-indexer/src/main.rs Outdated Show resolved Hide resolved
DATABASE_URL=127.0.0.1:9042
DATABASE_USER=cassandra
DATABASE_PASSWORD=cassandra
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does epoch-indexer use S3?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! It takes blocks from s3.

@kobayurii kobayurii requested a review from khorolets December 12, 2023 13:20
Copy link
Member

@khorolets khorolets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think this is ready to be merged. Great work!

@kobayurii kobayurii merged commit 05f88e0 into main Dec 14, 2023
3 checks passed
@kobayurii kobayurii deleted the epoch_indexer branch December 20, 2023 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants