Skip to content

Commit

Permalink
refactor: Update error handling in cat-gateway (#425)
Browse files Browse the repository at this point in the history
* docs: Add CatalystDataGatewayRepository docs (#388)

* test: Fix tests after cat-gateway update.

* docs: Add `CatalystDataGatewayRepository` usage examples.

* docs: Move docs from README to code comment.

* test: Update `CatalystDataGatewayRepository` tests.

* test: Use Fake instead of Mock.

* chore: Fix Markdown errors.

* chore: Explicit use of `HttpStatus` codes.

* adds one more worker and compression for catgateway logs (#400)

* feat: Collect flutter code coverage (#404)

* test: Fix tests after cat-gateway update.

* docs: Add `CatalystDataGatewayRepository` usage examples.

* docs: Move docs from README to code comment.

* test: Update `CatalystDataGatewayRepository` tests.

* test: Use Fake instead of Mock.

* chore: Fix Markdown errors.

* chore: Explicit use of `HttpStatus` codes.

* chore(deps-dev): bump vite in /utilities/wallet-tester (#397)

Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.1.6 to 5.1.7.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Apisit Ritreungroj <[email protected]>

* feat: Get slot number by provided date-time endpoint (#387)

* fix CardanoStakeAddress error handling

* refactor, add sync_state_get endpoint

* refactor types

* refactor

* add block_hash validation

* wip

* wip

* wip

* wip

* add check_network fn

* fix

* fix schematisis test

* try

* wip

* try

* try

* try

* try

* wip

* try

* try

* fix

* update Network

* add test_utxo test

* try

* fix

* try

* fix

* wip

* fix

* fix docket-compose.yml file

* try

* try

* fix

* try

* try

* try

* try

* wip

* fix

* wip

* try

* try

* wip

* try

* try

* revert

* wip

* wip

* wip

* fix

* fix

* fix

* remove mithril_snapshot loader

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* add stake addr bech32 encode utils function

* wip

* wip

* update indexing of the utxo data

* fix spelling

* wip

* wip

* finish utxo test

* fix deny

* fix check

* fix

* fix

* update earthly builder versions

* wip

* ignore test_utxo.py in CI

* dont ignore tests

* add date_time_to_slot_number_get endpoint

* add sql queries

* fix

* update slot info, fix follower indexing block time issue

* add previous slot info field

* fix

* refactor

* fix sync_state_get

* wip

* fix check

* try

* fix

* finish slot_info test, fix queries

* fix

* cleanup

* wip

* wip

* wip

* feat: RBAC Documentation Drafting (#332)

* chore: wip

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* Update 0005-flutter-app.md

* docs(docs): Use latest docs builders and fix concepts page

* docs(cips): Start drafting the CIPS for milestone 2

* docs(cips): More text for RBAC metadata draft

* docs(cips): WIP updates to draft cip for role registration

* docs(cips): define draft specification for a ULID cbor tag

* docs(cips): Further WIP edits to RBAC

* docs(cips): fix ulid spec binary encoding reference

* docs(cips): Add a tag to the epoch time.

* docs(cips): Add CBOR tag cip for ED25519-BIP32 Keys, Derivation paths and Signatures

* docs(cips): Properly define the field tags to use where known, and clean up Stake Address specification.

* docs(cips): Fix nonce so its reliable without needing blockchain data

* docs(cips): updates

* docs(docs): Add CDDL definition for POC x509 envelope metadata

* fix(vscode): update vscode extension recommendations

* docs(cips): rbac x509 envelope fix

* docs(cips): wip updates to high level docs

* docs(cips): Add overview of cardano transaction processign and data

* docs(cips): update cardano block to be complete for clarity

* docs(cips): fix layout engine

* docs(cips): wip cddl for envelope metadata

* docs(cips): Add cddl specs and diagrams for x509 rbac registration work

* docs(cips): Add full transaction/metadata relationship diagram

* refactor(cips): reorganize documentation ready for drafting descriptive prose about the formats and uses

* docs(cips): add cip draft for catalyst roles using the x509-rbac standard

* docs(cips): Add c509 cddl with restrictions and enhancements for plutus usage

* docs(cips): Metadata envelope specification draft complete

* Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/x509-role-registration-metadata/x509-roles.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/x509-role-registration-metadata/x509-roles.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl

Co-authored-by: bkioshn <[email protected]>

* Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl

Co-authored-by: bkioshn <[email protected]>

* docs(cips): Fix time and algorithm comments

* build(frontend): Relax flutter/dart version requirements to last minor release

* docs(cips): wip

* fix(cips): rename draft x509 envelope CIP so its easier to identify

* docs(cips): WIP updates to x509 roles

* fix(cips): rename RBAC definition CIP draft so its easier to identify

* docs(cips): x509 certificate registration format fully defined

* docs(cips): Document the restricted plutus subset.

* docs(cips): Add document detailing how CIP-30 is used to sign the transaction

* fix(cips): remove trailing spaces

* fix(cips): Fix line lengths

* fix(cips): Correct spelling

* fix(cips): spelling

* fix(frontend): revert changes to flutter/dart versions

* fix(frontend): more flutter/dart version corrections

* fix(frontend): Revert flutter files to same as main branch

* fix(frontend): revert more flutter .yml files to those in main

* fix(cips): Fix links between files

* docs(cips): Add catalyst specific role registration documentation

* docs(spelling): fix spelling

---------

Co-authored-by: minikin <[email protected]>
Co-authored-by: bkioshn <[email protected]>

* ci: configure static analysis & code formatting check

* style: formatting

* fix: revert browser installation scripts

* style: format code

* ci: optimize directions include in repo-catalyst-voices-all artifact to include only needed ones

* refactor: remove empty tests

* ci: add melos script to generate test reports

* ci: melos script to generate test report

* style: revert previously generated files formatting

* style: format files

* ci: update melos to exclude generated code form code coverage

* ci: cleanup build script

* ci: generate multiple junit test reports and save them at /test_reports

* ci: depend on melos analyze instead of custom command

* docs: improve melos docs

* ci: remove unused melos scripts

* ci: format files in test & integration_test directories

* ci: break code to make sure CI will report failure for demonstration purposes

* style: fix lint issues

* ci: change WORKDIR after creating the user to make sure it will be owned by that user

* ci: restore root user

* Revert "Merge branch 'main' into feat/collect-flutter-code-coverage"

This reverts commit d0f66b2, reversing
changes made to 39ce401.

* style: format code

* ci: revert test-unit target name

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Lucio Baglione <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Apisit Ritreungroj <[email protected]>
Co-authored-by: Alex Pozhylenkov <[email protected]>
Co-authored-by: Steven Johnson <[email protected]>
Co-authored-by: minikin <[email protected]>
Co-authored-by: bkioshn <[email protected]>

* refactor: update schema_validation check (#414)

* update schema_validation check

* wip

* fix

* update MismatchedSchema and NoDatabaseUrl error handling

* update get_follower_config error handling

* update follower queries error handling

* fix

* update legacy queries

* update utxo queries error handling

* fix

* wip

* remove service error type

* update registration error handling

* wip

* fix

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Lucio Baglione <[email protected]>
Co-authored-by: Stefano Cunego <[email protected]>
Co-authored-by: Dominik Toton <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Apisit Ritreungroj <[email protected]>
Co-authored-by: Steven Johnson <[email protected]>
Co-authored-by: minikin <[email protected]>
Co-authored-by: bkioshn <[email protected]>
  • Loading branch information
9 people authored Apr 16, 2024
1 parent 3db8f17 commit fb19645
Show file tree
Hide file tree
Showing 28 changed files with 225 additions and 295 deletions.
13 changes: 1 addition & 12 deletions catalyst-gateway/bin/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,6 @@ use crate::{
state::State,
};

#[derive(thiserror::Error, Debug)]
/// All service errors
pub(crate) enum Error {
#[error(transparent)]
/// Service oriented errors
Service(#[from] service::Error),
#[error(transparent)]
/// DB oriented errors
EventDb(#[from] crate::event_db::error::Error),
}

#[derive(Parser)]
#[clap(rename_all = "kebab-case")]
/// Simple service CLI options
Expand Down Expand Up @@ -77,7 +66,7 @@ impl Cli {

match event_db.get_follower_config().await {
Ok(config) => break config,
Err(err) => error!("no config {:?}", err),
Err(err) => error!("No follower config found, error: {err}"),
}
};

Expand Down
19 changes: 9 additions & 10 deletions catalyst-gateway/bin/src/event_db/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ use std::str::FromStr;
use cardano_chain_follower::Network;
use serde::{Deserialize, Serialize};

use crate::event_db::{Error, EventDB};
use super::error::NotFoundError;
use crate::event_db::EventDB;

/// Representation of the `config` table id fields `id`, `id2`, `id3`
enum ConfigId {
Expand Down Expand Up @@ -51,7 +52,7 @@ pub(crate) struct MithrilSnapshotConfig {

impl EventDB {
/// Config query
pub(crate) async fn get_follower_config(&self) -> Result<Vec<FollowerConfig>, Error> {
pub(crate) async fn get_follower_config(&self) -> anyhow::Result<Vec<FollowerConfig>> {
let conn = self.pool.get().await?;

let id = "cardano";
Expand All @@ -66,31 +67,29 @@ impl EventDB {

let mut follower_configs = Vec::new();
for row in rows {
let network = Network::from_str(row.try_get::<_, &str>(ConfigId::Id3.as_str())?)
.map_err(|e| Error::Unknown(e.to_string()))?;
let network = Network::from_str(row.try_get::<_, &str>(ConfigId::Id3.as_str())?)?;
let config: serde_json::Value = row.try_get("value")?;

let relay = config
.get("relay")
.ok_or(Error::JsonParseIssue(
.ok_or(anyhow::anyhow!(
"Cardano follower config does not have `relay` property".to_string(),
))?
.as_str()
.ok_or(Error::JsonParseIssue(
.ok_or(anyhow::anyhow!(
"Cardano follower config `relay` not a string type".to_string(),
))?
.to_string();

let mithril_snapshot = serde_json::from_value(
config
.get("mithril_snapshot")
.ok_or(Error::JsonParseIssue(
.ok_or(anyhow::anyhow!(
"Cardano follower config does not have `mithril_snapshot` property"
.to_string(),
))?
.clone(),
)
.map_err(|e| Error::JsonParseIssue(e.to_string()))?;
)?;

follower_configs.push(FollowerConfig {
network,
Expand All @@ -100,7 +99,7 @@ impl EventDB {
}

if follower_configs.is_empty() {
Err(Error::NoConfig)
Err(NotFoundError.into())
} else {
Ok(follower_configs)
}
Expand Down
56 changes: 3 additions & 53 deletions catalyst-gateway/bin/src/event_db/error.rs
Original file line number Diff line number Diff line change
@@ -1,56 +1,6 @@
//! Database Errors
use bb8::RunError;

/// Event database errors
/// DB not found error
#[derive(thiserror::Error, Debug, PartialEq, Eq)]
pub(crate) enum Error {
/// Schema in database does not match schema supported by the Crate.
#[error(" Schema in database does not match schema supported by the Crate. The current schema version: {was}, the schema version we expected: {expected}")]
MismatchedSchema {
/// The current DB schema version.
was: i32,
/// The expected DB schema version.
expected: i32,
},
/// No DB URL was provided
#[error("DB URL is undefined")]
NoDatabaseUrl,
/// Cannot find this item
#[error("Cannot find this item")]
NotFound,
/// DB connection timeout
#[error("Connection to DB timed out")]
TimedOut,
/// Unknown error
#[error("error: {0}")]
Unknown(String),
/// No config
#[error("No config")]
NoConfig,
/// JSON Parsing error
#[error("Unable to parse database data: {0}")]
JsonParseIssue(String),
/// Unable to extract policy assets
#[error("Unable parse assets: {0}")]
AssetParsingIssue(String),
/// Unable to extract hashed witnesses
#[allow(dead_code)]
#[error("Unable to extract hashed witnesses: {0}")]
HashedWitnessExtraction(String),
}

impl From<RunError<tokio_postgres::Error>> for Error {
fn from(val: RunError<tokio_postgres::Error>) -> Self {
match val {
RunError::TimedOut => Self::TimedOut,
RunError::User(_) => Self::Unknown(val.to_string()),
}
}
}

impl From<tokio_postgres::Error> for Error {
fn from(val: tokio_postgres::Error) -> Self {
Self::Unknown(val.to_string())
}
}
#[error("Cannot find this item")]
pub(crate) struct NotFoundError;
27 changes: 11 additions & 16 deletions catalyst-gateway/bin/src/event_db/follower/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
use cardano_chain_follower::Network;
use handlebars::Handlebars;

use crate::event_db::{Error, EventDB};
use super::error::NotFoundError;
use crate::event_db::EventDB;

/// Block time
pub type DateTime = chrono::DateTime<chrono::offset::Utc>;
Expand Down Expand Up @@ -55,7 +56,7 @@ struct SlotInfoQueryTmplFields {

impl SlotInfoQueryType {
/// Get SQL query
fn get_sql_query(&self) -> Result<String, Error> {
fn get_sql_query(&self) -> anyhow::Result<String> {
let tmpl_fields = match self {
SlotInfoQueryType::Previous => {
SlotInfoQueryTmplFields {
Expand All @@ -81,8 +82,7 @@ impl SlotInfoQueryType {
// disable default `html_escape` function
// which transforms `<`, `>` symbols to `&lt`, `&gt`
reg.register_escape_fn(|s| s.into());
reg.render_template(SLOT_INFO_SQL_HBS, &tmpl_fields)
.map_err(|e| Error::Unknown(e.to_string()))
Ok(reg.render_template(SLOT_INFO_SQL_HBS, &tmpl_fields)?)
}
}

Expand All @@ -91,7 +91,7 @@ impl EventDB {
pub(crate) async fn index_follower_data(
&self, slot_no: SlotNumber, network: Network, epoch_no: EpochNumber, block_time: DateTime,
block_hash: BlockHash,
) -> Result<(), Error> {
) -> anyhow::Result<()> {
let conn = self.pool.get().await?;

let _rows = conn
Expand All @@ -110,7 +110,7 @@ impl EventDB {
/// Get slot info for the provided date-time and network and query type
pub(crate) async fn get_slot_info(
&self, date_time: DateTime, network: Network, query_type: SlotInfoQueryType,
) -> Result<(SlotNumber, BlockHash, DateTime), Error> {
) -> anyhow::Result<(SlotNumber, BlockHash, DateTime)> {
let conn = self.pool.get().await?;

let rows = conn
Expand All @@ -120,9 +120,7 @@ impl EventDB {
])
.await?;

let Some(row) = rows.first() else {
return Err(Error::NotFound);
};
let row = rows.first().ok_or(NotFoundError)?;

let slot_number: SlotNumber = row.try_get(SLOT_NO_COLUMN)?;
let block_hash = row.try_get(BLOCK_HASH_COLUMN)?;
Expand All @@ -134,16 +132,14 @@ impl EventDB {
/// Start follower from where previous follower left off.
pub(crate) async fn last_updated_metadata(
&self, network: Network,
) -> Result<(SlotNumber, BlockHash, DateTime), Error> {
) -> anyhow::Result<(SlotNumber, BlockHash, DateTime)> {
let conn = self.pool.get().await?;

let rows = conn
.query(SELECT_UPDATE_STATE_SQL, &[&network.to_string()])
.await?;

let Some(row) = rows.first() else {
return Err(Error::NotFound);
};
let row = rows.first().ok_or(NotFoundError)?;

let slot_no = row.try_get(SLOT_NO_COLUMN)?;
let block_hash = row.try_get(BLOCK_HASH_COLUMN)?;
Expand All @@ -157,7 +153,7 @@ impl EventDB {
pub(crate) async fn refresh_last_updated(
&self, last_updated: DateTime, slot_no: SlotNumber, block_hash: BlockHash,
network: Network, machine_id: &MachineId,
) -> Result<(), Error> {
) -> anyhow::Result<()> {
let conn = self.pool.get().await?;

// Rollback or update
Expand All @@ -169,8 +165,7 @@ impl EventDB {
// All future additions are just updates on ended, slot_no and block_hash
let _rows = conn
.query(INSERT_UPDATE_STATE_SQL, &[
&i64::try_from(network_id)
.map_err(|_| Error::Unknown("Network id out of range".to_string()))?,
&i64::try_from(network_id)?,
&last_updated,
&last_updated,
&machine_id,
Expand Down
10 changes: 5 additions & 5 deletions catalyst-gateway/bin/src/event_db/legacy/queries/event/ballot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
use std::collections::HashMap;

use crate::event_db::{
error::Error,
error::NotFoundError,
legacy::types::{
ballot::{
Ballot, BallotType, GroupVotePlans, ObjectiveBallots, ObjectiveChoices, ProposalBallot,
Expand Down Expand Up @@ -52,7 +52,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_ballot(
&self, event: EventId, objective: ObjectiveId, proposal: ProposalId,
) -> Result<Ballot, Error> {
) -> anyhow::Result<Ballot> {
let conn = self.pool.get().await?;

let rows = conn
Expand All @@ -62,7 +62,7 @@ impl EventDB {
&proposal.0,
])
.await?;
let row = rows.first().ok_or(Error::NotFound)?;
let row = rows.first().ok_or(NotFoundError)?;
let choices = row.try_get("objective")?;

let rows = conn
Expand Down Expand Up @@ -95,7 +95,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_objective_ballots(
&self, event: EventId, objective: ObjectiveId,
) -> Result<Vec<ProposalBallot>, Error> {
) -> anyhow::Result<Vec<ProposalBallot>> {
let conn = self.pool.get().await?;

let rows = conn
Expand Down Expand Up @@ -145,7 +145,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_event_ballots(
&self, event: EventId,
) -> Result<Vec<ObjectiveBallots>, Error> {
) -> anyhow::Result<Vec<ObjectiveBallots>> {
let conn = self.pool.get().await?;

let rows = conn
Expand Down
8 changes: 4 additions & 4 deletions catalyst-gateway/bin/src/event_db/legacy/queries/event/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
use chrono::{NaiveDateTime, Utc};

use crate::event_db::{
error::Error,
error::NotFoundError,
legacy::types::event::{
Event, EventDetails, EventGoal, EventId, EventRegistration, EventSchedule, EventSummary,
VotingPowerAlgorithm, VotingPowerSettings,
Expand Down Expand Up @@ -44,7 +44,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_events(
&self, limit: Option<i64>, offset: Option<i64>,
) -> Result<Vec<EventSummary>, Error> {
) -> anyhow::Result<Vec<EventSummary>> {
let conn = self.pool.get().await?;

let rows = conn
Expand Down Expand Up @@ -76,11 +76,11 @@ impl EventDB {

/// Get event query
#[allow(dead_code)]
pub(crate) async fn get_event(&self, event: EventId) -> Result<Event, Error> {
pub(crate) async fn get_event(&self, event: EventId) -> anyhow::Result<Event> {
let conn = self.pool.get().await?;

let rows = conn.query(Self::EVENT_QUERY, &[&event.0]).await?;
let row = rows.first().ok_or(Error::NotFound)?;
let row = rows.first().ok_or(NotFoundError)?;

let ends = row
.try_get::<&'static str, Option<NaiveDateTime>>("end_time")?
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
//! Objective Queries
use crate::event_db::{
error::Error,
legacy::types::{
event::EventId,
objective::{
Expand Down Expand Up @@ -33,7 +32,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_objectives(
&self, event: EventId, limit: Option<i64>, offset: Option<i64>,
) -> Result<Vec<Objective>, Error> {
) -> anyhow::Result<Vec<Objective>> {
let conn = self.pool.get().await?;

let rows = conn
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Proposal Queries
use crate::event_db::{
error::Error,
error::NotFoundError,
legacy::types::{
event::EventId,
objective::ObjectiveId,
Expand Down Expand Up @@ -32,15 +32,15 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_proposal(
&self, event: EventId, objective: ObjectiveId, proposal: ProposalId,
) -> Result<Proposal, Error> {
) -> anyhow::Result<Proposal> {
let conn: bb8::PooledConnection<
bb8_postgres::PostgresConnectionManager<tokio_postgres::NoTls>,
> = self.pool.get().await?;

let rows = conn
.query(Self::PROPOSAL_QUERY, &[&event.0, &objective.0, &proposal.0])
.await?;
let row = rows.first().ok_or(Error::NotFound)?;
let row = rows.first().ok_or(NotFoundError)?;

let proposer = vec![ProposerDetails {
name: row.try_get("proposer_name")?,
Expand Down Expand Up @@ -71,7 +71,7 @@ impl EventDB {
#[allow(dead_code)]
pub(crate) async fn get_proposals(
&self, event: EventId, objective: ObjectiveId, limit: Option<i64>, offset: Option<i64>,
) -> Result<Vec<ProposalSummary>, Error> {
) -> anyhow::Result<Vec<ProposalSummary>> {
let conn = self.pool.get().await?;

let rows = conn
Expand Down
Loading

0 comments on commit fb19645

Please sign in to comment.