Introduce intrinsic fees for transaction validation beyond script & b…

…yte costs (#529) closes: #461 Increases the granularity of how the base fee for a transaction is computed. These base fee components are referred to as intrinsic costs, which are intended to cover the overhead of operations such as: - read, insert, and removal of transaction outputs - witness recovery with secp256k1 - vm initialization for predicates & scripts - contract bytecode bmt (binary merkle tree) root calculation - storage slot smt (sparse merkle tree) insertions --------- Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Brandon Vrooman <[email protected]> Co-authored-by: Brandon Vrooman <[email protected]> Co-authored-by: Hannes Karppila <[email protected]>
FuelLabs · Dec 12, 2023 · d7e18fc · d7e18fc
1 parent 0904457
commit d7e18fc
Show file tree

Hide file tree

Showing 3 changed files with 178 additions and 52 deletions.
diff --git a/src/protocol/tx-validity.md b/src/protocol/tx-validity.md
@@ -33,9 +33,9 @@ Read-only access list:
 Write-destroy access list:
 
 - For each [input `InputType.Coin`](../tx-format/input.md#inputcoin)
-  - The [UTXO ID](../identifiers/utxo-id.md) `(txID, outputIndex)`
+  - The [UTXO ID](../identifiers/utxo-id.md) `(txId, outputIndex)`
 - For each [input `InputType.Contract`](../tx-format/input.md#inputcontract)
-  - The [UTXO ID](../identifiers/utxo-id.md) `(txID, outputIndex)`
+  - The [UTXO ID](../identifiers/utxo-id.md) `(txId, outputIndex)`
 - For each [input `InputType.Message`](../tx-format/input.md#inputmessage)
   - The [message ID](../identifiers/utxo-id.md#message-id) `messageID`
 
@@ -46,7 +46,7 @@ Write-create access list:
 - For each output
   - The [created UTXO ID](../identifiers/utxo-id.md)
 
-Note that block proposers use the contract ID `contractID` for inputs and outputs of type [`InputType.Contract`](../tx-format/input.md#inputcontract) and [`OutputType.Contract`](../tx-format/output.md#outputcontract) rather than the pair of `txID` and `outputIndex`.
+Note that block proposers use the contract ID `contractID` for inputs and outputs of type [`InputType.Contract`](../tx-format/input.md#inputcontract) and [`OutputType.Contract`](../tx-format/output.md#outputcontract) rather than the pair of `txId` and `outputIndex`.
 
 ## VM Precondition Validity Rules
 
@@ -71,94 +71,184 @@ for input in tx.inputs:
         if not input.nonce in messages:
                 return False
     else:
-        if not (input.txID, input.outputIndex) in state:
+        if not (input.txId, input.outputIndex) in state:
             return False
 return True
 ```
 
-If this check passes, the UTXO ID `(txID, outputIndex)` fields of each contract input is set to the UTXO ID of the respective contract. The `txPointer` of each input is also set to the TX pointer of the UTXO with ID `utxoID`.
+If this check passes, the UTXO ID `(txId, outputIndex)` fields of each contract input is set to the UTXO ID of the respective contract. The `txPointer` of each input is also set to the TX pointer of the UTXO with ID `utxoID`.
 
 ### Sufficient Balance
 
-For each asset ID `asset_id` in the input and output set:
+For each asset ID `assetId` in the input and output set:
 
 ```py
-def sum_data_messages(tx, asset_id) -> int:
+def gas_to_fee(gas, gasPrice) -> int:
+    """
+    Converts gas units into a fee amount
+    """
+    return ceil(gas * gasPrice / GAS_PRICE_FACTOR)
+
+
+def sum_data_messages(tx, assetId) -> int:
     """
     Returns the total balance available from messages containing data
     """
     total: int = 0
-    if asset_id == 0:
+    if assetId == 0:
         for input in tx.inputs:
             if input.type == InputType.Message and input.dataLength > 0:
                 total += input.amount
     return total
 
-def sum_inputs(tx, asset_id) -> int:
+
+def sum_inputs(tx, assetId) -> int:
     total: int = 0
     for input in tx.inputs:
-        if input.type == InputType.Coin and input.asset_id == asset_id:
+        if input.type == InputType.Coin and input.assetId == assetId:
             total += input.amount
-        elif input.type == InputType.Message and asset_id == 0 and input.dataLength == 0:
+        elif input.type == InputType.Message and assetId == 0 and input.dataLength == 0:
             total += input.amount
     return total
 
-def sum_predicate_gas_used(tx) -> int:
-    total: int = 0
-    for input in tx.inputs:
-        if input.type == InputType.Coin:
-            total += input.predicateGasUsed
-        elif input.type == InputType.Message:
-            total += input.predicateGasUsed
-    return total
 
-"""
-Returns any minted amounts by the transaction
-"""
-def minted(tx, asset_id) -> int:
-    if tx.type != TransactionType.Mint or asset_id != tx.mint_asset_id:
+def transaction_size_gas_fees(tx) -> int:
+    """
+    Computes the intrinsic gas cost of a transaction based on size in bytes
+    """
+    return size(tx) * GAS_PER_BYTE
+
+
+def minted(tx, assetId) -> int:
+    """
+    Returns any minted amounts by the transaction
+    """
+    if tx.type != TransactionType.Mint or assetId != tx.mintAssetId:
         return 0
     return tx.mint_amount
 
-def sum_outputs(tx, asset_id) -> int:
+
+def sum_outputs(tx, assetId) -> int:
     total: int = 0
     for output in tx.outputs:
-        if output.type == OutputType.Coin and output.asset_id == asset_id:
+        if output.type == OutputType.Coin and output.assetId == assetId:
             total += output.amount
     return total
 
-def available_balance(tx, asset_id) -> int:
+
+def input_gas_fees(tx) -> int:
+    """
+    Computes the intrinsic gas cost of verifying input utxos
+    """
+    total: int = 0
+    witnessIndices = set()
+    for input in tx.inputs:
+        if input.type == InputType.Coin or input.type == InputType.Message:
+            # add fees allocated for predicate execution
+            if input.predicateLength == 0:
+                # notate witness index if input is signed
+                witnessIndices.add(input.witnessIndex)
+            else:
+                # add intrinsic gas cost of predicate merkleization based on number of predicate bytes
+                total += contract_code_root_gas_fee(input.predicateLength)
+                total += input.predicateGasUsed
+                # add intrinsic cost of vm initialization
+                total += vm_initialization_gas_fee()
+    # add intrinsic cost of verifying witness signatures
+    total += len(witnessIndices) * eck1_recover_gas_fee()
+    return total
+
+
+def metadata_gas_fees(tx) -> int:
+    """
+    Computes the intrinsic gas cost of processing transaction outputs
+    """
+    total: int = 0
+    if tx.type == TransactionType.Create:
+        for output in tx.outputs:
+            if output.type == OutputType.OutputContractCreated:
+                # add intrinsic cost of calculating the code root based on the size of the contract bytecode
+                total += contract_code_root_gas_fee(tx.witnesses[tx.bytecodeWitnessIndex].dataLength)
+                # add intrinsic cost of calculating the state root based on the number of sotrage slots
+                total += contract_state_root_gas_fee(tx.storageSlotCount)
+                # add intrinsic cost of calculating the contract id 
+                # size = 4 byte seed + 32 byte salt + 32 byte code root + 32 byte state root
+                total += sha256_gas_fee(100)
+                # add intrinsic cost of calculating the transaction id
+                total += sha256_gas_fee(size(tx))
+    elif tx.type == TransactionType.Script:
+        # add intrinsic cost of calculating the transaction id
+        total += sha256_gas_fee(size(tx))
+    return total
+
+
+def intrinsic_gas_fees(tx) -> int:
+    """
+    Computes intrinsic costs for a transaction
+    """
+    fees: int = 0
+    # add the cost of initializing a vm for the script
+    if tx.type == TransactionType.Create or tx.type == TransactionType.Script:
+        fees += vm_initialization_gas_fee()
+        fees += metadata_gas_fees(tx)
+        fees += intrinsic_input_gas_fees(tx)
+    return fees
+
+
+def min_gas(tx) -> int:
+    """
+    Comutes the minimum amount of gas required for a transaction to begin processing.
+    """
+    gas = transaction_size_gas_fees(tx) + intrinsic_gas_fees(tx)
+    return gas
+
+
+def max_gas(tx) -> int:
+    """
+    Computes the amount of gas required to process a transaction.
+    """
+    gas = min_gas(tx)
+    gas = gas + (tx.witnessBytesLimit - tx.witnessBytes) * GAS_PER_BYTE
+    if tx.type == TransactionType.Script:
+       gas = gas + tx.gasLimit
+    return gas
+
+
+def reserved_feeBalance(tx, assetId) -> int:
+    """
+    Computes the maximum potential amount of fees that may need to be charged to process a transaction.
+    """
+    maxGas = max_gas(tx)
+    feeBalance = gas_to_fee(maxGas, tx.gasPrice)
+    # Only base asset can be used to pay for gas
+    if assetId == 0:
+        return feeBalance
+    else:
+        return 0
+
+
+def available_balance(tx, assetId) -> int:
     """
     Make the data message balance available to the script
     """
-    availableBalance = sum_inputs(tx, asset_id) + sum_data_messages(tx, asset_id) + minted(tx, asset_id)
+    availableBalance = sum_inputs(tx, assetId) + sum_data_messages(tx, assetId) + minted(tx, assetId)
     return availableBalance
 
-def unavailable_balance(tx, asset_id) -> int:
-    sentBalance = sum_outputs(tx, asset_id)
+
+def unavailable_balance(tx, assetId) -> int:
+    sentBalance = sum_outputs(tx, assetId)
     # Total fee balance
-    feeBalance = fee_balance(tx, asset_id)
+    feeBalance = reserved_fee_balance(tx, assetId)
     # Only base asset can be used to pay for gas
-    if asset_id == 0:
+    if assetId == 0:
         return sentBalance + feeBalance
     return sentBalance
 
-def fee_balance(tx, asset_id) -> int:
-    gas = tx.scriptGasLimit + sum_predicate_gas_used(tx)
-    gasBalance = gasPrice * gas / GAS_PRICE_FACTOR
-    bytesBalance = size(tx) * GAS_PER_BYTE * gasPrice / GAS_PRICE_FACTOR
-    # Total fee balance
-    feeBalance = ceiling(gasBalance + bytesBalance)
-    # Only base asset can be used to pay for gas
-    if asset_id == 0:
-        return feeBalance
-    else:
-        return 0
 
 # The sum_data_messages total is not included in the unavailable_balance since it is spendable as long as there 
 # is enough base asset amount to cover gas costs without using data messages. Messages containing data can't
 # cover gas costs since they are retryable.
-return available_balance(tx, asset_id) >= (unavailable_balance(tx, asset_id) + sum_data_messages(tx, asset_id))
+return available_balance(tx, assetId) >= (unavailable_balance(tx, assetId) + sum_data_messages(tx, assetId))
 ```
 
 ### Valid Signatures
@@ -168,7 +258,7 @@ def address_from(pubkey: bytes) -> bytes:
     return sha256(pubkey)[0:32]
 
 for input in tx.inputs:
-    if (input.type == InputType.Coin || input.type == InputType.Message) and input.predicateLength == 0:
+    if (input.type == InputType.Coin or input.type == InputType.Message) and input.predicateLength == 0:
         # ECDSA signatures must be 64 bytes
         if tx.witnesses[input.witnessIndex].dataLength != 64:
             return False
@@ -192,10 +282,10 @@ Given transaction `tx`, the following checks must pass:
 
 If `tx.scriptLength == 0`, there is no script and the transaction defines a simple balance transfer, so no further checks are required.
 
-If `tx.scriptLength > 0`, the script must be executed. For each asset ID `asset_id` in the input set, the free balance available to be moved around by the script and called contracts is `freeBalance[asset_id]`. The initial message balance available to be moved around by the script and called contracts is `messageBalance`:
+If `tx.scriptLength > 0`, the script must be executed. For each asset ID `assetId` in the input set, the free balance available to be moved around by the script and called contracts is `freeBalance[assetId]`. The initial message balance available to be moved around by the script and called contracts is `messageBalance`:
 
 ```py
-freeBalance[asset_id] = available_balance(tx, asset_id) - unavailable_balance(tx, asset_id)
+freeBalance[assetId] = available_balance(tx, assetId) - unavailable_balance(tx, assetId)
 messageBalance = sum_data_messages(tx, 0)
 ```
 
@@ -205,13 +295,48 @@ Once the free balances are computed, the [script is executed](../fuel-vm/index.m
 1. The unspent free balance `unspentBalance` for each asset ID.
 1. The unspent gas `unspentGas` from the `$ggas` register.
 
-The fees incurred for a transaction are `ceiling(((size(tx) * GAS_PER_BYTE) + (tx.scriptGasLimit - unspentGas) + sum(tx.inputs[i].predicateGasUsed)) * tx.gasPrice / GAS_PRICE_FACTOR)`.
-
-`size(tx)` includes the entire transaction serialized according to the transaction format, including witness data.
+`size(tx)` encompasses the entire transaction serialized according to the transaction format, including witness data.
 This ensures every byte of block space either on Fuel or corresponding DA layer can be accounted for.
 
 If the transaction as included in a block does not match this final transaction, the block is invalid.
 
+### Fees
+
+The cost of a transaction can be described by:
+
+```py
+cost(tx) = gas_to_fee(min_gas(tx) + tx.gasLimit - unspentGas, tx.gasPrice)
+```
+
+where:
+
+- `min_gas(tx)` is the minimum cost of the transaction in gas, including intrinsic gas fees incurred from:
+  - The number of bytes comprising the transaction
+  - Processing inputs, including predicates
+  - Processing outputs
+  - VM initialization
+- `unspentGas` is the amount gas left over after intrinsic fees and execution of the transaction, extracted from the `$ggas` register. Converting unspent gas to a fee describes how much "change" is left over from the user's payment; the block producer collects this unspent gas as reward.
+- `gas_to_fee` is a function that converts gas to a concrete fee based on a given gas price.
+
+Fees incurred by transaction processing outside the context of execution are collectively referred to as intrinsic fees. Intrinsic fees include the cost of storing the transaction, calculated on a per-byte basis, the cost of processing inputs and outputs, including predicates and signature verification, and initialization of the VM prior to any predicate or script execution. Because intrinsic fees are independent of execution, they can be determined _a priori_ and represent the bare minimum cost of the transaction.
+
+A naturally occurring result of a variable gas limit is the concept of minimum and maximum fees. The minimum fee is, thus, the exact fee required to pay the fee balance, while the maximum fee is the minimum fee plus the gas limit:
+
+```py
+min_gas = min_gas(tx)
+max_gas = min_gas + (tx.witnessBytesLimit - tx.witnessBytes) * GAS_PER_BYTE + tx.gasLimit
+min_fee = gas_to_fee(min_gas, tx.gasPrice)
+max_fee = gas_to_fee(max_gas, tx.gasPrice)
+```
+
+The cost of the transaction `cost(tx)` must lie within the range defined by [`min_fee`, `max_fee`]. `min_gas` is defined as the sum of all intrinsic costs of the transaction known prior to execution. The definition of `max_gas` illustrates that the delta between minimum gas and maximum gas is the sum of:
+- The remaining allocation of witness bytes, converted to gas 
+- The user-defined `tx.gasLimit`
+
+Note that `gasLimit` applies to transactions of type `Script`. `gasLimit` is not applicable for transactions of type `Create` and is defined to equal `0` in the above formula.   
+
+A transaction cost `cost(tx)`, in gas, greater than `max_gas` is invalid and must be rejected; this signifies that the user must provide a higher gas limit for the given transaction. `min_fee` is the minimum reward the producer is guaranteed to collect, and `max_fee` is the maximum reward the producer is potentially eligible to collect. In practice, the user is always charged intrinsic fees; thus, `unspentGas` is the remainder of `max_gas` after intrinsic fees and the variable cost of execution. Calculating a conversion from `unspentGas` to an unspent fee describes the reward the producer will collect in addition to `min_fee`.
+
 ## VM Postcondition Validity Rules
 
 This section defines _VM postcondition validity rules_ for transactions: the requirements for a transaction to be valid after it has been executed.
@@ -242,7 +367,7 @@ In order for a coinbase transaction to be valid:
 1. It must be a [Mint](../tx-format/transaction.md#TransactionMint) transaction.
 1. The coinbase transaction must be the last transaction within a block, even if there are no other transactions in the block and the fee is zero.
 1. The `mintAmount` doesn't exceed the total amount of fees processed from all other transactions within the same block.
-1. The `mintAssetId` matches the `asset_id` that fees are paid in (`asset_id == 0`).
+1. The `mintAssetId` matches the `assetId` that fees are paid in (`assetId == 0`).
 
 The minted amount of the coinbase transaction intrinsically increases the balance corresponding to the `inputContract`.
 This means the balance of `mintAssetId` is directly increased by `mintAmount` on the input contract,

diff --git a/src/tx-format/policy.md b/src/tx-format/policy.md
@@ -51,4 +51,4 @@ Transaction is invalid if:
 Transaction is invalid if:
 
 - `max_fee > sum_inputs(tx, BASE_ASSET_ID) - sum_outputs(tx, BASE_ASSET_ID)`
-- `max_fee < fee_balance(tx, BASE_ASSET_ID)`
+- `max_fee < reserved_fee_balance(tx, BASE_ASSET_ID)`
diff --git a/src/tx-format/transaction.md b/src/tx-format/transaction.md
@@ -80,6 +80,7 @@ Given helper `sum_variants()` that sums all variants of an enum.
 
 Transaction is invalid if:
 
+- More than one output is of type `OutputType.Change` with identical `asset_id` fields.
 - Any output is of type `OutputType.ContractCreated`
 - `scriptLength > MAX_SCRIPT_LENGTH`
 - `scriptDataLength > MAX_SCRIPT_DATA_LENGTH`