Runtime Upgrade

This document explains how to support forkless upgrades in Substrate-based networks through runtime versioning and storage migration.

Forkless runtime upgrade is a core feature of the Substrate framework for blockchain development. The ability to update runtime logic without forking the codebase allows the blockchain to evolve and improve over time. This feature is made possible by including the definition of the runtime WebAssembly blob, which is a runtime state element, in the blockchain's runtime state.

Since the runtime is part of the blockchain state, network administrators can safely improve the runtime using a trusted decentralized consensus mechanism.

In the FRAME system for runtime development, the set_code call is defined, which is used to update the runtime definition. The Upgrade a Running Network tutorial demonstrates two methods to upgrade the runtime without shutting down nodes or interrupting operations. However, both upgrades in the tutorial only demonstrate adding functionality to the runtime, not updating the existing runtime state. If a runtime upgrade requires changes to the existing state, storage migration is likely necessary.

Runtime Versioning

In the build process, we learned that compiling a node generates platform-specific binaries and WebAssembly binaries, and we can control which binaries to use at different points in the block production process through command-line options for execution strategy. The component used to communicate with the runtime execution environment is called the executor. While it is possible to override the default execution strategy in custom scenarios, in most cases, the executor evaluates the following information from the native runtime and WebAssembly runtime to select the binary to use:

spec_name
spec_version
authoring_version

To provide this information to the executor process, the runtime includes a runtime version struct similar to the following:

pub const VERSION: RuntimeVersion = RuntimeVersion {
  spec_name: create_runtime_str!("node-template"),
  impl_name: create_runtime_str!("node-template"),
  authoring_version: 1,
  spec_version: 1,
  impl_version: 1,
  apis: RUNTIME_API_VERSIONS,
  transaction_version: 1,
};

The parameters of the struct provide the following information:

Parameter Description

Parameter	Description
`spec_name`	An identifier that distinguishes different Substrate runtimes.
`impl_name`	The implementation name of the specification. This is not significant for nodes but is used to differentiate code from different implementation teams.
`authoring_version`	If the authoring node does not have the same value as the native runtime, it will not produce blocks.
`spec_version`	The version of the runtime specification. The overall node does not use the WebAssembly runtime instead of the native runtime unless `spec_name`, `spec_version`, and `authoring_version` are the same in both WebAssembly and native binaries. Updating `spec_version` can be automated through a CI process and is done in the Polkadot network. This parameter typically increases when there is an update to `transaction_version`.
`impl_version`	The implementation version of the specification. The node ignores this value. It is used to indicate that the code may be different. The native and WebAssembly binaries perform the same operations even if `authoring_version` and `spec_version` are the same. Changes to `impl_version` typically occur due to optimizations that do not break logic.
`transaction_version`	The version of the transaction processing interface. This parameter is useful for hardware wallets to synchronize firmware updates to ensure secure signing of runtime transactions. This number should be updated when there are changes to the index of palettes or the parameters or parameter types of dispatchable functions in the `construct_runtime!` macro. When this number is updated, `spec_version` should also be updated.
`apis`	A list of supported runtime APIs and their versions.

spec_name

An identifier that distinguishes different Substrate runtimes.

impl_name

The implementation name of the specification. This is not significant for nodes but is used to differentiate code from different implementation teams.

authoring_version

If the authoring node does not have the same value as the native runtime, it will not produce blocks.

spec_version

The version of the runtime specification. The overall node does not use the WebAssembly runtime instead of the native runtime unless spec_name, spec_version, and authoring_version are the same in both WebAssembly and native binaries. Updating spec_version can be automated through a CI process and is done in the Polkadot network. This parameter typically increases when there is an update to transaction_version.

impl_version

The implementation version of the specification. The node ignores this value. It is used to indicate that the code may be different. The native and WebAssembly binaries perform the same operations even if authoring_version and spec_version are the same. Changes to impl_version typically occur due to optimizations that do not break logic.

transaction_version

The version of the transaction processing interface. This parameter is useful for hardware wallets to synchronize firmware updates to ensure secure signing of runtime transactions. This number should be updated when there are changes to the index of palettes or the parameters or parameter types of dispatchable functions in the construct_runtime! macro. When this number is updated, spec_version should also be updated.

apis

A list of supported runtime APIs and their versions.

The orchestration engine(sometimes referred to as the executor) verifies that the native runtime has the same consensus logic as the WebAssembly before executing it. However, since runtime versioning is manually set, if the runtime version is incorrectly indicated, the orchestration engine may make inappropriate decisions.

Accessing the Runtime Version

The FRAME system exposes the runtime version information through the state.getRuntimeVersion RPC endpoint. This endpoint allows an optional block identifier. However, in most cases, the runtime metadata is used to understand the APIs exposed by the runtime and how to interact with them. The runtime metadata should only be changed when the runtime's spec_version changes.

Forkless Runtime Upgrade

Traditional blockchains require a hard fork when upgrading the state transition function, which means all node operators need to stop their nodes and manually upgrade to the latest executable. Coordinating a hard fork upgrade in a distributed production network can be a complex process.

The runtime versioning properties allow Substrate-based blockchains to upgrade the runtime logic in real-time without forking the network.

To perform a runtime upgrade without forking, Substrate updates the WebAssembly runtime stored on the blockchain with a new version of logic that breaks the existing consensus. This upgrade is propagated to all full nodes in the network as part of the consensus process. When the WebAssembly runtime is upgraded, the orchestration engine detects that the spec_name, spec_version, or authoring_version of the native runtime does not match the new WebAssembly runtime. As a result, the orchestration engine executes the normative WebAssembly runtime instead of the native runtime in any execution process.

Storage Migration

Storage migration is a custom one-time function that updates the storage to adapt to changes in the runtime. For example, if a runtime upgrade changes the data type representing user balances from an unsigned integer to a signed integer, the storage migration reads the existing value as an unsigned integer and then writes the updated value converted to a signed integer. If such a change in data storage is required and not performed, the runtime cannot interpret the storage values correctly and may result in undefined behavior.

Storage migration using FRAME

FRAME storage migration is implemented using the OnRuntimeUpgrade trait. The OnRuntimeUpgrade trait defines a single function called on_runtime_upgrade that specifies the logic to be executed immediately after a runtime upgrade.

Preparing for Storage Migration

Preparing for storage migration means understanding the changes defined by the runtime upgrade. The Substrate repository uses the E1-runtimemigration label to specify such changes.

Writing Migrations

Each storage migration is different based on requirements and complexity. However, when performing a storage migration, the following recommendations can be used as guidance:

Extract the migration into a reusable function and write tests for that function.
Include logging in the migration for debugging purposes.
The migration code should include deprecated types within the context of the upgraded runtime, as shown in this example.
Use storage versions to make the migration more declarative and safer, as shown in this example.

Migration Order

By default, FRAME orders the execution of the on_runtime_upgrade functions based on the order in which the palettes appear in the construct_runtime! macro. For upgrades, the functions are executed in reverse order, starting from the last palette. Custom order can also be specified if needed, as shown in this example.

FRAME storage migrations are executed in the following order:

Custom on_runtime_upgrade function if using a custom order.
System frame_system::on_runtime_upgrade function.
All on_runtime_upgrade functions defined in the construct_runtime! macro, starting from the last palette.

Migration Testing

Testing storage migration is important. Some tools that can be used to test storage migration are:

The Substrate Debug Kit includes the remote externalities tool, which allows safe unit testing of storage migration with live chain data.
The fork-off-substrate script makes it easy to generate chain specs to bootstrap a local test chain for testing runtime upgrades and storage migration.

Next Steps

PreviousMonitoring NextTutorials

Last updated 7 months ago