Add Offchasin Workers
Illustrates how to modify a pallet to include an offchain worker and how to submit transactions from an offchain worker to update the on-chain state.
Last updated
Illustrates how to modify a pallet to include an offchain worker and how to submit transactions from an offchain worker to update the on-chain state.
Last updated
This tutorial illustrates how to modify a pallet to include an offchain worker and configure the pallet and runtime to enable the offchain worker to submit transactions that update the on-chain state.
If you use offchain workers to perform long-running computations or fetch data from offline sources, it's likely that you'll want to store the results of those operations on-chain. However, offchain storage is separate from on-chain resources and you can't save data processed by offchain workers directly to on-chain storage. To store any data processed by offchain workers as part of the on-chain state, you must create transactions to send the data from the offchain worker storage to the on-chain storage system.
This tutorial illustrates how to create offchain workers with the ability to send signed or unsigned transactions to store offchain data on-chain. In general, signed transactions are more secure, but require the calling account to handle transaction fees. For example:
Use signed transactions if you want to record the associated transaction caller account and deduct the transaction fee from the caller account.
Use unsigned transactions with signed payload if you want to record the associated transaction caller, but do not want the caller be responsible for the transaction fee payment.
It's also possible to submit unsigned transactions without a signed payload—for example, because you don't want to record the associated transaction caller at all. However, there's significant risk in allowing unsigned transactions to modify the chain state. Unsigned transactions represent a potential attack vector that a malicious user could exploit. If you are going to allow offchain workers to send unsigned transactions, you should include logic that ensures the transaction is authorized. For an example of how unsigned transactions can be verified using on-chain state, see the ValidateUnsigned
implementation in the call. In that example, the call validates the unsigned transaction by verifying that the given code hash was previously authorized.
It is also important to consider that even an unsigned transaction with a signed payload could be exploited because offchain workers can't be assumed to be a reliable source unless you implement strict logic to check the validity of the transaction. In most cases, checking whether a transaction was submitted by an offchain worker before writing to storage isn't sufficient to protect the network. Instead of assuming that the offchain worker can be trusted without safeguards, you should intentionally set restrictive permissions that limit access to the process and what it can do.
Remember that unsigned transactions are essentially an open door into your runtime. You should only use them after careful consideration of the conditions under which they should be allowed to execute. Without safeguards, malicious actors could impersonate offchain workers and access runtime storage.
Before you begin, verify the following:
You have configured your environment for Substrate development by installing .
You have completed the tutorial and have the Substrate node template from the Developer Hub installed locally.
You are familiar with how to use FRAME macros and edit the logic for a pallet.
You are familiar with how to modify the configuration trait for a pallet in the runtime.
By completing this tutorial, you will be able to:
Identify the risks involved in using unsigned transactions.
Add an offchain worker function to a pallet.
Configure the pallet and the runtime to enable the offchain worker to submit signed transactions.
Configure the pallet and the runtime to enable the offchain worker to submit unsigned transactions.
Configure the pallet and the runtime to enable the offchain worker to submit unsigned transactions with a signed payload.
To submit signed transactions, you must configure your pallet and the runtime to enable at least one account for offchain workers to use. At a high level, configuring a pallet to use an office chain worker and submit signed transactions involves the following steps:
To enable offchain workers to send signed transactions:
Open the src/lib.rs
file for your pallet in a text editor.
Add the #[pallet::hooks]
macro and the entry point for offchain workers to the code.
For example:
Add the logic for the offchain_worker
function.
Add an AuthorityId
type to the pallet Config
trait:
Add a crypto
module with an sr25519
signature key to ensure that your pallet owns an account that can be used for signing transactions.
Initialize an account for the offchain worker to use to send a signed transaction to on-chain storage.
This code enables you to retrieve all signers that this pallet owns.
Use send_signed_transaction()
to create a signed transaction call:
Check if the transaction is successfully submitted on-chain and perform proper error handling by checking the returned results
.
Open the runtime/src/lib.rs
file for the node template in a text editor.
Add the AuthorityId
to the configuration for your pallet and make sure it uses the TestAuthId
from the crypto
module:
Implement the CreateSignedTransaction
trait in the runtime.
Because you configured your pallet to implement the CreateSignedTransaction
trait, you also need to implement that trait for the runtime.
This code snippet is long, but, in essence, it illustrates the following main steps:
Create and prepare extra
of SignedExtra
type, and put various checkers in place.
Create a raw payload based on the passed in call
and extra
.
Sign the raw payload with the account public key.
Bundle all data up and return a tuple of the call, the caller, its signature, and any signed extension data.
Implement SigningTypes
and SendTransactionTypes
in the runtime to support submitting transactions, whether they are signed or unsigned.
At this point, you have prepared your pallet to use offchain workers. Preparing the pallet involved the following steps:
Adding the offchain_worker
function and related logic for sending signed transactions.
Adding CreateSignedTransaction
and AuthorityId
to the Config
trait for your pallet.
Adding the crypto
module to describe the account the pallet will use to sign transaction.
You have also updated the runtime with the code to support offchain workers and sending signed transactions. Updating the runtime involved the following steps:
Adding the AuthorityId
to the runtime configuration for your pallet.
Implementing the CreateSignedTransaction
trait and create_transaction()
function.
Implementing SigningTypes
and SendTransactionTypes
for offchain workers from the frame_system
pallet.
However, before your pallet offchain workers can submit signed transactions, you must specify at least one account for the offchain worker to use. To enable the offchain worker to sign transactions, you must generate the account key for the pallet to own and add that key to the node keystore.
There are several ways to accomplish this final step and the method you choose might vary depending on whether you are running a node in development mode for testing, using a custom chain specification, or deploying into a production environment.
If you are running a node in development mode—with --dev
command-line option—you can manually generate and insert the account key for a development account by modifying the node/src/service.rs
file as follows:
In a production environment, you can use other tools—such as subkey
—to generate keys that are specifically for offchain workers to use. After you generate one or more keys for offchain workers to own, you can add them to the node keystore by:
Modifying the configuration of your chain specification file.
Passing parameters using the author_insertKey
RPC method.
Note that the keyType parameter demo
in this example matches the KEY_TYPE
declared in the offchain worker pallet.
Now, your pallet is ready to send signed transactions on-chain from offchain workers.
By default, all unsigned transactions are rejected in Substrate. To enable Substrate to accept certain unsigned transactions, you must implement the ValidateUnsigned
trait for the pallet.
Although you must implement the ValidateUnsigned
trait to send unsigned transactions, this check doesn't guarantee that only offchain workers are able to send the transaction. You should always consider the consequences of malicious actors sending these transactions as an attempt to tamper with the state of your chain. Unsigned transactions always represent a potential attack vector that a malicious user could exploit and offchain workers can't be assumed to be a reliable source without additional safeguards.
You should never assume that unsigned transactions can only be submitted by an offchain worker. By definition, anyone can submit them.
To enable offchain workers to send unsigned transactions:
Open the src/lib.rs
file for your pallet in a text editor.
For example:
Implement the trait as follows:
Check the calling extrinsics to determine if the call is allowed and return ValidTransaction
if the call is allowed or TransactionValidityError
if the call is not allowed.
For example:
In this example, users can only call the specific my_unsigned_tx
function without a signature. If there are other functions, calling them would require a signed transaction.
Add the #[pallet::hooks]
macro and the offchain_worker
function to send unsigned transactions as follows:
Enable the ValidateUnsigned
trait for the pallet in the runtime by adding the ValidateUnsigned
type to the construct_runtime
macro.
For example:
Sending unsigned transactions with signed payloads is similar to sending unsigned transactions. You need to:
Implement the ValidateUnsigned
trait for the pallet.
Add the ValidateUnsigned
type to the runtime when using this pallet.
Send the transaction with the signed payload.
Keep in mind that unsigned transactions always represent a potential attack vector and that offchain workers can't be assumed to be a reliable source without additional safeguards. In most cases, you should implement restrictive permissions or additional logic to verify the transaction submitted by an offchain worker is valid.
The differences between sending unsigned transactions and sending unsigned transactions with signed payload are illustrated in the following code examples.
To make your data structure signable:
For example:
In the offchain_worker
function, call the signer, then the function to send the transaction:
This code retrieves the signer
then calls send_unsigned_transaction()
with two function closures. The first function closure returns the payload to be used. The second function closure returns the on-chain call with payload and signature passed in. This call returns an Option<(Account<T>, Result<(), ()>)>
result type to allow for the following results:
None
if no account is available for sending the transaction.
Some((account, Ok(())))
if the transaction is successfully sent.
Some((account, Err(())))
if an error occurs when sending the transaction.
Check whether a provided signature matches the public key used to sign the payload:
This tutorial provides simple examples of how you can use offchain workers to send transactions for on-chain storage. To learn more, explore the following resources:
.
.
.
Add to the Config
trait for your pallet. For example, your pallet Config
trait should look similar to this:
The declares an account with an sr25519
signature that is identified by KEY_TYPE
. In this example, the KEY_TYPE
is demo
. Note that this macro doesn't create a new account. The macro simply declares that a crypto
account is available for this pallet to use.
By looking at , you can see that you only need to implement the function create_transaction()
for the runtime. For example:
You can see an example of this code in the .
You can see an example of this implementation in the .
This example manually adds the key for the Alice
account to the keystore identified by the KEY_TYPE
defined in your pallet. For a working example, see this sample file.
For example, you can use the , Polkadot-JS API, or a curl
command to select the author_insertKey
method and specify the key type, secret phrase, and public key parameters for the account to use:
Add the macro.
For an example of how ValidateUnsigned
is implemented in a pallet, see the code for the .
This code prepares the call in the let call = ...
line, submits the transaction using , and performs any necessary error handling in the callback function passed in.
Implement the SendTransactionTypes
trait for the runtime as described in .
For a full example, see the examples pallet.
Prepare the data structure to be signed—the signed payload—by implementing the trait.
You can refer to the section on for more information about implementing the ValidateUnsigned
trait and adding the ValidateUnsigned
type to the runtime.
Implement .
For an example of a signed payload, see the code for the .
This example uses to verify that the public key in the payload has the same signature as the one provided. However, you should note that the code in the example only checks whether the provided signature
is valid for the public
key contained inside payload
. This check doesn't validate whether the signer is an offchain worker or authorized to call the specified function. This simple check wouldn't prevent an unauthorized actor from using the signed payload to modify state.
For working examples of this code, see the and the implementation of .