How to benchmark the performance of your Hyperledger blockchain dApps?

Benchmarking the performance of blockchain dApps is very important because scalability is one of the biggest challenges faced by blockchain dApps. When the number of users in a blockchain network increases, the number of transactions and blocks added to the chain increases. This transaction history eventually slows down the network, and more participants join the network. This article will explain how to benchmark Hyperledger blockchain applications using Hyperledger Caliper.

In the example, we’ll benchmark a private blockchain with the IBFT2.0 consensus mechanism created using Hyperledger besu. 

Hyperledger Caliper is a blockchain benchmark tool it allows users to measure the performance of a blockchain implementation with a set of predefined use cases. Hyperledger Caliper will produce reports containing many performance indicators to serve as a reference when using the following blockchain solutions:

1. Hyperledger Besu
2. Hyperledger Burrow
3. Ethereum
4. Hyperledger Fabric
5. FISCO BCOS,
6. Hyperledger Iroha
7. Hyperledger Sawtooth.

In the example, we’ll consider a private blockchain with four nodes. First, we have to configure the connectors and benchmarking scenarios. Hyperledger has provided some generic examples for different cases in this repo. But in this tutorial, we’ll focus on a specific use case, that is, to test the ERC20 token transfer characteristics of our private blockchain.

You can find the final code used in this example here. Before jumping into details, I recommend going through Hyperledger Caliper’s documentation. You can use either the source code of Caliper or the docker image. I am going to use a docker image of Caliper for benchmarking. So I have included a docker-compose file in the root that looks like this. 

version: '3'
services: 
caliper:
container_name: caliper
working_dir: /hyperledger/caliper/workspace
image: hyperledger/caliper:0.4.2 
command: launch manager 
network_mode: host
environment:
- CALIPER_BIND_SUT=besu:latest 
- CALIPER_BENCHCONFIG=benchmarks/scenario/simple/config.yaml 
- CALIPER_NETWORKCONFIG=networks/networkconfig.json 
- CALIPER-FLOW-SKIP-INSTALL=true 
volumes: 
- ./caliper-benchmarks-erc20:/hyperledger/caliper/workspace

You may have to consider some env variables defined in this code. CALIPER_BIND_SUT is to specify the System Under Test (SUT). Currently, caliper supports only a bunch of networks. We have specified it as Besu because we benchmark a network created by Hyperledger Besu. Then there are two important files mentioned here, CALIPER_BENCHCONFIG and CALIPER_NETWORKCONFIG. Benchconfig points to the file which contains the test cases or benchmarking scenarios. Network config specifies the SUT. We will look into both of these later. 

Another variable is CALIPER-FLOW-SKIP-INSTALL. Generally, there is an order for the benchmarking process of Caliper, which is specified as init, install, start, test and end. 

Under the hood, Caliper is deploying a predefined smart contract to the network and executing some function calls to this contract. By monitoring these contract function calls, the Caliper generates a report. In the example given by Hyperledger, they provided a simple contract. It only has three functions create, modify and query. When we run Caliper using that example, it will first deploy this contract to SUT and execute these functions in different conditions. That is the general way of testing. But what if you want to use our smart contracts? That is also possible. We can add our contract details into configurations.

However, Caliper has a small drawback in this area. We cannot use smart contracts that accept arguments in the constructor because there isn’t any provision to pass arguments while deploying the smart contract, given that Caliper manages deployment. A solution to this problem is to deploy a contract by ourselves and give the details to the Caliper. That is what we are doing here. In that situation, we have to inform Caliper that you don’t have to deploy any contracts, and CALIPER-FLOW-SKIP-INSTALL does that.

Moving into the core, there are three things that we have to consider in the code.

• Benchmark configuration file
• Network configuration file
• Workload modules

Benchmark configuration file

The benchmark configuration file describes how the benchmark should be executed. It tells Caliper how many rounds it should execute, at what rate the TXs should be submitted, and which module will generate the TX content. It also includes settings about monitoring the SUT.

simpleArgs: &simple-args
moneyToTransfer: &money-to-transfer 1
numberOfAccounts: 1

test:
name: erc20 
description: >-
To benchmark the network transactions. 
workers:
type: local 
number: 1 
rounds: 
- label: 100 txns with 10-100tps
description: Transfering erc20 token between accounts. 
txNumber: 100 
rateControl:
type: linear-rate 
opts:
startingTps: 10
finishingTps: 100 
workload: 
module: benchmarks/scenario/simple/transfer.js 
arguments:
<< : *simple-args
money: *money-to-transfer 
- label: 100 txns with 100tps
description: Transfering erc20 token between accounts. 
txNumber: 100 
rateControl:
type: fixed-rate 
opts:
tps: 100 
workload:
module: benchmarks/scenario/simple/transfer.js 
arguments:
<< : *simple-args 
money: *money-to-transfer

In this case, we have specified two test cases, 100 transactions with a linear rate of 10 –100 transactions per second and 100 transactions at a fixed rate. You can configure this to your preferences. But, there are more types other than linear rate and fixed rate. You can try that also. Consider starting with smaller values. Otherwise, there is a chance that your node will be overloaded. Don’t panic! Just stop the caliper and restart the node. You are ready to test again. Anyway, we are going with this configuration for the time being.

Related article on: What to know before implementing blockchain in your business?

Network configuration file

The content of the network configuration file is SUT-specific. The file usually describes the topology of the SUT, where its nodes are (their endpoint addresses), what identities/clients are present in the network, and what smart contracts Caliper should deploy or need to interact with.

{
 "caliper": {
 "blockchain": "ethereum"
 }
"ethereum": {
"url": "ws://localhost:8546", 
"contractDeployerAddress": "XXXXXXXXXXXXXXXXXX", 
"contractDeploverAddressPrivateKey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", 
"fromAddress": "XXXXXXXXXXXXXXXXXX",
"fromAddressPrivateKey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", 
"transactionConfirmationBlocks": 1, 
"contracts": { 
"erc20": {
"address": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX", 
"estimateGas": true, 
"gas": {
"transfer": 80000
 }
   "abi": []
   }
  }
 }
}

First, we have specified the connector as “Ethereum” since besu is a fork of Ethereum. Then SUT specifics are given. Please remember to expose the socket RPC URL in your connecting node. Because socket connection is more reliable than HTTP RPC connection when benchmarking.

We need to deploy an ERC20 contract to the network before editing this file. Then provide the addresses here in the config. Also need to specify the gas limit for the function and abi of the contract. (I left it empty in this image because it’s lengthy!!!)

Related article on: How to approach blockchain app development in 2022?

Workload modules

Workload modules are the brain of a benchmark. Since Caliper is a general benchmark framework, it does not include any concrete benchmark implementation. When Caliper schedules TXs for a given round, it is the task of the round’s workload module to generate the content of the TXs and submit it. Each round can have a different associated workload module, so separating your workload implementation based on phases/behavior should be easy. Workload modules are Node.JS modules that must export a given factory function. Other than that, the workload module logic can be arbitrary. Anything you can code in Node.JS.

Since it is the brain, I haven’t changed much on that. Even though small changes are done to it, you can find it yourself.

'use strict';
const OperationBase = require('./utils/operation-base');
 const SimpleState = require('./utils/simple-state');
 /** 
* Workload module for transferring money between accounts. 
*/
 class Transfer extends Operation Base {

/** 
* Initializes the instance.
*/
constructor() {
super();
}
/** 
* Create a pre-configured state representation. 
* @return {SimpleState} The state instance.
*/

createSimpleState() {
const accountsPerworker = this.numberOfAccounts / this. total Workers;
return new SimpleState(this.workerIndex, this.moneyToTransfer, accounts PerWorker);
}
/** 
* Assemble TXs for transferring money.
*/ 
async submitTransaction() {
const transferArgs = this.simplestate.getTransferArguments();
await this.sutAdapter.send Requests(this.createConnectorRequest('transfer', transferArgs));
}
}
 /** 
 * Create a new instance of the workload module.  
* @return {WorkloadModuleInterface} 
*/
function createWorkloadModule() {
return new Transfer(); 
}  
module.exports.createWorkloadModule = createWorkloadModule; 

Then we need to implement the transfer function that we are using for testing. In line 30, you can see how the function interacts with the contract using the connector.

'use strict';

 /**
 * Class for managing simple account states. 
 */ 
class SimpleState {
/** 
* Initializes the instance.
*/
constructor(workerIndex, moneyToTransfer, accounts = 0) {
this.accountsGenerated = accounts; 
this.moneyToTransfer = moneyToTransfer; 
this.accountPrefix = this._get26Num( workerIndex);
}

/*
* Generate string by picking characters from the dictionary variable. 
* @param {number} number Character to select. 
* @returns {string} string Generated string based on the input number. 
* @private
*/
_get26Num( number){
let result = '';
while(number > 0) {
result += Dictionary.charAt(number % Dictionary.length); 
number = parseInt(number / Dictionary.length);
}
return result;
}
/**
* Get the arguments for transfering money between accounts. 
* @returns {object} The account arguments.
*/
getTransferArguments() { 
return {
target: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", 
amount: this.moneyToTransfer
};
} 
}
module. exports = SimpleState;

Moreover, this is a helper file for transfer.js. We have specified a single address as a target because we send all 100 txns to the same account. If you want to, you can put some logic so that it will send 100 txns to 100 different accounts. The hyperledger example has such logic in it.

'use strict';
const { WorkloadModuleBase } = require( '@hyperledger/caliper-core');
const SupportedConnectors = ['ethereum'];
/**  
* Base class for simple operations. 
*/ 
class OperationBase extends WorkloadModuleBase { 
/** 
* Initializes the base class.
*/
constructor() {
super();
}
/**
* Initialize the workload module with the given parameters.
*@param {number} workerIndex The 0-based index of the worker instantiating the workload module.
* @param {number} totalworkers The total number of workers participating in the round. 
* @param {number} roundIndex The 0-based index of the currently executing round. 
* @param {Object} roundArguments The user-provided arguments for the round from the benchmark configuration file.
* @param {ConnectorBase} sutAdapter The adapter of the underlying SUT. 
* @param {Object} sutContext The custom context object provided by the SUT
adapter. 
* @async

*/
 async initializeWorkloadModule(workerIndex, totalworkers, roundIndex, roundArguments, sutAdapter, sutContext) {
await super.initializeWorkloadModule(workerIndex, totalworkers, roundIndex, roundArguments, sutAdapter, sutContext);

this.assertConnectorType(); 
// this.assertSetting('initialMoney'); 
this.assertSetting('moneyToTransfer'); 
// this.assertSetting('numberOfAccounts');

// this.initialMoney = this.roundArguments.initialMoney; 
this.moneyToTransfer = this.roundArguments.moneyToTransfer; 
// this. numberOfAccounts = this.roundArguments. numberOfAccounts; 
this.simpleState = this.createSimpleState();
}
/**
* Performs the operation mode-specific initialization. 
* @return {SimpleState} the initialized SimpleState instance. 
* @protected
*/
createSimpleState() {
throw new Error('Simple workload error: "createSimpleState" must be overridden in derived classes');
}

/**
* Assert that the used connector type is supported. Only Fabric is supported currently.
* @protected
*/
assertConnectorType() {
this.connector Type = this.sutAdapter.getType(); 
if (!SupportedConnectors.includes(this.connectorType)) {
throw new Error Connector type ${this.connectorType} is not supported by the benchmark'); 
}
}
/**
* Assert that a given setting is present among the arguments. 
* @param {string} settingName The name of the setting. 
* @protected 
*/ 
assertSetting(settingName) { 
if(!this.roundArguments.hasOwn Property(settingName)) {
throw new Error(Simple workload error: module setting "${settingName}" is missing from the benchmark configuration file'); 
}
}
/**
*Assemble a connector-specific request from the business parameters. 
* @param {string} operation The name of the operation to invoke.
* @param {object} args The object containing the arguments. 77 
* @return {object} The connector-specific request.
* @protected
*/
createConnectorRequest(operation, args) { 
switch (this.connectorType) { 
case 'ethereum':
return this._createEthereumConnectorRequest(operation, args); 
default:
// this shouldn't happen

throw new Error( Connector type ${this.connectorType} is not supported by the benchmark');
}
}

/** 
* Assemble a Ethereum-specific request from the business parameters. 
* @param {string} operation The name of the operation to invoke. 
* @param {object} args The object containing the arguments. 
* @return {object} The Ethereum-specific request. 
* @private
*/
_createEthereumConnectorRequest(operation, args) { 
return { 
contract: 'erc20', 
verb: operation,
args: Object.keys(args ).map(k => args[k]), 
readOnly: false 
};
}
}
 module.exports = OperationBase;

In addition, this is another helper, and it is more connector specific. On line 98, we have a function to generate the smart contract function call request.

Now we all set. Execute docker-compose up to run the caliper. Check the logs of the caliper and the node too. To make sure we are not overloading our node. After successful execution, you will get something like this in the console.

And, a report like this.

I hope you find this article helpful. Feel free to reach out to me if you have any questions or doubts about benchmarking your blockchain dApps. In the upcoming articles, we’ll learn how to benchmark blockchains on other prominent blockchain networks.

This article was repurposed from this original article