mukan-consensus/scripts/qa/reporting/README.md

Reporting Scripts

This directory contains some utility scripts used in the reporting/QA.

• latency_throughput.py is a Python script that uses matplotlib to plot a graph of transaction latency vs throughput rate based on the CSV output generated by the loadtime reporting tool.

• latency_plotter.py is a Python script that uses matplotlib and pandas to plot graph of transaction latency vs throughput rate based on the CSV output generated by the loadtime reporting tool, for multiple experiments and configurations.

• prometheus_plotter.py is a Python script that uses matplotlib and pandas to plot graphs of several metrics from Prometheus.

Setup

Execute the following within this directory (the same directory as the latency_throughput.py file).

# Create a virtual environment into which to install your dependencies
python3 -m venv .venv

# Activate the virtual environment
source .venv/bin/activate

# Install dependencies listed in requirements.txt
pip install -r requirements.txt

Latency vs Throughput Plotting

To show the instructions and parameter options, execute

./latency_throughput.py --help

Example:

# Do the following while ensuring that the virtual environment is activated (see
# the Setup steps).
#
# This will generate a plot in a PNG file called 'tm034.png' in the current
# directory based on the reporting tool CSV output in the "raw.csv" file. The
# '-t' flag overrides the default title at the top of the plot.

./latency_throughput.py \
    -t 'CometBFT v0.34.x Latency vs Throughput' \
    ./tm034.png \
    /path/to/csv/files/raw.csv

Latency vs Throughput Plotting (version 2)

Example:

# Do the following while ensuring that the virtual environment is activated (see
# the Setup steps).
#
# This will generate a series of plots in the `imgs` folder.
# Plots include combined experiment plots and experiments as subplots.
# - all_experiments - plots of all experiments as individual subplots.
# - all_configs - plots of all experiments, grouped by configuration (r,c).
# cXrY.png - Independent plot of experiments of configuration (c=X,r=Y) as different curves.
# cXrY_merged.png - Independent plot of experiments of configuration (c=X,r=Y) combined as single curve.
# e_ID.png - independent plot with just experiment with id ID as a single curve.

mkdir -p imgs
python3 latency_plotter.py /path/to/csv/files/raw.csv

Prometheus metrics

1. Ensure that Prometheus is running locally and listening on port 9090.
2. Tweak the script to your needs
   1. Adjust the time window
   2. Select the right fork
   3. Select the right test case
   4. Tweak/add/remove metrics
3. Run the script as follows

   # Do the following while ensuring that the virtual environment is activated (see
   # the Setup steps).
   #
   # This will generate a series of plots in the folder `imgs` of the current folder.
   
   mkdir imgs
   python3 prometheus_plotter.py
   

4. Plots are saved in the imgs folder.