Mukan Erkin Törük
ef24c0b67e
Some checks are pending
docker-build-cometbft / vars (push) Waiting to run
docker-build-cometbft / build-images (amd64, ubuntu-24.04) (push) Blocked by required conditions
docker-build-cometbft / build-images (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
docker-build-cometbft / merge-images (push) Blocked by required conditions
docker-build-e2e-node / vars (push) Waiting to run
docker-build-e2e-node / build-images (amd64, ubuntu-24.04) (push) Blocked by required conditions
docker-build-e2e-node / build-images (arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
docker-build-e2e-node / merge-images (push) Blocked by required conditions
97 lines
3.2 KiB
Markdown
97 lines
3.2 KiB
Markdown
# Reporting Scripts
|
|
|
|
This directory contains some utility scripts used in the reporting/QA.
|
|
|
|
* [`latency_throughput.py`](./latency_throughput.py) is a Python script that uses
|
|
[matplotlib] to plot a graph of transaction latency vs throughput rate based on
|
|
the CSV output generated by the [loadtime reporting
|
|
tool](../../../test/loadtime/cmd/report/).
|
|
|
|
* [`latency_plotter.py`](./latency_plotter.py) is a Python script that uses
|
|
[matplotlib] and [pandas] to plot graph of transaction latency vs throughput rate based on
|
|
the CSV output generated by the [loadtime reporting
|
|
tool](../../../test/loadtime/cmd/report/), for multiple experiments and configurations.
|
|
|
|
* [`prometheus_plotter.py`](./prometheus_plotter.py) is a Python script that uses
|
|
[matplotlib] and [pandas] to plot graphs of several metrics from Prometheus.
|
|
|
|
## Setup
|
|
|
|
Execute the following within this directory (the same directory as the
|
|
`latency_throughput.py` file).
|
|
|
|
```bash
|
|
# Create a virtual environment into which to install your dependencies
|
|
python3 -m venv .venv
|
|
|
|
# Activate the virtual environment
|
|
source .venv/bin/activate
|
|
|
|
# Install dependencies listed in requirements.txt
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Latency vs Throughput Plotting
|
|
To show the instructions and parameter options, execute
|
|
|
|
```bash
|
|
./latency_throughput.py --help
|
|
```
|
|
|
|
Example:
|
|
|
|
```bash
|
|
# Do the following while ensuring that the virtual environment is activated (see
|
|
# the Setup steps).
|
|
#
|
|
# This will generate a plot in a PNG file called 'tm034.png' in the current
|
|
# directory based on the reporting tool CSV output in the "raw.csv" file. The
|
|
# '-t' flag overrides the default title at the top of the plot.
|
|
|
|
./latency_throughput.py \
|
|
-t 'CometBFT v0.34.x Latency vs Throughput' \
|
|
./tm034.png \
|
|
/path/to/csv/files/raw.csv
|
|
```
|
|
|
|
## Latency vs Throughput Plotting (version 2)
|
|
Example:
|
|
|
|
```bash
|
|
# Do the following while ensuring that the virtual environment is activated (see
|
|
# the Setup steps).
|
|
#
|
|
# This will generate a series of plots in the `imgs` folder.
|
|
# Plots include combined experiment plots and experiments as subplots.
|
|
# - all_experiments - plots of all experiments as individual subplots.
|
|
# - all_configs - plots of all experiments, grouped by configuration (r,c).
|
|
# cXrY.png - Independent plot of experiments of configuration (c=X,r=Y) as different curves.
|
|
# cXrY_merged.png - Independent plot of experiments of configuration (c=X,r=Y) combined as single curve.
|
|
# e_ID.png - independent plot with just experiment with id ID as a single curve.
|
|
|
|
mkdir -p imgs
|
|
python3 latency_plotter.py /path/to/csv/files/raw.csv
|
|
```
|
|
|
|
## Prometheus metrics
|
|
|
|
1. Ensure that Prometheus is running locally and listening on port 9090.
|
|
2. Tweak the script to your needs
|
|
1. Adjust the time window
|
|
2. Select the right fork
|
|
3. Select the right test case
|
|
4. Tweak/add/remove metrics
|
|
3. Run the script as follows
|
|
```bash
|
|
# Do the following while ensuring that the virtual environment is activated (see
|
|
# the Setup steps).
|
|
#
|
|
# This will generate a series of plots in the folder `imgs` of the current folder.
|
|
|
|
mkdir imgs
|
|
python3 prometheus_plotter.py
|
|
```
|
|
4. Plots are saved in the `imgs` folder.
|
|
|
|
[matplotlib]: https://matplotlib.org/
|
|
[pandas]: https://pandas.pydata.org
|