# Reporting Scripts

This directory contains some utility scripts used in the reporting/QA.

* [`latency_throughput.py`](./latency_throughput.py) is a Python script that uses
    [matplotlib] to plot a graph of transaction latency vs throughput rate based on
    the CSV output generated by the [loadtime reporting
    tool](../../../test/loadtime/cmd/report/).

* [`latency_plotter.py`](./latency_plotter.py) is a Python script that uses
    [matplotlib] and [pandas] to plot graph of transaction latency vs throughput rate based on
    the CSV output generated by the [loadtime reporting
    tool](../../../test/loadtime/cmd/report/), for multiple experiments and configurations.

* [`prometheus_plotter.py`](./prometheus_plotter.py) is a Python script that uses
    [matplotlib] and [pandas] to plot graphs of several metrics from Prometheus.

## Setup

Execute the following within this directory (the same directory as the
`latency_throughput.py` file).

```bash
# Create a virtual environment into which to install your dependencies
python3 -m venv .venv

# Activate the virtual environment
source .venv/bin/activate

# Install dependencies listed in requirements.txt
pip install -r requirements.txt
```

## Latency vs Throughput Plotting
To show the instructions and parameter options, execute 

```bash
./latency_throughput.py --help
```

Example:

```bash
# Do the following while ensuring that the virtual environment is activated (see
# the Setup steps).
#
# This will generate a plot in a PNG file called 'tm034.png' in the current
# directory based on the reporting tool CSV output in the "raw.csv" file. The
# '-t' flag overrides the default title at the top of the plot.

./latency_throughput.py \
    -t 'CometBFT v0.34.x Latency vs Throughput' \
    ./tm034.png \
    /path/to/csv/files/raw.csv
```

## Latency vs Throughput Plotting (version 2)
Example:

```bash
# Do the following while ensuring that the virtual environment is activated (see
# the Setup steps).
#
# This will generate a series of plots in the `imgs` folder.
# Plots include combined experiment plots and experiments as subplots.
# - all_experiments - plots of all experiments as individual subplots.
# - all_configs - plots of all experiments, grouped by configuration (r,c).
# cXrY.png - Independent plot of experiments of configuration (c=X,r=Y) as different curves.
# cXrY_merged.png - Independent plot of experiments of configuration (c=X,r=Y) combined as single curve.
# e_ID.png - independent plot with just experiment with id ID as a single curve.

mkdir -p imgs
python3 latency_plotter.py /path/to/csv/files/raw.csv
```

## Prometheus metrics

1. Ensure that Prometheus is running locally and listening on port 9090. 
2. Tweak the script to your needs
   1. Adjust the time window
   2. Select the right fork 
   3. Select the right test case
   4. Tweak/add/remove metrics
3. Run the script as follows
   ```bash
   # Do the following while ensuring that the virtual environment is activated (see
   # the Setup steps).
   #
   # This will generate a series of plots in the folder `imgs` of the current folder.

   mkdir imgs
   python3 prometheus_plotter.py
   ```
4. Plots are saved in the `imgs` folder.

[matplotlib]: https://matplotlib.org/
[pandas]: https://pandas.pydata.org