Skip to content

Post Processing: Add a timeseries report generator#354

Open
harriscr wants to merge 1 commit into
ceph:mainfrom
harriscr:ch_wip_time_series_plots
Open

Post Processing: Add a timeseries report generator#354
harriscr wants to merge 1 commit into
ceph:mainfrom
harriscr:ch_wip_time_series_plots

Conversation

@harriscr

@harriscr harriscr commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Purpose of Change

This PR implements a comprehensive time-series reporting capability for the CBT (Ceph Benchmarking Tool) post-processing framework. The implementation adds the ability to parse FIO time-series data, generate time-series plots (IOPS, bandwidth, and latency over time), and create reports in both GitHub markdown and PDF formats.

Key Features

  • FIO Time-Series Parser: Parses FIO time-series log data into a common intermediate format
  • Time-Series Plotters: Specialized plotters for IOPS, bandwidth, and latency metrics over time
  • Timestamp Alignment: Aligns timestamps across multiple data sources for accurate comparison
  • Report Generation: Creates time-series reports in markdown and PDF formats
  • Common Intermediate Format: Analogous to the existing hockey-stick curve format, enabling a single plotter to handle time-series data from multiple benchmark tools

Design Approach

The implementation follows existing design patterns in the codebase:

  • Uses axis plotter sub-classes for each plot type, allowing multiple data lines on a single plot
  • Extends the common intermediate format pattern previously implemented for hockey-stick curves
  • Maintains consistency with the existing post-processing architecture
  • Modular design enables future extension to other benchmark tools beyond FIO

Automated Testing

  • New Unit Tests: Added comprehensive unit tests covering all new functionality
  • Extended Coverage: Enhanced test coverage for modified existing files
  • Code Quality Checks: All changes pass:
    • Ruff (linting)
    • Pylint (static analysis)
    • Mypy (type checking)
    • Black (code formatting)

Test Files Added/Modified:

  • test_time_series_report_generator.py (418 lines)
  • test_time_series_plotter.py (536 lines)
  • test_time_series_output_formatter.py (248 lines)
  • test_time_series_formatter.py (388 lines)
  • test_fio_log_parser.py (294 lines)
  • test_timestamp_aligner.py (354 lines)
  • test_fio_benchmarks.py (947 lines)
  • test_run_result.py (462 lines)
  • Plus updates to 15+ existing test files

Manual Testing

A time series report was generated from a CBT RBD FIO benchmark run:
timeseries_performance_report_260602_141547.pdf

A regression test was also run, producing a comparison report and a simple report to verify the new code still produces identical reports

Additional Notes

  • IBM Bob v1.02 and v1.03 were used to assist with these changes
  • New tool added: tools/generate_timeseries_performance_report.py for easy report generation
  • Updated requirements.txt with necessary dependencies
  • Comprehensive documentation maintained throughout the codebase

Related Documentation

  • Implementation follows patterns described in existing post_processing README files
  • New parsers documented in post_processing/parsers/ directory
  • Plotter usage documented in post_processing/plotter/README.md

@harriscr harriscr self-assigned this Jun 2, 2026
@harriscr harriscr force-pushed the ch_wip_time_series_plots branch from b03d758 to 9932aa9 Compare June 4, 2026 10:23
Implements the code to:
     - Parse FIO time-series data into a common intermediate format for time-series data.
     -  Generate a plot of the time series data:
         - IOPS over time
         - bandwidth over time
         - latency over time
     - create a timeseries report in github markdown and pdf format

A common intermediate format for time-series data is analogous to the common intemediate format for hockey-stick curves prevously implemented. It allows for a single plotter to be able to handle time-series data from sevear benchmark tools.

The code uses the existing design patterns, with different axis plotter sub-classes for each plot. This allows for adding multiple data lines to a single plot if required.

Additional unit tests have been added to cover the changed code, and to extend coverage for files that were touched

Ruff, pylint, mypy and black checks all pass on the changed files

IBM Bob v1.02 and v1.03 was used to help with these changes.

Signed-off-by: Chris Harris <harriscr@uk.ibm.com>
@harriscr harriscr force-pushed the ch_wip_time_series_plots branch from 9932aa9 to 695ea4d Compare June 5, 2026 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant