Skip to content

visualization.md

Latest commit

 

History

History
507 lines (364 loc) · 14.5 KB

visualization.md

File metadata and controls

507 lines (364 loc) · 14.5 KB

Lambda Gopher Benchmark Visualization Guide

This guide provides detailed instructions for visualizing and analyzing benchmark results generated by the Lambda Gopher Benchmark platform.

Table of Contents

  1. Introduction
  2. Visualizer Tool Overview
  3. Basic Usage
  4. Output Formats
  5. Visualization Options
  6. Sample Visualizations
  7. Advanced Usage
  8. Programmatic Access
  9. Troubleshooting

Introduction

The Lambda Gopher Benchmark platform includes a powerful visualization tool designed to help you analyze and compare benchmark results across different database systems, operations, and configurations. The visualizer can generate various output formats including tables, charts, and raw data for further processing.

Visualizer Tool Overview

The visualizer tool is a Go application located at cmd/visualizer/main.go. It processes benchmark result files generated by the benchmark runner and produces various visualizations to help you understand the performance characteristics of the databases under test.

Key capabilities:

  • Compare performance across different database systems
  • Analyze latency, throughput, and error rates
  • Generate visual representations of benchmark results
  • Export results in various formats for further analysis

Basic Usage

Visualizing a Single Result File

go run cmd/visualizer/main.go \
  --input results/result_20240601_120000.json \
  --output visualizations

Comparing Multiple Result Files

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations/comparison

Running Sample Visualizations

The platform includes sample visualizations that you can run to see the capabilities of the visualizer:

# For Linux/macOS
./examples/run_sample_visualization.sh

# For Windows
.\examples\run_sample_visualization.ps1

Output Formats

The visualizer can generate output in several formats:

HTML Format

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --format html

HTML output provides interactive charts and tables that allow you to:

  • Toggle between different metrics
  • Sort data by various criteria
  • Zoom in on specific parts of charts
  • Export data to CSV

CSV Format

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --format csv

CSV output is useful for importing data into spreadsheet applications or other data analysis tools.

PNG Format

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --format png

PNG output generates static chart images suitable for inclusion in reports or presentations.

JSON Format

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --format json

JSON output provides structured data for programmatic processing or custom visualization.

Visualization Options

Filtering by Database Type

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --databases dynamodb,immudb

This command generates visualizations only for DynamoDB and ImmuDB results, excluding other database types.

Filtering by Operation Type

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --operations write,read

This command generates visualizations only for write and read operations, excluding other operation types.

Focusing on Specific Metrics

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --metrics latency,throughput

This command generates visualizations focusing on latency and throughput metrics.

Grouping Results

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --group-by database

This command groups results by database type, making it easier to compare performance across different databases.

Other grouping options include:

  • --group-by operation: Group by operation type
  • --group-by dataSize: Group by data size
  • --group-by concurrency: Group by concurrency level

Sample Visualizations

The platform includes sample result files that demonstrate the expected format and can be used to test the visualization capabilities:

Database Comparison Chart

This visualization compares the performance of different database systems across various operations:

go run cmd/visualizer/main.go \
  --input examples/sample_results/comparison_results.json \
  --output visualizations \
  --format png

Operation Performance Chart

This visualization shows the performance of different operations for a specific database:

go run cmd/visualizer/main.go \
  --input examples/sample_results/dynamodb_results.json \
  --output visualizations \
  --group-by operation \
  --format png

Latency Distribution Chart

This visualization shows the distribution of latency values for different operations:

go run cmd/visualizer/main.go \
  --input examples/sample_results/latency_distribution.json \
  --output visualizations \
  --metrics latency \
  --format png

Advanced Usage

Custom Chart Titles and Labels

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --title "DynamoDB vs ImmuDB Performance Comparison" \
  --x-label "Operation Type" \
  --y-label "Latency (ms)"

Setting Chart Dimensions

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --width 1200 \
  --height 800

Generating Reports

go run cmd/visualizer/main.go \
  --input-dir results \
  --output visualizations \
  --generate-report

This command generates a comprehensive report including tables, charts, and analysis of the benchmark results.

Programmatic Access

The visualizer can also be used programmatically in your Go applications:

package main

import (
    "fmt"
    "github.com/yourusername/lambda-gopher-benchmark/pkg/visualizer"
)

func main() {
    // Create a new visualizer
    v := visualizer.New()
    
    // Load benchmark results
    err := v.LoadFromFile("results/result_20240601_120000.json")
    if err != nil {
        fmt.Printf("Error loading benchmark results: %v\n", err)
        return
    }
    
    // Generate visualizations
    err = v.GenerateCharts("visualizations", visualizer.FormatPNG)
    if err != nil {
        fmt.Printf("Error generating visualizations: %v\n", err)
        return
    }
    
    fmt.Println("Visualizations generated successfully!")
}

Troubleshooting

Missing Dependencies

If you encounter errors related to missing dependencies, make sure you have installed all required Go packages:

go get github.com/olekukonko/tablewriter
go get github.com/wcharczuk/go-chart/v2

Invalid Result Format

If the visualizer fails to process a result file, check that the file follows the expected format:

{
  "id": "benchmark_id",
  "timestamp": "2024-06-01T12:00:00Z",
  "config": { ... },
  "results": [
    {
      "database": "dynamodb",
      "operation": "write",
      "metrics": {
        "latency": {
          "min": 10.5,
          "max": 100.2,
          "avg": 45.6,
          "p50": 42.1,
          "p90": 75.3,
          "p99": 95.7
        },
        "throughput": 156.2,
        "errorRate": 0.01
      },
      "parameters": { ... }
    },
    ...
  ]
}

No Output Generated

If no output is generated, check that:

  1. The input file exists and is readable
  2. The output directory exists and is writable
  3. The benchmark result file contains valid data
# Check if the input file exists
ls -l results/result_20240601_120000.json

# Create the output directory if it doesn't exist
mkdir -p visualizations

# Validate the JSON format of the result file
jq . results/result_20240601_120000.json

Low-Quality Charts

If the generated charts have low quality or are difficult to read:

  1. Try increasing the dimensions with --width and --height flags
  2. Use a different format like SVG for better scalability
  3. Reduce the number of data points being visualized by filtering or grouping

Visualization Guide

The Lambda Gopher Benchmark platform includes a powerful visualizer component that helps you analyze and compare benchmark results across different databases and operations.

Overview

The visualizer reads benchmark result files and generates various outputs:

  • Text Reports: Tables showing key metrics in console output
  • CSV Files: Detailed results for further analysis in spreadsheets
  • Charts: Visual comparison of database performance
  • Markdown Reports: Easy-to-share documentation of results

Running the Visualizer

The visualizer is a command-line tool that can be run as follows:

go run cmd/visualizer/main.go --input <input_path> --output <output_directory>

Command Line Options

Option Description Default
--input Path to benchmark results directory or specific result file -
--output Directory to store visualization outputs "visualizations"
--format Output format (text, csv, chart, all) "all"
--group-by Group results by database or operation "database"
--metric Metric to visualize (throughput, latency) "throughput"
--databases Comma-separated list of databases to include All
--operations Comma-separated list of operations to include All
--start-date Start date filter (YYYY-MM-DD) -
--end-date End date filter (YYYY-MM-DD) -

Visualization Formats

Text Reports

Text reports display a table with key metrics directly in the console and save them to text files:

| Database  | Operation | Avg Latency (ms) | Throughput (ops/s) |
|-----------|-----------|------------------|-------------------|
| DynamoDB  | Write     | 12.45            | 80.32             |
| ImmuDB    | Write     | 8.76             | 114.15            |
| Timestream| Write     | 15.32            | 65.27             |

The text reports are saved to the output directory as summary_<groupBy>_<metricType>.txt.

CSV Files

CSV files provide detailed data for further analysis in spreadsheet applications. They include all available metrics and can be easily imported into tools like Excel or Google Sheets.

The CSV file is saved as benchmark_results.csv in the output directory.

Charts

The visualizer generates comparative bar charts showing performance metrics across different databases and operations:

  • Database Comparison: Shows how different databases perform for the same operation
  • Operation Comparison: Shows how different operations perform on the same database

Charts are saved as PNG files in the output directory, with filenames like database_comparison_chart.png and operation_comparison_chart.png.

Filtering and Comparing Results

The visualizer provides several ways to filter and compare benchmark results:

Database and Operation Filtering

# Compare only DynamoDB and ImmuDB
go run cmd/visualizer/main.go --input results --output visualizations --databases "dynamodb,immudb"

# Compare only read and write operations
go run cmd/visualizer/main.go --input results --output visualizations --operations "read,write"

Metric Selection

# Focus on latency metrics
go run cmd/visualizer/main.go --input results --output visualizations --metric "latency"

# Focus on throughput metrics
go run cmd/visualizer/main.go --input results --output visualizations --metric "throughput"

Grouping Results

# Group results by database type
go run cmd/visualizer/main.go --input results --output visualizations --group-by "database"

# Group results by operation type
go run cmd/visualizer/main.go --input results --output visualizations --group-by "operation"

Date Filtering

# Filter results from a specific date range
go run cmd/visualizer/main.go --input results --output visualizations --start-date "2024-06-01" --end-date "2024-06-15"

Visualization Best Practices

  1. Use standardized metrics: Make sure all benchmarks use the same configuration parameters for fair comparison
  2. Include all database types: Always include all databases in your comparison for a comprehensive view
  3. Focus on relevant operations: Filter operations to match your application's actual usage patterns
  4. Generate all formats: Use --format all to generate all visualization types for comprehensive analysis
  5. Use consistent naming: Use descriptive test names in your benchmark configurations for clearer visualization

Examples

Comparing Database Write Performance

go run cmd/visualizer/main.go --input results --output visualizations --operations "write" --metric "throughput" --group-by "database"

Comparing DynamoDB Operations

go run cmd/visualizer/main.go --input results --output visualizations --databases "dynamodb" --group-by "operation"

Generating a Report for a Presentation

# Generate charts and text reports for a presentation
go run cmd/visualizer/main.go --input results --output presentation-data --format "chart,text" --group-by "database"

Troubleshooting

Missing Results

If your visualization is missing expected results:

  1. Check that the benchmark completed successfully and generated result files
  2. Verify that the result files are in the directory specified by --input
  3. Check your filter parameters (databases, operations, dates) to ensure they match your result data

Chart Generation Issues

If charts aren't generating correctly:

  1. Ensure you have the required Go packages installed: go get github.com/wcharczuk/go-chart/v2
  2. Check that your output directory is writable
  3. Verify that your result files contain valid data with numeric metrics

Custom Visualization

To create custom visualizations, you can:

  1. Use the CSV output with your preferred visualization tool
  2. Modify the cmd/visualizer/main.go code to create custom charts or output formats

Next Steps

After visualizing your results:

  1. Analyze performance patterns: Look for trends across different databases and operations
  2. Identify bottlenecks: Focus on operations with the highest latency or lowest throughput
  3. Refine benchmarks: Update your benchmark configurations based on initial findings
  4. Iterate: Run additional benchmarks with adjusted parameters to explore performance further