Server

Gallery

Installation

See https://github.com/perfguru87/perftracker/blob/master/README.md

Features set

  • Tests
    • test scores, deviations, command line, description
    • number of errors, warnings
    • test start, end, duration
    • link to Grafana, custom HTTP links
  •  Jobs
    • REST API for Job results upload
    • UI for manual job results download/upload
    • Job details – summary, tests list and tests details
  • Comparisons
    • correct results comparison (considering deviations)
    • line and bar charts, tables
    • note: comparisons are manual only at the moment
  • Regression tracking
    • automatic build-to-build comparison according to job regression tag
    • automatic recognition of test cases with:
      • degradations > 50%
      • degradation > 10%
      • improvement > 10%
      • improvement > 50%
  • Artifacts management
    • a test or a job may have custom artifacts such as logs or data uploaded to the server
  • Hosts management
    • Hosts list
    • Hosts details – os, title, ip, hostname, num of CPUs, RAM size, HDD details, Network details, etc
    • Host locking (manual or through API) with customizable lock types

Sources

https://github.com/perfguru87/perftracker

Release Notes

  • 0.4.4 Dec
    • Features:
      • comparison view – add test description when clicking on test details
      • comparison view – improve chart line smoothness
      • job upload – test and job uuids now are optional
      • job upload – skip empty tests on job results upload
      • hw env details – show cpus_topology property, e.g (2S x 4C x 2T) means 2 sockets, 4 cores, 2 hyper-threads
    • Bugfixes:
      • fix comparison charts
      • comparison view – fix group section collapse/expand
  • 0.4.3 Nov
    • Features:
      • added test endpoint @ /test_endpoint/ to explain the performance CI workflow example in the documentation
      • add the ‘Project’ column in regression/comparison/job tables in ‘All projects’ mode
    • Bugfixes:
      • fix comparisons, jobs and regressions list in the ‘All projects’ mode
      • jobs view – fix the link to the job details which was recently broken
      • models/test.py – increase storage for the ‘scores’ string of a test
      • regression details – fix links to the jobs
      • models/comparison.py – fix DivisionByZero in case of an average score is zero
      • comparisons view – fix the link to comparisons
      • comparison view – fix an issue on the chart when job A has test1 score and job B has only test2 score
  • 0.4.2 
    • Features:
      • Hardware management – implement search hw nodes by node name, os, hostname, purpose and model
      • Regressions – show the total number of jobs in the regression in the header
      • Jobs upload – allow to upload a job w/o the tests, useful for just started jobs
    • Bugfixes:
      • fix manual job results upload
      • fix artifacts uploading and download
      • fix node locks management in the admin panel
      • fix job json upload for jobs with hardware nodes info
      • make the ‘purpose’ field optional when edit/create it in the admin panel
  • 0.4.1 
    • Hardware management:
      • feature: show lock edit dialog when clicking on the lock in the locks timeline
      • feature: hardware nodes now have the ‘purpose’ column
  • 0.4.0 
    • Hardware management:
      • feature: added custom lock types (name, background, foreground)
      • feature: align lock name to the left side of the viewport and make it always visible
      • bugfix: pre-select given node when creating a lock
      • bugfix: disable locks clustering
      • bugfix: fix the issue with multiple lock acquisition
      • bugfix: fix the link to hardware edit form
      • bugfix: fix an issue when Unlock button is disabled after a user opens the lock creation dialog and cancels it
    • Jobs comparison:
      • bugfix: fix jobs order in the comparison
    • Other:
      • bugfix: fix server start on Windows
      • bugifx: added missed psycopg2 requirement to be able to start perftracker on postgres
  • 0.3.1 
    • Jobs view:
      • feature: show the job artifacts on the job details panel
  • 0.3.0 
    • Artifacts management:
      • feature: new ‘artifacts’ management API: upload, download, update, info, list, delete, link/unlink a test/job
    • Results upload:
      • bugfix: do not allow to upload jobs with the duplicated combination of the ‘group’, ‘tag’ and ‘category’ attributes of the tests
    • Comparison view:
      • bugfix: fix test details view of the jobs with multiple identical tests (group, tag, category). Such suites and jobs were allow
        ed before Perftracker 3.0
    • Regressions view:
      • feature: added a link to the job on the regression page
      • feature: added ability to include/exclude jobs in regression
      • bugfix: determine ‘less_better’ regression attribute based on the last test in the last job
      • bugfix: skip jobs from another project (even if regression tag matches)
    • Tests view:
      • feature: show test group on the job tests list page
      • bugfix: fix test duration (sec) in the tests list
    • Jobs view:
      • feature: implement job removal
      • feature: show the product name and product version on the jobs list page
      • feature: move job link to the ‘id’ column, not the ‘Run title’ column (because a title can be empty)
      • bugfix: fix column titles
    • Hardware view:
      • bugfix: fix hardware nodes list for all projects (project_id=0)
      • bugfix: fix nodes selection when creating a new lock on the hardware farm page
      • bugfix: filter out the hosts which are locked already when trying to acquire a new lock
  • 0.2.2
    • fixed the job ‘append’ mode:
      • do not delete existing tests results if job already exists
      • delete prev tests results if a job was marked as ‘deleted’ before
  • 0.2.1 |
    • /api/v1.0/0/project – new API to fetch known projects
    • POST /api/v1.0/0/job – put the list of existing projects to the error message if project_name in json doesn’t exist
  • 0.2.0 |
    • added UI for job results download/upload
  • 0.1.0 |
    • Let it be the initial version…

ToDo

  • Settings management
    • which tabs to show
    • host locking on/off
    • custom tabs/pages
    • default columns
  • Authentication / Authorization
    • sign in form
    • do not allow to release foreign hardware locks
    • hide jobs/reports from anonymous users
  • Jobs list / details
    • Highlight the jobs in progress
    • Highlight the test in progress
    • Download as XLS
    • Download as JSONdone
    • Upload buttondone
  • Comparisons
    • ‘Preview’ button
    • Comparison edit dialog
    • Comparison delete
    • Bugfix – ‘back’ button doesn’t work after creating the comparison
    • Better autocolors
    • Ability to quickly hide/show/change charts
    • Ability to quickly hide/show/change table
    • Table columns specifier (deviation, test time, etc)
  • Regressions
    • Automatically create/update comparison of 2 first vs 2 last jobs
    • Mark regression as ‘investigated
    • Email notifications about new degradations/improvements
    • JIRA items lifecycle on the regression chart
    • Detailed regression chart
    • A quick comparison of selected builds
    • Automatically create new regression if doesn’t exist
    • Exclude job runs from regressions
    • Exclude individual tests from regressions
  • Test results
    • custom blobs upload and view in test details
    • Error/warning messages
    • Percentiles charts
    • 3D charts (2 sets of categories)
  • Hosts view
    • Change SSH login message on a host when it is locked
    • Small CPU/RAM usage chart from Grafana (svg+polyline)
  • Custom links page
  • Custom pages
    • iframe
    • page rendered as some arbitrary command line output
  • JIRA page
    • Additional fields for JIRA items
    • JIRA items tree view