About - DPDK Performance Test Lab
About the DPDK Community Lab
Objective and Scope of the Project
The DPDK Community Lab is an open, independent testing resource for the DPDK
project. Its purpose is to perform automated testing which ensures that the
performance and quality of DPDK is maintained.
Hardware for the lab is provided by equipment vendors and is used to run a
series of performance tests on new patch sets. The results of these tests are
published on the results dashboard which is publicly accessible, and are
also integrated into the DPDK instance of Patchwork. The Community Lab does not
publish raw performance numbers for vendor hardware, it only publishes
deltas (% change) to flag bad DPDK patches. The results will be used by the project's
maintainers to help determine whether new patch sets should be accepted into the
project.
The initial goal of the DPDK Community Lab is:
Identify any regression in DPDK performance. The DPDK Community
Lab will host equipment provided by multiple Members to UNH-IOL and run
basic performance regression tests on new patches/patch sets on an automated basis.
The aim of these tests is to determine if there has been any unexpected
drop in DPDK performance as a result of recent changes.
In future, the following additional items may be added to the scope of the lab:
Identify any regression in the performance of DPDK-enabled applications.
DPDK Members will be able to submit software applications to be run in the
DPDK Community Lab. The details of how this can be managed may be
complex, so this capability may be added at a later stage.
Demonstrate any new feature performance of DPDK. In each release, there may
be some new performance optimizations or some new solutions. DPDK Members
will be able to utilize the platforms in the performance test lab to show
the new performance gains through DPDK-enabled applications.
The DPDK Community Lab may also be used as a training or demo lab for DPDK events.
DPDK Performance Test Lab Usage Guidelines
Data generated by the DPDK Performance Test Lab shall not be used for
competitive marketing purposes between DPDK Member equipment providers.
Data from the test lab may be used to demonstrate and market the
performance improvement of new DPDK features in a DPDK Member agnostic manner.
About the Dashboard
There are two main components to the Dashboard; the Results Database and
the Dashboard itself. All results are uploaded via the Results Database
REST API. The Dashboard also uses the REST API to populate its pages. The
REST API can only be accessed by participating members. Meta information,
such as the series title and patch submitter, utilize the Patchwork REST API.
Possible Patch Statuses
Apply Error
The patch series could not be applied
Build Error
The patch series could not be built
Downstream Failure
A downstream project could not be built.
Downstream Indeterminate
A downstream project's result could not be determined and could mean that the test did not run to completion
Indeterminate
A result could not be determined and could mean that the test did not run to completion
Not Applicable
The patch series is not active in Patchwork or is not required to be tested
Pass
All test results were within the tolerance threshold from the expected result
Pending
There are pending tests for this patch series
Possible Regression
At least one test result was below the tolerance threshold from the expected result
Waiting
A tarball has been generated but no test results are available yet
Test Cases
nic_single_core_perf
ovs_perf
dpdk_compile
dpdk_compile_ovs
dpdk_compile_spdk
dpdk_unit_test
dpdk_meson_compile
spdk_unit_test
dpdk_mingw64_compile
Legacy_DTS_mtu_update
legacy_DTS_stats_checks
Legacy_DTS_dynamic_config
dynamic_queue
legacy_dts_mac_filter
legacy_dts_queue_start_stop
legacy_dts_scatter
unit_tests_mbuf
abi_test
cppcheck
spell_check
lpm_autotest
infrastructure_failure
static_analysis
rte_flow
virtio_smoke
vhost_virtio_user_interrupt
dpdk_kmod_compile_torvalds_linux
cryptodev_sw_snow3g_autotest
cryptodev_sw_zuc_autotest
dpdk_fips_validation
generate_coverage_reports
compressdev_zlib_pmd
crypto_perf_cryptodev_perf
dpdk_msvc_compile
cmdline
coremask
l2fwd
rxtx_callbacks
tso
ipv4_reassembly
dpdk_llvm_clang_compile
scatter (new DTS)
dts-check-format
blocklist
checksum_offload
dual_vlan
mac_filter
dynamic_config
dynamic_queue_conf
hello_world
l2_fwd
mtu
pmd_buffer_scatter
port_restart_config_persistency
promisc_support
queue_start_stop
smoke_tests
softnic
uni_pkt
vlan
packet_capture
port_control
port_stats
rx_tx_offload
pmd_rss
rss
buffer_scatter
flow_offload
qinq
virtio_fwd
single_core_forward_perf
cryptodev_throughput
Patch Details
The Dashboard calls groups of related patches a Patch Set. This is equivalent
to Patchwork's Series. In the Results Database, the model mostly contains
build information and result summaries about the series. It also contains a
link to the series itself to allow for grabbing meta information about the
series.
Times for patches and tests are displayed in UTC time.
Relative performance is measured by comparing the actual result with the
baseline result. The baseline itself is saved on the test machine; only
the delta results are submitted. The database has the capability of saving the
baseline in order to calculate an absolute result, but that must be
explicitly requested by the participating member.
Fluctuation in these results are expected, hence a tolerance is set to
determine if the result passes or fails.
DPDK Community Lab Policy Guidelines
Latest Policy Document
Please note that some of the text has been taken from the DPDK Community
Policies and Procedures document. If you would like to make modifications
to the text, please provide suggestions via the document.
Getting Involved
If an existing DPDK member is interested in providing hardware or software,
contact the DPDK Community Lab CI at
dpdklab@iol.unh.edu
Priority support and installation is provided based on membership status in the DPDK project.
If you're interested in becoming a
DPDK member
visit the
DPDK website
to join.
Send bug reports or feature requests to the
DPDK issue tracker
For general CI questions and comments, contact the DPDK CI mailing list at
ci@dpdk.org
Requesting a Patch Retest
If CI reports a failure for a test label on Patchwork, it is possible to request a retest of a patch or series on those label(s).
For a list of Patchwork testing labels, send an email reply on the patch or cover letter like so:
Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot
Note that you may omit any labels which are not failing.
This is currently supported by the UNH-IOL Community Lab, and the GitHub Actions robot, so just those Patchwork testing labels beginning wih “iol-” or “github-robot”.
Need Help?
Visit our
Bugzilla
or our
Slack workspace
with any questions about infrastructure.
Copyright © 2024
DPDK Project
Hosted by the
University of New Hampshire InterOperability Laboratory