Skip to content

Update system-tests to 590162e1f5223cbbcd2876fdb820837cb13a85e1#8515

Merged
mhlidd merged 1 commit intomasterfrom
mhlidd/system-tests-update
Mar 6, 2025
Merged

Update system-tests to 590162e1f5223cbbcd2876fdb820837cb13a85e1#8515
mhlidd merged 1 commit intomasterfrom
mhlidd/system-tests-update

Conversation

@mhlidd
Copy link
Contributor

@mhlidd mhlidd commented Mar 5, 2025 •
edited
Loading

What Does This Do

Unblock CI for previously failing system-tests for the following PR. Fixed by the following system-tests PR.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mhlidd mhlidd requested a review from a team as a code owner March 5, 2025 21:16
@mhlidd mhlidd requested review from PerfectSlayer and mcculls March 5, 2025 21:16
@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@mhlidd mhlidd added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 5, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 5, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/system-tests-update
git_commit_date 1741211198 1741209307
git_commit_sha 516cb80 3dca1a6
release_version 1.48.0-SNAPSHOT~516cb807cc 1.48.0-SNAPSHOT~3dca1a6518
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741213560 1741213560
ci_job_id 834729426 834729426
ci_pipeline_id 57858871 57858871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-upmzzdd-project-304-concurrent-0-te0cylwa 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-upmzzdd-project-304-concurrent-0-te0cylwa 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.037 s) : 0, 1036568
Total [baseline] (10.501 s) : 0, 10500649
Agent [candidate] (1.041 s) : 0, 1040568
Total [candidate] (10.466 s) : 0, 10466142
section appsec
Agent [baseline] (1.188 s) : 0, 1187593
Total [baseline] (10.754 s) : 0, 10753639
Agent [candidate] (1.187 s) : 0, 1187072
Total [candidate] (10.742 s) : 0, 10741526
section iast
Agent [baseline] (1.178 s) : 0, 1178392
Total [baseline] (10.977 s) : 0, 10977242
Agent [candidate] (1.171 s) : 0, 1171395
Total [candidate] (10.928 s) : 0, 10928089
section profiling
Agent [baseline] (1.268 s) : 0, 1268122
Total [baseline] (10.893 s) : 0, 10892550
Agent [candidate] (1.259 s) : 0, 1258743
Total [candidate] (10.899 s) : 0, 10898627
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.037 s -
Agent appsec 1.188 s 151.025 ms (14.6%)
Agent iast 1.178 s 141.824 ms (13.7%)
Agent profiling 1.268 s 231.554 ms (22.3%)
Total tracing 10.501 s -
Total appsec 10.754 s 252.99 ms (2.4%)
Total iast 10.977 s 476.593 ms (4.5%)
Total profiling 10.893 s 391.901 ms (3.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent appsec 1.187 s 146.504 ms (14.1%)
Agent iast 1.171 s 130.828 ms (12.6%)
Agent profiling 1.259 s 218.175 ms (21.0%)
Total tracing 10.466 s -
Total appsec 10.742 s 275.384 ms (2.6%)
Total iast 10.928 s 461.947 ms (4.4%)
Total profiling 10.899 s 432.484 ms (4.1%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.915 ms) : 0, 714915
BytebuddyAgent [candidate] (717.612 ms) : 0, 717612
GlobalTracer [baseline] (238.94 ms) : 0, 238940
GlobalTracer [candidate] (239.167 ms) : 0, 239167
AppSec [baseline] (55.044 ms) : 0, 55044
AppSec [candidate] (55.293 ms) : 0, 55293
Remote Config [baseline] (685.033 µs) : 0, 685
Remote Config [candidate] (687.787 µs) : 0, 688
Telemetry [baseline] (12.118 ms) : 0, 12118
Telemetry [candidate] (12.955 ms) : 0, 12955
section appsec
BytebuddyAgent [baseline] (738.284 ms) : 0, 738284
BytebuddyAgent [candidate] (738.009 ms) : 0, 738009
GlobalTracer [baseline] (237.425 ms) : 0, 237425
GlobalTracer [candidate] (237.41 ms) : 0, 237410
AppSec [baseline] (176.535 ms) : 0, 176535
AppSec [candidate] (177.007 ms) : 0, 177007
Remote Config [baseline] (660.724 µs) : 0, 661
Remote Config [candidate] (662.929 µs) : 0, 663
Telemetry [baseline] (8.687 ms) : 0, 8687
Telemetry [candidate] (8.336 ms) : 0, 8336
IAST [baseline] (21.983 ms) : 0, 21983
IAST [candidate] (21.414 ms) : 0, 21414
section iast
BytebuddyAgent [baseline] (842.581 ms) : 0, 842581
BytebuddyAgent [candidate] (837.147 ms) : 0, 837147
GlobalTracer [baseline] (231.317 ms) : 0, 231317
GlobalTracer [candidate] (230.81 ms) : 0, 230810
AppSec [baseline] (57.07 ms) : 0, 57070
AppSec [candidate] (56.524 ms) : 0, 56524
Remote Config [baseline] (623.219 µs) : 0, 623
Remote Config [candidate] (606.053 µs) : 0, 606
Telemetry [baseline] (8.735 ms) : 0, 8735
Telemetry [candidate] (8.647 ms) : 0, 8647
IAST [baseline] (22.876 ms) : 0, 22876
IAST [candidate] (22.733 ms) : 0, 22733
section profiling
BytebuddyAgent [baseline] (713.931 ms) : 0, 713931
BytebuddyAgent [candidate] (709.916 ms) : 0, 709916
GlobalTracer [baseline] (350.996 ms) : 0, 350996
GlobalTracer [candidate] (349.309 ms) : 0, 349309
AppSec [baseline] (55.586 ms) : 0, 55586
AppSec [candidate] (53.98 ms) : 0, 53980
Remote Config [baseline] (669.397 µs) : 0, 669
Remote Config [candidate] (664.761 µs) : 0, 665
Telemetry [baseline] (8.972 ms) : 0, 8972
Telemetry [candidate] (8.852 ms) : 0, 8852
ProfilingAgent [baseline] (97.254 ms) : 0, 97254
ProfilingAgent [candidate] (95.761 ms) : 0, 95761
Profiling [baseline] (97.278 ms) : 0, 97278
Profiling [candidate] (95.785 ms) : 0, 95785
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.04 s) : 0, 1039584
Total [baseline] (8.664 s) : 0, 8664194
Agent [candidate] (1.041 s) : 0, 1041449
Total [candidate] (8.646 s) : 0, 8646330
section iast
Agent [baseline] (1.176 s) : 0, 1176367
Total [baseline] (9.233 s) : 0, 9232588
Agent [candidate] (1.175 s) : 0, 1174710
Total [candidate] (9.227 s) : 0, 9227285
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1171810
Total [baseline] (9.18 s) : 0, 9180265
Agent [candidate] (1.181 s) : 0, 1180687
Total [candidate] (9.22 s) : 0, 9220478
section iast_TELEMETRY_OFF
Agent [baseline] (1.167 s) : 0, 1167422
Total [baseline] (9.21 s) : 0, 9210126
Agent [candidate] (1.168 s) : 0, 1168360
Total [candidate] (9.236 s) : 0, 9235662
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.04 s -
Agent iast 1.176 s 136.783 ms (13.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 132.226 ms (12.7%)
Agent iast_TELEMETRY_OFF 1.167 s 127.838 ms (12.3%)
Total tracing 8.664 s -
Total iast 9.233 s 568.394 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.18 s 516.07 ms (6.0%)
Total iast_TELEMETRY_OFF 9.21 s 545.931 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent iast 1.175 s 133.261 ms (12.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.181 s 139.238 ms (13.4%)
Agent iast_TELEMETRY_OFF 1.168 s 126.911 ms (12.2%)
Total tracing 8.646 s -
Total iast 9.227 s 580.955 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.22 s 574.148 ms (6.6%)
Total iast_TELEMETRY_OFF 9.236 s 589.331 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.935 ms) : 0, 717935
BytebuddyAgent [candidate] (718.85 ms) : 0, 718850
GlobalTracer [baseline] (239.492 ms) : 0, 239492
GlobalTracer [candidate] (239.557 ms) : 0, 239557
AppSec [baseline] (55.295 ms) : 0, 55295
AppSec [candidate] (55.359 ms) : 0, 55359
Remote Config [baseline] (693.018 µs) : 0, 693
Remote Config [candidate] (693.972 µs) : 0, 694
Telemetry [baseline] (11.316 ms) : 0, 11316
Telemetry [candidate] (12.113 ms) : 0, 12113
section iast
BytebuddyAgent [baseline] (841.55 ms) : 0, 841550
BytebuddyAgent [candidate] (839.417 ms) : 0, 839417
GlobalTracer [baseline] (230.861 ms) : 0, 230861
GlobalTracer [candidate] (231.045 ms) : 0, 231045
AppSec [baseline] (55.766 ms) : 0, 55766
AppSec [candidate] (56.864 ms) : 0, 56864
Remote Config [baseline] (618.202 µs) : 0, 618
Remote Config [candidate] (612.943 µs) : 0, 613
Telemetry [baseline] (8.686 ms) : 0, 8686
Telemetry [candidate] (8.849 ms) : 0, 8849
IAST [baseline] (23.849 ms) : 0, 23849
IAST [candidate] (22.994 ms) : 0, 22994
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (837.391 ms) : 0, 837391
BytebuddyAgent [candidate] (844.0 ms) : 0, 844000
GlobalTracer [baseline] (230.46 ms) : 0, 230460
GlobalTracer [candidate] (231.864 ms) : 0, 231864
AppSec [baseline] (56.888 ms) : 0, 56888
AppSec [candidate] (57.115 ms) : 0, 57115
Remote Config [baseline] (592.378 µs) : 0, 592
Remote Config [candidate] (618.893 µs) : 0, 619
Telemetry [baseline] (8.668 ms) : 0, 8668
Telemetry [candidate] (8.745 ms) : 0, 8745
IAST [baseline] (22.859 ms) : 0, 22859
IAST [candidate] (23.242 ms) : 0, 23242
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (833.939 ms) : 0, 833939
BytebuddyAgent [candidate] (834.862 ms) : 0, 834862
GlobalTracer [baseline] (230.028 ms) : 0, 230028
GlobalTracer [candidate] (230.445 ms) : 0, 230445
AppSec [baseline] (55.98 ms) : 0, 55980
AppSec [candidate] (56.545 ms) : 0, 56545
Remote Config [baseline] (614.686 µs) : 0, 615
Remote Config [candidate] (611.401 µs) : 0, 611
Telemetry [baseline] (8.664 ms) : 0, 8664
Telemetry [candidate] (8.54 ms) : 0, 8540
IAST [baseline] (23.356 ms) : 0, 23356
IAST [candidate] (22.42 ms) : 0, 22420
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-05T21:54:08 2025-03-05T22:01:56
git_branch master mhlidd/system-tests-update
git_commit_date 1741211198 1741209307
git_commit_sha 516cb80 3dca1a6
release_version 1.48.0-SNAPSHOT~516cb807cc 1.48.0-SNAPSHOT~3dca1a6518
start_time 2025-03-05T21:53:54 2025-03-05T22:01:42
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741212516 1741212516
ci_job_id 834729427 834729427
ci_pipeline_id 57858871 57858871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-mxhwuccu-project-304-concurrent-0-k5mwbujl 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-mxhwuccu-project-304-concurrent-0-k5mwbujl 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.373 ms) : 1353, 1392
.   : milestone, 1373,
appsec (1.744 ms) : 1719, 1768
.   : milestone, 1744,
appsec_no_iast (1.758 ms) : 1734, 1782
.   : milestone, 1758,
code_origins (1.711 ms) : 1679, 1743
.   : milestone, 1711,
iast (1.53 ms) : 1505, 1555
.   : milestone, 1530,
profiling (1.512 ms) : 1487, 1536
.   : milestone, 1512,
tracing (1.518 ms) : 1493, 1542
.   : milestone, 1518,
section candidate
no_agent (1.361 ms) : 1341, 1380
.   : milestone, 1361,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.746 ms) : 1721, 1770
.   : milestone, 1746,
code_origins (1.691 ms) : 1658, 1723
.   : milestone, 1691,
iast (1.529 ms) : 1505, 1553
.   : milestone, 1529,
profiling (1.546 ms) : 1523, 1569
.   : milestone, 1546,
tracing (1.485 ms) : 1460, 1511
.   : milestone, 1485,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.373 ms [1.353 ms, 1.392 ms] -
appsec 1.744 ms [1.719 ms, 1.768 ms] 371.161 µs (27.0%)
appsec_no_iast 1.758 ms [1.734 ms, 1.782 ms] 385.25 µs (28.1%)
code_origins 1.711 ms [1.679 ms, 1.743 ms] 338.275 µs (24.6%)
iast 1.53 ms [1.505 ms, 1.555 ms] 157.127 µs (11.4%)
profiling 1.512 ms [1.487 ms, 1.536 ms] 138.97 µs (10.1%)
tracing 1.518 ms [1.493 ms, 1.542 ms] 144.905 µs (10.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.361 ms [1.341 ms, 1.38 ms] -
appsec 1.746 ms [1.722 ms, 1.77 ms] 384.941 µs (28.3%)
appsec_no_iast 1.746 ms [1.721 ms, 1.77 ms] 384.713 µs (28.3%)
code_origins 1.691 ms [1.658 ms, 1.723 ms] 329.938 µs (24.2%)
iast 1.529 ms [1.505 ms, 1.553 ms] 168.006 µs (12.3%)
profiling 1.546 ms [1.523 ms, 1.569 ms] 185.116 µs (13.6%)
tracing 1.485 ms [1.46 ms, 1.511 ms] 124.51 µs (9.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.87 µs) : 363, 403
.   : milestone, 383,
iast (519.901 µs) : 498, 542
.   : milestone, 520,
iast_FULL (741.782 µs) : 720, 764
.   : milestone, 742,
iast_GLOBAL (567.037 µs) : 545, 589
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (528.236 µs) : 507, 550
.   : milestone, 528,
iast_INACTIVE (464.553 µs) : 443, 486
.   : milestone, 465,
iast_TELEMETRY_OFF (505.645 µs) : 483, 528
.   : milestone, 506,
tracing (461.113 µs) : 440, 482
.   : milestone, 461,
section candidate
no_agent (390.272 µs) : 370, 411
.   : milestone, 390,
iast (536.67 µs) : 515, 559
.   : milestone, 537,
iast_FULL (739.521 µs) : 718, 761
.   : milestone, 740,
iast_GLOBAL (560.708 µs) : 539, 582
.   : milestone, 561,
iast_HARDCODED_SECRET_DISABLED (519.812 µs) : 498, 542
.   : milestone, 520,
iast_INACTIVE (469.694 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (503.588 µs) : 481, 526
.   : milestone, 504,
tracing (462.809 µs) : 442, 484
.   : milestone, 463,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.87 µs [363.193 µs, 402.547 µs] -
iast 519.901 µs [498.005 µs, 541.797 µs] 137.031 µs (35.8%)
iast_FULL 741.782 µs [719.646 µs, 763.917 µs] 358.912 µs (93.7%)
iast_GLOBAL 567.037 µs [545.146 µs, 588.928 µs] 184.167 µs (48.1%)
iast_HARDCODED_SECRET_DISABLED 528.236 µs [506.612 µs, 549.86 µs] 145.366 µs (38.0%)
iast_INACTIVE 464.553 µs [443.232 µs, 485.874 µs] 81.683 µs (21.3%)
iast_TELEMETRY_OFF 505.645 µs [483.262 µs, 528.029 µs] 122.775 µs (32.1%)
tracing 461.113 µs [440.253 µs, 481.973 µs] 78.243 µs (20.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 390.272 µs [369.665 µs, 410.88 µs] -
iast 536.67 µs [514.836 µs, 558.505 µs] 146.398 µs (37.5%)
iast_FULL 739.521 µs [717.644 µs, 761.398 µs] 349.249 µs (89.5%)
iast_GLOBAL 560.708 µs [539.044 µs, 582.373 µs] 170.436 µs (43.7%)
iast_HARDCODED_SECRET_DISABLED 519.812 µs [497.966 µs, 541.658 µs] 129.54 µs (33.2%)
iast_INACTIVE 469.694 µs [448.403 µs, 490.984 µs] 79.421 µs (20.4%)
iast_TELEMETRY_OFF 503.588 µs [480.868 µs, 526.308 µs] 113.316 µs (29.0%)
tracing 462.809 µs [441.818 µs, 483.8 µs] 72.537 µs (18.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/system-tests-update
git_commit_date 1741211198 1741209307
git_commit_sha 516cb80 3dca1a6
release_version 1.48.0-SNAPSHOT~516cb807cc 1.48.0-SNAPSHOT~3dca1a6518
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741213110 1741213110
ci_job_id 834729428 834729428
ci_pipeline_id 57858871 57858871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-w3rafupo-project-304-concurrent-0-txxvcykd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-w3rafupo-project-304-concurrent-0-txxvcykd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.975 s) : 14975000, 14975000
.   : milestone, 14975000,
appsec (15.032 s) : 15032000, 15032000
.   : milestone, 15032000,
iast (18.482 s) : 18482000, 18482000
.   : milestone, 18482000,
iast_GLOBAL (18.049 s) : 18049000, 18049000
.   : milestone, 18049000,
profiling (15.792 s) : 15792000, 15792000
.   : milestone, 15792000,
tracing (14.862 s) : 14862000, 14862000
.   : milestone, 14862000,
section candidate
no_agent (15.388 s) : 15388000, 15388000
.   : milestone, 15388000,
appsec (14.835 s) : 14835000, 14835000
.   : milestone, 14835000,
iast (18.415 s) : 18415000, 18415000
.   : milestone, 18415000,
iast_GLOBAL (18.23 s) : 18230000, 18230000
.   : milestone, 18230000,
profiling (15.223 s) : 15223000, 15223000
.   : milestone, 15223000,
tracing (14.876 s) : 14876000, 14876000
.   : milestone, 14876000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.975 s [14.975 s, 14.975 s] -
appsec 15.032 s [15.032 s, 15.032 s] 57.0 ms (0.4%)
iast 18.482 s [18.482 s, 18.482 s] 3.507 s (23.4%)
iast_GLOBAL 18.049 s [18.049 s, 18.049 s] 3.074 s (20.5%)
profiling 15.792 s [15.792 s, 15.792 s] 817.0 ms (5.5%)
tracing 14.862 s [14.862 s, 14.862 s] -113.0 ms (-0.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.388 s [15.388 s, 15.388 s] -
appsec 14.835 s [14.835 s, 14.835 s] -553.0 ms (-3.6%)
iast 18.415 s [18.415 s, 18.415 s] 3.027 s (19.7%)
iast_GLOBAL 18.23 s [18.23 s, 18.23 s] 2.842 s (18.5%)
profiling 15.223 s [15.223 s, 15.223 s] -165.0 ms (-1.1%)
tracing 14.876 s [14.876 s, 14.876 s] -512.0 ms (-3.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~3dca1a6518, baseline=1.48.0-SNAPSHOT~516cb807cc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.346 ms) : 2302, 2390
.   : milestone, 2346,
iast (2.123 ms) : 2067, 2179
.   : milestone, 2123,
iast_GLOBAL (2.16 ms) : 2105, 2216
.   : milestone, 2160,
profiling (1.967 ms) : 1924, 2011
.   : milestone, 1967,
tracing (1.945 ms) : 1903, 1986
.   : milestone, 1945,
section candidate
no_agent (1.476 ms) : 1465, 1488
.   : milestone, 1476,
appsec (2.34 ms) : 2297, 2383
.   : milestone, 2340,
iast (2.12 ms) : 2065, 2176
.   : milestone, 2120,
iast_GLOBAL (2.17 ms) : 2115, 2226
.   : milestone, 2170,
profiling (2.402 ms) : 2237, 2568
.   : milestone, 2402,
tracing (1.95 ms) : 1908, 1992
.   : milestone, 1950,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.346 ms [2.302 ms, 2.39 ms] 872.718 µs (59.2%)
iast 2.123 ms [2.067 ms, 2.179 ms] 649.796 µs (44.1%)
iast_GLOBAL 2.16 ms [2.105 ms, 2.216 ms] 687.259 µs (46.7%)
profiling 1.967 ms [1.924 ms, 2.011 ms] 494.0 µs (33.5%)
tracing 1.945 ms [1.903 ms, 1.986 ms] 471.294 µs (32.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.465 ms, 1.488 ms] -
appsec 2.34 ms [2.297 ms, 2.383 ms] 863.772 µs (58.5%)
iast 2.12 ms [2.065 ms, 2.176 ms] 644.007 µs (43.6%)
iast_GLOBAL 2.17 ms [2.115 ms, 2.226 ms] 693.792 µs (47.0%)
profiling 2.402 ms [2.237 ms, 2.568 ms] 925.735 µs (62.7%)
tracing 1.95 ms [1.908 ms, 1.992 ms] 473.533 µs (32.1%)

PerfectSlayer
PerfectSlayer approved these changes Mar 6, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be nice to link the system test PR that fixes the issue 😉

smola
smola approved these changes Mar 6, 2025
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For next time, running .circleci/update_pinned_system_tests.sh will create a commit updating to the latest commit.

@smola smola changed the title Update System-tests Commit Hash Update system-tests to 590162e1f5223cbbcd2876fdb820837cb13a85e1 Mar 6, 2025
@mhlidd mhlidd added the type: enhancement Enhancements and improvements label Mar 6, 2025
@mhlidd mhlidd merged commit 71cbb06 into master Mar 6, 2025
213 of 215 checks passed
@mhlidd mhlidd deleted the mhlidd/system-tests-update branch March 6, 2025 14:25
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes

@mcculls mcculls mcculls approved these changes

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

1.48.0

Development

Successfully merging this pull request may close these issues.

4 participants

@mhlidd @smola @mcculls @PerfectSlayer