EAS Development for Mainline Linux

EAS development for mainline Linux takes place on LKML and Linaro eas-dev mailing lists where proposed patches are discussed with the community. Arm now provides a pre-integrated version of latest EAS patches for mainline, for upstream development use.

EAS mainline is a moving integration branch tracking tip sched/core, mainline release candidate or linux-pm testing and adding the latest versions of EAS related patch sets targeted at the mainline kernel. The patches may already be under discussion on relevant open source mailing lists or waiting here while their dependencies get resolved and merged upstream.

The integration branch comes as is with synthetic test case testing only. The intention is to have a single branch containing the full EAS picture for mainline Linux kernel developers interested in tracking progress. This EAS mainline integration branch is not suitable for product development. For product, please use the AOSP common kernel.

Source code

EAS mainline integration branches are made available regularly on:

        git://linux-arm.org/linux-power.git

The releases are tagged with the date e.g. -20190310 so old integrations are available for comparison

Latest: integration-20200925

Tree: eas/next/integration

Base: eas/next/integration-base

Features:

  1. Instrument sched domain flags (in master)
  2. CPPC FFH support using AMUs (v2 on lkml)
  3. Cpufreq FIE cleanup (in master)

Features in Linux mainline:

  1. v4.13 - CPU Invariant Engine (CIE)
  2. v4.15 - Frequency Invariant Engine (FIE)
  3. v4.17 - Idle CPU Per-Entity Load-Tracking (PELT) update
  4. v4.17 - Util(ization) Est(imated)
  5. v4.20 - Misfit task, i.e. forcing migration of running tasks that do not fit on the CPU they are currently running on
  6. v4.20 - Runtime scheduler domain flag detection
  7. v5.0   - Per-cpu Energy Model (EM) and Energy Aware Scheduling (EAS)
  8. v5.3   - Tracepoints (PELT and over-utilzation)
  9. v5.3   - Util(ization) clamping (core and per-task interface)
  10. v5.4   - Util(ization) clamping (cgroup interface)
  11. v5.4   - Patch-set 'sched/fair: Reduce complexity of energy calculation'
  12. v5.5   - Thermal/Cpu Cooling as Energy Model (EM) user
  13. v5.7   - Activity Monitor Unit (AMU) support
  14. v5.8   - Streamline select_task_rq() & select_task_rq_fair() (partly (4/9 patches))
  15. v5.9   - (Devfreq) devices as Energy Model user
  16. v5.9   - SCHED_DEADLINE capacity awareness

Platforms we test on:

  1. Arm Juno r0 (arm64)
  2. Arm Juno r2 (arm64)
  3. Hikey960 (arm64) (needs out-of-tree patches  (hikey960_mainline topic))
  4. DB845C (arm64) (needs out-of-tree patch (db845c_mainline topic))
  5. Arm TC2 (arm32)

Next work items:

  1. Pushing Features upstream
  2. 100% test pass rate on all platforms
  • Test Results (integration 20200925) 

    (1) juno-r0:

    testcase                                                                old%   new%  delta%       pvalue fix_iter# 
    -------------------------------------------------------------------------------------------------------------------
    Invariance:iter_invariance_items[cpu=0@575000]:test_load_correctness   39.2%   0.8%  -38.4%     8.07e-32           
    Invariance:iter_invariance_items[cpu=0@575000]:test_util_correctness   40.0%   0.8%  -39.2%     1.21e-32           
    Invariance:test_load_correctness                                       42.4%   4.0%  -38.4%     1.32e-26           
    Invariance:test_util_correctness                                       43.2%   4.0%  -39.2%     2.21e-27           
    OneTaskCPUMigration:test_util_task_migration                            0.4% 100.0%   99.6%    4.30e-147         1 

    test/eas_behaviour (LISA-test) in 0:08:39 [BAD]
        EnergyModelWakeMigration:test_dmesg:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_slack:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_task_placement: undecided 1/250   (0.4%)
        OneSmallTask:test_dmesg:                      passed    250/250 (100.0%)
        OneSmallTask:test_slack:                      passed    250/250 (100.0%)
        OneSmallTask:test_task_placement:             passed    250/250 (100.0%)
        RampDown:test_dmesg:                          passed    250/250 (100.0%)
        RampDown:test_slack:                          passed    250/250 (100.0%)
        RampDown:test_task_placement:                 passed    250/250 (100.0%)
        RampUp:test_dmesg:                            passed    250/250 (100.0%)
        RampUp:test_slack:                            passed    250/250 (100.0%)
        RampUp:test_task_placement:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_dmesg:                   FAILED    1/250   (0.4%)
        ThreeSmallTasks:test_slack:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_task_placement:          passed    250/250 (100.0%)
        TwoBigTasks:test_dmesg:                       FAILED    1/250   (0.4%)
        TwoBigTasks:test_slack:                       passed    250/250 (100.0%)
        TwoBigTasks:test_task_placement:              undecided 108/250 (43.2%)
        TwoBigThreeSmall:test_dmesg:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_slack:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_task_placement:         undecided 81/250  (32.4%)
        
        Error: 0/21, Failed: 2/21, Undecided: 3/21, Skipped: 0/21, Passed: 16/21

    test/load_tracking (LISA-test) in 0:17:12 [BAD]
        Invariance:iter_invariance_items[cpu=0@450000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@575000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@575000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@575000]:test_load_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@575000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@575000]:test_util_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@775000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@775000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@775000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@775000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@775000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@850000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@850000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@850000]:test_load_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@850000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@850000]:test_util_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=1@450000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@450000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@450000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@450000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@450000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@625000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@625000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@625000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@625000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@625000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_load_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_util_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=1@950000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@950000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@950000]:test_load_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=1@950000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@950000]:test_util_correctness:  FAILED 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_dmesg:            passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_load_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_load_correctness: passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_util_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_util_correctness: passed 250/250 (100.0%)
        Invariance:test_cpu_invariance:                                        passed 250/250 (100.0%)
        Invariance:test_freq_invariance:                                       passed 250/250 (100.0%)
        Invariance:test_load_behaviour:                                        passed 250/250 (100.0%)
        Invariance:test_load_correctness:                                      FAILED 10/250  (4.0%)
        Invariance:test_util_behaviour:                                        passed 250/250 (100.0%)
        Invariance:test_util_correctness:                                      FAILED 10/250  (4.0%)
        NTasksCPUMigration:test_dmesg:                                         passed 250/250 (100.0%)
        NTasksCPUMigration:test_util_task_migration:                           passed 250/250 (100.0%)
        OneTaskCPUMigration:test_dmesg:                                        passed 250/250 (100.0%)
        OneTaskCPUMigration:test_util_task_migration:                          FAILED 250/250 (100.0%)
        TwoTasksCPUMigration:test_dmesg:                                       passed 250/250 (100.0%)
        TwoTasksCPUMigration:test_util_task_migration:                         passed 250/250 (100.0%)
        
        Error: 0/62, Failed: 15/62, Undecided: 0/62, Skipped: 0/62, Passed: 47/62

    test/misfit (LISA-test) in 0:02:15 [BAD]
        StaggeredFinishes:test_dmesg:        FAILED 1/250   (0.4%)
        StaggeredFinishes:test_preempt_time: passed 250/250 (100.0%)
        StaggeredFinishes:test_throughput:   passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/util_tracking (LISA-test) in 0:02:35 [BAD]
        UtilConvergence:test_activations: passed 250/250 (100.0%)
        UtilConvergence:test_dmesg:       passed 250/250 (100.0%)
        UtilConvergence:test_means:       FAILED 1/250   (0.4%)
        
        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/other (LISA-test) in 0:04:34 [BAD]
        CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
        UserspaceSanity:test_dmesg:              FAILED 1/250   (0.4%)
        UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/hotplug (LISA-test) in 0:01:39 [GOOD]
        HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
        HotplugTorture:test_target_alive: passed 250/250 (100.0%)
        
        Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2

    (2) juno-r2:

    testcase                                                                old%   new%  delta%       pvalue fix_iter# 
    -------------------------------------------------------------------------------------------------------------------
    Invariance:iter_invariance_items[cpu=0@800000]:test_load_correctness    3.2%   0.0%   -3.2%     7.38e-03           
    Invariance:iter_invariance_items[cpu=0@800000]:test_util_correctness    3.2%   0.0%   -3.2%     7.38e-03           
    Invariance:iter_invariance_items[cpu=0@950000]:test_load_correctness   10.8%   1.6%   -9.2%     1.97e-05           
    Invariance:iter_invariance_items[cpu=0@950000]:test_util_correctness   10.4%   2.0%   -8.4%     1.22e-04           
    Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness    1.2%   4.0%    2.8%     8.81e-02        83 
    Invariance:test_load_correctness                                       16.0%   6.8%   -9.2%     1.76e-03           
    Invariance:test_util_correctness                                       14.8%   6.8%   -8.0%     5.77e-03           
    OneTaskCPUMigration:test_util_task_migration                            0.0%  98.4%   98.4%    2.90e-141         1 
    TwoBigThreeSmall:test_task_placement                                   40.2%  58.0%   17.8%     1.13e-04        17 

    test/eas_behaviour (LISA-test) in 0:08:24 [BAD]
        EnergyModelWakeMigration:test_dmesg:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_slack:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_task_placement: passed    250/250 (100.0%)
        OneSmallTask:test_dmesg:                      passed    250/250 (100.0%)
        OneSmallTask:test_slack:                      passed    250/250 (100.0%)
        OneSmallTask:test_task_placement:             undecided 8/250   (3.2%)
        RampDown:test_dmesg:                          passed    250/250 (100.0%)
        RampDown:test_slack:                          passed    250/250 (100.0%)
        RampDown:test_task_placement:                 passed    250/250 (100.0%)
        RampUp:test_dmesg:                            passed    250/250 (100.0%)
        RampUp:test_slack:                            passed    250/250 (100.0%)
        RampUp:test_task_placement:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_dmesg:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_slack:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_task_placement:          undecided 8/250   (3.2%)
        TwoBigTasks:test_dmesg:                       passed    250/250 (100.0%)
        TwoBigTasks:test_slack:                       passed    250/250 (100.0%)
        TwoBigTasks:test_task_placement:              undecided 12/250  (4.8%)
        TwoBigTasks:test_task_placement:              FAILED    1/250   (0.4%)
        TwoBigThreeSmall:test_dmesg:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_slack:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_task_placement:         undecided 12/250  (4.8%)
        TwoBigThreeSmall:test_task_placement:         FAILED    138/250 (55.2%)
        
        Error: 0/21, Failed: 2/21, Undecided: 4/21, Skipped: 0/21, Passed: 17/21

    test/load_tracking (LISA-test) in 0:12:25 [BAD]
        Invariance:iter_invariance_items[cpu=0@450000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_load_behaviour:    undecided 21/250  (8.4%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_load_correctness:  FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_util_behaviour:    undecided 21/250  (8.4%)
        Invariance:iter_invariance_items[cpu=0@450000]:test_util_correctness:  FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_load_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@950000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@950000]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@950000]:test_load_correctness:  FAILED    4/250   (1.6%)
        Invariance:iter_invariance_items[cpu=0@950000]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@950000]:test_util_correctness:  FAILED    5/250   (2.0%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_load_behaviour:    FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_load_correctness:  FAILED    11/250  (4.4%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_util_behaviour:    FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness:  FAILED    10/250  (4.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_load_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_util_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_load_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_load_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_util_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:test_cpu_invariance:                                        passed    250/250 (100.0%)
        Invariance:test_freq_invariance:                                       passed    250/250 (100.0%)
        Invariance:test_load_behaviour:                                        FAILED    1/250   (0.4%)
        Invariance:test_load_correctness:                                      FAILED    17/250  (6.8%)
        Invariance:test_util_behaviour:                                        FAILED    1/250   (0.4%)
        Invariance:test_util_correctness:                                      FAILED    17/250  (6.8%)
        NTasksCPUMigration:test_dmesg:                                         passed    250/250 (100.0%)
        NTasksCPUMigration:test_util_task_migration:                           passed    250/250 (100.0%)
        OneTaskCPUMigration:test_dmesg:                                        passed    250/250 (100.0%)
        OneTaskCPUMigration:test_util_task_migration:                          FAILED    246/250 (98.4%)
        TwoTasksCPUMigration:test_dmesg:                                       passed    250/250 (100.0%)
        TwoTasksCPUMigration:test_util_task_migration:                         passed    250/250 (100.0%)
        
        Error: 0/42, Failed: 16/42, Undecided: 4/42, Skipped: 0/42, Passed: 22/42

    test/misfit (LISA-test) in 0:02:19 [BAD]
        StaggeredFinishes:test_dmesg:        FAILED    8/250   (3.2%)
        StaggeredFinishes:test_preempt_time: passed    250/250 (100.0%)
        StaggeredFinishes:test_throughput:   undecided 7/250   (2.8%)
        
        Error: 0/3, Failed: 1/3, Undecided: 1/3, Skipped: 0/3, Passed: 1/3

    test/util_tracking (LISA-test) in 0:02:39 [BAD]
        UtilConvergence:test_activations: passed 250/250 (100.0%)
        UtilConvergence:test_dmesg:       passed 250/250 (100.0%)
        UtilConvergence:test_means:       FAILED 1/250   (0.4%)
        
        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/other (LISA-test) in 0:03:51 [GOOD]
        CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
        UserspaceSanity:test_dmesg:              passed 250/250 (100.0%)
        UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 0/3, Undecided: 0/3, Skipped: 0/3, Passed: 3/3

    test/hotplug (LISA-test) in 0:01:47 [GOOD]
        HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
        HotplugTorture:test_target_alive: passed 250/250 (100.0%)
        
        Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2

    (3) hikey960:

    testcase                                                                old%   new%  delta%       pvalue fix_iter# 
    -------------------------------------------------------------------------------------------------------------------
    OneTaskCPUMigration:test_util_task_migration                            0.0%  62.0%   62.0%     1.24e-62         3 

    test/eas_behaviour (LISA-test) in 0:09:24 [GOOD]
        EnergyModelWakeMigration:test_dmesg:          passed 250/250 (100.0%)
        EnergyModelWakeMigration:test_slack:          passed 250/250 (100.0%)
        EnergyModelWakeMigration:test_task_placement: passed 250/250 (100.0%)
        OneSmallTask:test_dmesg:                      passed 250/250 (100.0%)
        OneSmallTask:test_slack:                      passed 250/250 (100.0%)
        OneSmallTask:test_task_placement:             passed 250/250 (100.0%)
        RampDown:test_dmesg:                          passed 250/250 (100.0%)
        RampDown:test_slack:                          passed 250/250 (100.0%)
        RampDown:test_task_placement:                 passed 250/250 (100.0%)
        RampUp:test_dmesg:                            passed 250/250 (100.0%)
        RampUp:test_slack:                            passed 250/250 (100.0%)
        RampUp:test_task_placement:                   passed 250/250 (100.0%)
        ThreeSmallTasks:test_dmesg:                   passed 250/250 (100.0%)
        ThreeSmallTasks:test_slack:                   passed 250/250 (100.0%)
        ThreeSmallTasks:test_task_placement:          passed 250/250 (100.0%)
        TwoBigTasks:test_dmesg:                       passed 250/250 (100.0%)
        TwoBigTasks:test_slack:                       passed 250/250 (100.0%)
        TwoBigTasks:test_task_placement:              passed 250/250 (100.0%)
        TwoBigThreeSmall:test_dmesg:                  passed 250/250 (100.0%)
        TwoBigThreeSmall:test_slack:                  passed 250/250 (100.0%)
        TwoBigThreeSmall:test_task_placement:         passed 250/250 (100.0%)
        
        Error: 0/21, Failed: 0/21, Undecided: 0/21, Skipped: 0/21, Passed: 21/21

    test/load_tracking (LISA-test) in 0:19:29 [BAD]
        Invariance:iter_invariance_items[cpu=0@533000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@533000]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@533000]:test_load_correctness:  FAILED    6/250   (2.4%)
        Invariance:iter_invariance_items[cpu=0@533000]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@533000]:test_util_correctness:  FAILED    9/250   (3.6%)
        Invariance:iter_invariance_items[cpu=0@999000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@999000]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@999000]:test_load_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@999000]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@999000]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1402000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1402000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1402000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1402000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1402000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1709000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1709000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1709000]:test_load_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@1709000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1709000]:test_util_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@1844000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1844000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1844000]:test_load_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@1844000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1844000]:test_util_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@903000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@903000]:test_load_behaviour:    undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@903000]:test_load_correctness:  FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@903000]:test_util_behaviour:    undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@903000]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1421000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1421000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1421000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1421000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1421000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1805000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1805000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1805000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1805000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1805000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2112000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2112000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2112000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2112000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2112000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2362000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2362000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2362000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2362000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2362000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:test_cpu_invariance:                                        passed    250/250 (100.0%)
        Invariance:test_freq_invariance:                                       passed    250/250 (100.0%)
        Invariance:test_load_behaviour:                                        passed    250/250 (100.0%)
        Invariance:test_load_correctness:                                      FAILED    9/250   (3.6%)
        Invariance:test_util_behaviour:                                        passed    250/250 (100.0%)
        Invariance:test_util_correctness:                                      FAILED    11/250  (4.4%)
        NTasksCPUMigration:test_dmesg:                                         passed    250/250 (100.0%)
        NTasksCPUMigration:test_util_task_migration:                           passed    250/250 (100.0%)
        OneTaskCPUMigration:test_dmesg:                                        passed    250/250 (100.0%)
        OneTaskCPUMigration:test_util_task_migration:                          FAILED    155/250 (62.0%)
        TwoTasksCPUMigration:test_dmesg:                                       passed    250/250 (100.0%)
        TwoTasksCPUMigration:test_util_task_migration:                         passed    250/250 (100.0%)
        
        Error: 0/62, Failed: 10/62, Undecided: 2/62, Skipped: 0/62, Passed: 50/62

    test/misfit (LISA-test) in 0:02:47 [BAD]
        StaggeredFinishes:test_dmesg:        FAILED    250/250 (100.0%)
        StaggeredFinishes:test_preempt_time: passed    250/250 (100.0%)
        StaggeredFinishes:test_throughput:   undecided 129/250 (51.6%)
        
        Error: 0/3, Failed: 1/3, Undecided: 1/3, Skipped: 0/3, Passed: 1/3

    test/util_tracking (LISA-test) in 0:03:05 [GOOD]
        UtilConvergence:test_activations: passed 250/250 (100.0%)
        UtilConvergence:test_dmesg:       passed 250/250 (100.0%)
        UtilConvergence:test_means:       passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 0/3, Undecided: 0/3, Skipped: 0/3, Passed: 3/3

    test/other (LISA-test) in 0:05:58 [GOOD]
        CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
        UserspaceSanity:test_dmesg:              passed 250/250 (100.0%)
        UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 0/3, Undecided: 0/3, Skipped: 0/3, Passed: 3/3

    (4) TC2:

    testcase                                                                old%   new%  delta%       pvalue fix_iter# 
    -------------------------------------------------------------------------------------------------------------------
    Invariance:iter_invariance_items[cpu=1@1000000]:test_load_correctness  23.2%  80.4%   57.2%     3.32e-39         2 
    Invariance:iter_invariance_items[cpu=1@1000000]:test_util_correctness  23.2%  80.4%   57.2%     3.32e-39         2 
    Invariance:iter_invariance_items[cpu=1@1100000]:test_load_correctness  56.8%  82.4%   25.6%     5.88e-10         3 
    Invariance:iter_invariance_items[cpu=1@1100000]:test_util_correctness  56.8%  82.4%   25.6%     5.88e-10         3 
    Invariance:iter_invariance_items[cpu=1@1200000]:test_load_correctness  71.2%  78.8%    7.6%     6.28e-02       101 
    Invariance:iter_invariance_items[cpu=1@1200000]:test_util_correctness  71.2%  78.8%    7.6%     6.28e-02       101 
    Invariance:iter_invariance_items[cpu=1@700000]:test_load_correctness    2.0%  18.0%   16.0%     7.37e-10        15 
    Invariance:iter_invariance_items[cpu=1@700000]:test_util_correctness    1.6%  18.0%   16.4%     1.32e-10        15 
    Invariance:iter_invariance_items[cpu=1@800000]:test_load_correctness    2.8%  44.0%   41.2%     5.98e-31         5 
    Invariance:iter_invariance_items[cpu=1@800000]:test_util_correctness    2.8%  44.0%   41.2%     5.98e-31         5 
    Invariance:iter_invariance_items[cpu=1@900000]:test_load_correctness    6.8%  75.2%   68.4%     2.13e-60         2 
    Invariance:iter_invariance_items[cpu=1@900000]:test_util_correctness    6.8%  75.2%   68.4%     2.13e-60         2 
    Invariance:test_load_correctness                                       90.4%  99.6%    9.2%     9.09e-07         1 
    Invariance:test_util_correctness                                       90.4%  99.6%    9.2%     9.09e-07         1 
    StaggeredFinishes:test_throughput                                       5.2%   2.0%   -3.2%     9.03e-02           

    test/eas_behaviour (LISA-test) in 0:08:20 [BAD]
        EnergyModelWakeMigration:test_dmesg:          passed 249/249 (100.0%)
        EnergyModelWakeMigration:test_slack:          passed 249/249 (100.0%)
        EnergyModelWakeMigration:test_task_placement: passed 249/249 (100.0%)
        OneSmallTask:test_dmesg:                      passed 249/249 (100.0%)
        OneSmallTask:test_slack:                      passed 249/249 (100.0%)
        OneSmallTask:test_task_placement:             passed 249/249 (100.0%)
        RampDown:test_dmesg:                          passed 249/249 (100.0%)
        RampDown:test_slack:                          passed 249/249 (100.0%)
        RampDown:test_task_placement:                 passed 249/249 (100.0%)
        RampUp:test_dmesg:                            passed 249/249 (100.0%)
        RampUp:test_slack:                            passed 249/249 (100.0%)
        RampUp:test_task_placement:                   passed 249/249 (100.0%)
        ThreeSmallTasks:test_dmesg:                   passed 249/249 (100.0%)
        ThreeSmallTasks:test_slack:                   passed 249/249 (100.0%)
        ThreeSmallTasks:test_task_placement:          passed 249/249 (100.0%)
        TwoBigTasks:test_dmesg:                       passed 249/249 (100.0%)
        TwoBigTasks:test_slack:                       passed 249/249 (100.0%)
        TwoBigTasks:test_task_placement:              passed 249/249 (100.0%)
        TwoBigThreeSmall:test_dmesg:                  passed 249/249 (100.0%)
        TwoBigThreeSmall:test_slack:                  passed 249/249 (100.0%)
        TwoBigThreeSmall:test_task_placement:         FAILED 5/249   (2.0%)
        
        Error: 0/21, Failed: 1/21, Undecided: 0/21, Skipped: 0/21, Passed: 20/21

    test/load_tracking (LISA-test) in 0:26:32 [BAD]
        Invariance:iter_invariance_items[cpu=0@350000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@350000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@350000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@350000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@350000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@400000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@400000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@400000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@400000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@400000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@500000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@500000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@500000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@500000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@500000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@600000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@600000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@600000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@600000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@600000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@700000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@800000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=0@900000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@900000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@900000]:test_load_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@900000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@900000]:test_util_correctness:  passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1000000]:test_dmesg:            passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1000000]:test_load_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1000000]:test_load_correctness: FAILED 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=0@1000000]:test_util_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1000000]:test_util_correctness: FAILED 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=1@500000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@500000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@500000]:test_load_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@500000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@500000]:test_util_correctness:  FAILED 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_load_correctness:  FAILED 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness:  FAILED 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=1@700000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@700000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@700000]:test_load_correctness:  FAILED 45/250  (18.0%)
        Invariance:iter_invariance_items[cpu=1@700000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@700000]:test_util_correctness:  FAILED 45/250  (18.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_load_correctness:  FAILED 110/250 (44.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@800000]:test_util_correctness:  FAILED 110/250 (44.0%)
        Invariance:iter_invariance_items[cpu=1@900000]:test_dmesg:             passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@900000]:test_load_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@900000]:test_load_correctness:  FAILED 188/250 (75.2%)
        Invariance:iter_invariance_items[cpu=1@900000]:test_util_behaviour:    passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@900000]:test_util_correctness:  FAILED 188/250 (75.2%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_dmesg:            passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_load_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_load_correctness: FAILED 201/250 (80.4%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_util_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1000000]:test_util_correctness: FAILED 201/250 (80.4%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_dmesg:            passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_load_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_load_correctness: FAILED 206/250 (82.4%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_util_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1100000]:test_util_correctness: FAILED 206/250 (82.4%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_dmesg:            passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_load_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_load_correctness: FAILED 197/250 (78.8%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_util_behaviour:   passed 250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=1@1200000]:test_util_correctness: FAILED 197/250 (78.8%)
        Invariance:test_cpu_invariance:                                        passed 250/250 (100.0%)
        Invariance:test_freq_invariance:                                       passed 250/250 (100.0%)
        Invariance:test_load_behaviour:                                        passed 250/250 (100.0%)
        Invariance:test_load_correctness:                                      FAILED 249/250 (99.6%)
        Invariance:test_util_behaviour:                                        passed 250/250 (100.0%)
        Invariance:test_util_correctness:                                      FAILED 249/250 (99.6%)
        NTasksCPUMigration:test_dmesg:                                         passed 250/250 (100.0%)
        NTasksCPUMigration:test_util_task_migration:                           passed 250/250 (100.0%)
        OneTaskCPUMigration:test_dmesg:                                        passed 250/250 (100.0%)
        OneTaskCPUMigration:test_util_task_migration:                          passed 250/250 (100.0%)
        TwoTasksCPUMigration:test_dmesg:                                       passed 250/250 (100.0%)
        TwoTasksCPUMigration:test_util_task_migration:                         passed 250/250 (100.0%)
        
        Error: 0/92, Failed: 26/92, Undecided: 0/92, Skipped: 0/92, Passed: 66/92

    test/misfit (LISA-test) in 0:02:09 [BAD]
        StaggeredFinishes:test_dmesg:        passed 250/250 (100.0%)
        StaggeredFinishes:test_preempt_time: passed 250/250 (100.0%)
        StaggeredFinishes:test_throughput:   FAILED 5/250   (2.0%)
        
        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/util_tracking (LISA-test) in 0:02:29 [GOOD]
        UtilConvergence:test_activations: passed 250/250 (100.0%)
        UtilConvergence:test_dmesg:       passed 250/250 (100.0%)
        UtilConvergence:test_means:       passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 0/3, Undecided: 0/3, Skipped: 0/3, Passed: 3/3

    test/other (LISA-test) in 0:05:43 [GOOD]
        CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
        UserspaceSanity:test_dmesg:              passed 250/250 (100.0%)
        UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
        
        Error: 0/3, Failed: 0/3, Undecided: 0/3, Skipped: 0/3, Passed: 3/3

    (5) db845c:

    testcase                                                                old%   new%  delta%       pvalue fix_iter#
    -------------------------------------------------------------------------------------------------------------------
    OneTaskCPUMigration:test_util_task_migration                            0.0%  82.0%   82.0%     5.77e-96         2
    UtilConvergence:test_means                                             22.0%   0.8%  -21.2%     1.02e-15

    test/eas_behaviour (LISA-test) in 0:10:03 [BAD]
        EnergyModelWakeMigration:test_dmesg:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_slack:          passed    250/250 (100.0%)
        EnergyModelWakeMigration:test_task_placement: undecided 93/250  (37.2%)
        OneSmallTask:test_dmesg:                      passed    250/250 (100.0%)
        OneSmallTask:test_slack:                      passed    250/250 (100.0%)
        OneSmallTask:test_task_placement:             passed    250/250 (100.0%)
        RampDown:test_dmesg:                          passed    250/250 (100.0%)
        RampDown:test_slack:                          passed    250/250 (100.0%)
        RampDown:test_task_placement:                 undecided 20/250  (8.0%)
        RampUp:test_dmesg:                            passed    250/250 (100.0%)
        RampUp:test_slack:                            passed    250/250 (100.0%)
        RampUp:test_task_placement:                   undecided 5/250   (2.0%)
        ThreeSmallTasks:test_dmesg:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_slack:                   passed    250/250 (100.0%)
        ThreeSmallTasks:test_task_placement:          FAILED    2/250   (0.8%)
        TwoBigTasks:test_dmesg:                       passed    250/250 (100.0%)
        TwoBigTasks:test_slack:                       passed    250/250 (100.0%)
        TwoBigTasks:test_task_placement:              undecided 120/250 (48.0%)
        TwoBigThreeSmall:test_dmesg:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_slack:                  passed    250/250 (100.0%)
        TwoBigThreeSmall:test_task_placement:         passed    250/250 (100.0%)

        Error: 0/21, Failed: 1/21, Undecided: 4/21, Skipped: 0/21, Passed: 16/21

    test/load_tracking (LISA-test) in 0:30:04 [BAD]
        Invariance:iter_invariance_items[cpu=0@300000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@300000]:test_load_behaviour:    undecided 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@300000]:test_load_correctness:  FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@300000]:test_util_behaviour:    undecided 2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@300000]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@576000]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@576000]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@576000]:test_load_correctness:  FAILED    78/250  (31.2%)
        Invariance:iter_invariance_items[cpu=0@576000]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@576000]:test_util_correctness:  FAILED    75/250  (30.0%)
        Invariance:iter_invariance_items[cpu=0@825600]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@825600]:test_load_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@825600]:test_load_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@825600]:test_util_behaviour:    passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@825600]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1056000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1056000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1056000]:test_load_correctness: FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@1056000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1056000]:test_util_correctness: FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@1324800]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1324800]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1324800]:test_load_correctness: FAILED    3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=0@1324800]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1324800]:test_util_correctness: FAILED    3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=0@1516800]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1516800]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1516800]:test_load_correctness: FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=0@1516800]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1516800]:test_util_correctness: FAILED    4/250   (1.6%)
        Invariance:iter_invariance_items[cpu=0@1689600]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1689600]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1689600]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1689600]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1689600]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1766400]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1766400]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1766400]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1766400]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=0@1766400]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@825600]:test_dmesg:             passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@825600]:test_load_behaviour:    undecided 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=4@825600]:test_load_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@825600]:test_util_behaviour:    undecided 3/250   (1.2%)
        Invariance:iter_invariance_items[cpu=4@825600]:test_util_correctness:  passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1056000]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1056000]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1056000]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1056000]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1056000]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1459200]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1459200]:test_load_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1459200]:test_load_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1459200]:test_util_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1459200]:test_util_correctness: FAILED    1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1766400]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1766400]:test_load_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1766400]:test_load_correctness: FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=4@1766400]:test_util_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@1766400]:test_util_correctness: FAILED    2/250   (0.8%)
        Invariance:iter_invariance_items[cpu=4@1996800]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1996800]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1996800]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1996800]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@1996800]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2323200]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2323200]:test_load_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@2323200]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2323200]:test_util_behaviour:   undecided 1/250   (0.4%)
        Invariance:iter_invariance_items[cpu=4@2323200]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2553600]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2553600]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2553600]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2553600]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2553600]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2649600]:test_dmesg:            passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2649600]:test_load_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2649600]:test_load_correctness: passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2649600]:test_util_behaviour:   passed    250/250 (100.0%)
        Invariance:iter_invariance_items[cpu=4@2649600]:test_util_correctness: passed    250/250 (100.0%)
        Invariance:test_cpu_invariance:                                        passed    250/250 (100.0%)
        Invariance:test_freq_invariance:                                       passed    250/250 (100.0%)
        Invariance:test_load_behaviour:                                        passed    250/250 (100.0%)
        Invariance:test_load_correctness:                                      FAILED    86/250  (34.4%)
        Invariance:test_util_behaviour:                                        passed    250/250 (100.0%)
        Invariance:test_util_correctness:                                      FAILED    84/250  (33.6%)
        NTasksCPUMigration:test_dmesg:                                         passed    250/250 (100.0%)
        NTasksCPUMigration:test_util_task_migration:                           passed    250/250 (100.0%)
        OneTaskCPUMigration:test_dmesg:                                        passed    250/250 (100.0%)
        OneTaskCPUMigration:test_util_task_migration:                          FAILED    205/250 (82.0%)
        TwoTasksCPUMigration:test_dmesg:                                       passed    250/250 (100.0%)
        TwoTasksCPUMigration:test_util_task_migration:                         passed    250/250 (100.0%)

        Error: 0/92, Failed: 16/92, Undecided: 10/92, Skipped: 0/92, Passed: 66/92

    test/misfit (LISA-test) in 0:02:57 [BAD]
        StaggeredFinishes:test_dmesg:        passed    250/250 (100.0%)
        StaggeredFinishes:test_preempt_time: passed    250/250 (100.0%)
        StaggeredFinishes:test_throughput:   undecided 72/250  (28.8%)
        StaggeredFinishes:test_throughput:   FAILED    3/250   (1.2%)

        Error: 0/3, Failed: 1/3, Undecided: 1/3, Skipped: 0/3, Passed: 2/3

    test/util_tracking (LISA-test) in 0:03:17 [BAD]
        UtilConvergence:test_activations: passed 250/250 (100.0%)
        UtilConvergence:test_dmesg:       passed 250/250 (100.0%)
        UtilConvergence:test_means:       FAILED 2/250   (0.8%)

        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/other (LISA-test) in 0:06:47 [BAD]
        CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
        UserspaceSanity:test_dmesg:              passed 250/250 (100.0%)
        UserspaceSanity:test_performance_sanity: FAILED 11/250  (4.4%)

        Error: 0/3, Failed: 1/3, Undecided: 0/3, Skipped: 0/3, Passed: 2/3

    test/hotplug (LISA-test) in 0:02:20 [GOOD]
        HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
        HotplugTorture:test_target_alive: passed 250/250 (100.0%)

        Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2