EAS Development for Mainline Linux

EAS development for mainline Linux takes place on LKML and Linaro eas-dev mailing lists where proposed patches are discussed with the community. Arm now provides a pre-integrated version of latest EAS patches for mainline, for upstream development use.

EAS mainline is a moving integration branch tracking tip sched/core, mainline release candidate or linux-pm testing and adding the latest versions of EAS related patch sets targeted at the mainline kernel. The patches may already be under discussion on relevant open source mailing lists or waiting here while their dependencies get resolved and merged upstream.

The integration branch comes as is with synthetic test case testing only. The intention is to have a single branch containing the full EAS picture for mainline Linux kernel developers interested in tracking progress. This EAS mainline integration branch is not suitable for product development. For product, please use the AOSP common kernel.

Source code

EAS mainline integration branches are made available regularly on:

        git://linux-arm.org/linux-power.git

The releases are tagged with the date e.g. -20190310 so old integrations are available for comparison

Latest: integration-20200331

Tree: eas/next/integration

Base: eas/next/integration-base

Features:

  1. Schedutil as Energy Model user (v4 on lkml)
  2. Util(ization) update during Idle migration (v1 on lkml)
  3. Activity Monitor Unit (AMU) support (v6 on lkml)
  4. (Devfreq) devices as Energy Model user (v5 on lkml)
  5. Streamline select_task_rq() & select_task_rq_fair() (v2 on lkml)
  6. Instrument sched domain flags (v1 on lkml)
  7. SCHED_DEADLINE capacity awareness (v1 on lkml)

Features in Linux mainline:

  1. v4.13 - CPU Invariant Engine (CIE)
  2. v4.15 - Frequency Invariant Engine (FIE)
  3. v4.17 - Idle CPU Per-Entity Load-Tracking (PELT) update
  4. v4.17 - Util(ization) Est(imated)
  5. v4.20 - Misfit task, i.e. forcing migration of running tasks that do not fit on the CPU they are currently running on
  6. v4.20 - Runtime scheduler domain flag detection
  7. v5.0   - Per-cpu Energy Model (EM) and Energy Aware Scheduling (EAS)
  8. v5.3   - Tracepoints (PELT and over-utilzation)
  9. v5.3   - Util(ization) clamping (core and per-task interface)
  10. v5.4   - Util(ization) clamping (cgroup interface)
  11. v5.4   - Patch-set 'sched/fair: Reduce complexity of energy calculation'
  12. v5.5   - Thermal/Cpu Cooling as Energy Model (EM) user

Platforms we test on:

  1. Arm Juno r0 (arm64)
  2. Arm Juno r2 (arm64)
  3. Hikey960 (arm64) (needs out-of-tree patches  (hikey960_mainline topic))
  4. Pixel3 (arm64) (needs out-of-tree patch (blueline_mainline topic))
  5. DB845C (arm64) (needs out-of-tree patch (db845c_mainline topic))
  6. Arm TC2 (arm32)

Next work items:

  1. Pushing Features upstream
  2. 100% test pass rate on all platforms
  • Test Results (integration 20200331) 

    (1) juno-r0:
        
        testcase                                                                old%   new%  delta%       pvalue fix_iter#
        ------------------------------------------------------------------------------------------------------------------
        Invariance:iter_invariance_items[cpu=1@625000]:test_load_correctness    2.0%  12.0%   10.0%     1.16e-05        23
        Invariance:iter_invariance_items[cpu=1@625000]:test_util_correctness    2.0%  12.0%   10.0%     1.16e-05        23
        Invariance:test_load_correctness                                        2.4%  12.0%    9.6%     3.80e-05        23
        Invariance:test_util_correctness                                        2.4%  12.0%    9.6%     3.80e-05        23
        OneTaskCPUMigration:test_util_task_migration                           97.6% 100.0%    2.4%     3.03e-02         1
        TwoBigThreeSmall:test_task_placement                                    4.7%   0.0%   -4.7%     7.44e-03
            
        test/eas_behaviour (LISA-test) in 0:08:24 [GOOD]
            EnergyModelWakeMigration:test_slack:          passed    250/250 (100.0%)
            EnergyModelWakeMigration:test_task_placement: undecided 3/250   (1.2%)
            OneSmallTask:test_slack:                      passed    250/250 (100.0%)
            OneSmallTask:test_task_placement:             undecided 13/250  (5.2%)
            RampDown:test_slack:                          passed    250/250 (100.0%)
            RampDown:test_task_placement:                 passed    250/250 (100.0%)
            RampUp:test_slack:                            passed    250/250 (100.0%)
            RampUp:test_task_placement:                   passed    250/250 (100.0%)
            ThreeSmallTasks:test_slack:                   passed    250/250 (100.0%)
            ThreeSmallTasks:test_task_placement:          undecided 50/250  (20.0%)
            TwoBigTasks:test_slack:                       passed    250/250 (100.0%)
            TwoBigTasks:test_task_placement:              undecided 37/250  (14.8%)
            TwoBigThreeSmall:test_slack:                  passed    250/250 (100.0%)
            TwoBigThreeSmall:test_task_placement:         undecided 90/250  (36.0%)
            
            Error: 0/14, Failed: 0/14, Undecided: 5/14, Skipped: 0/14, Passed: 9/14
        
        test/load_tracking (LISA-test) in 0:15:54 [BAD]
            Invariance:iter_invariance_items[cpu=0@450000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@575000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@575000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@575000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@575000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@775000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@775000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@775000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@775000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@850000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@850000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@850000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@850000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@450000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@450000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@450000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@450000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@625000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@625000]:test_load_correctness:  FAILED 30/250  (12.0%)
            Invariance:iter_invariance_items[cpu=1@625000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@625000]:test_util_correctness:  FAILED 30/250  (12.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@950000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@950000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@950000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@950000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:test_cpu_invariance:                                        passed 250/250 (100.0%)
            Invariance:test_freq_invariance:                                       passed 250/250 (100.0%)
            Invariance:test_load_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_load_correctness:                                      FAILED 30/250  (12.0%)
            Invariance:test_util_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_util_correctness:                                      FAILED 30/250  (12.0%)
            NTasksCPUMigration:test_util_task_migration:                           passed 250/250 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          FAILED 250/250 (100.0%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed 250/250 (100.0%)
            
            Error: 0/49, Failed: 5/49, Undecided: 0/49, Skipped: 0/49, Passed: 44/49
        
        test/misfit (LISA-test) in 0:02:03 [GOOD]
            StaggeredFinishes:test_preempt_time: passed 250/250 (100.0%)
            StaggeredFinishes:test_throughput:   passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
        test/other (LISA-test) in 0:04:08 [GOOD]
            CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
            UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
        test/hotplug (LISA-test) in 0:01:44 [GOOD]
            HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
            HotplugTorture:test_target_alive: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
    (2) juno-r2:
        
        testcase                                                                old%   new%  delta%       pvalue fix_iter#
        ------------------------------------------------------------------------------------------------------------------
        Invariance:iter_invariance_items[cpu=1@600000]:test_load_correctness    2.0%   6.0%    4.0%     3.74e-02        84
        Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness    2.0%   6.0%    4.0%     3.74e-02        84
        Invariance:test_load_correctness                                        2.4%   6.8%    4.4%     3.07e-02        72
        Invariance:test_util_correctness                                        2.0%   6.4%    4.4%     2.36e-02        47
        OneTaskCPUMigration:test_util_task_migration                           97.2%  90.8%   -6.4%     4.00e-03
        
        test/eas_behaviour (LISA-test) in 0:08:16 [BAD]
            EnergyModelWakeMigration:test_slack:          passed    250/250 (100.0%)
            EnergyModelWakeMigration:test_task_placement: passed    250/250 (100.0%)
            OneSmallTask:test_slack:                      passed    250/250 (100.0%)
            OneSmallTask:test_task_placement:             passed    250/250 (100.0%)
            RampDown:test_slack:                          passed    250/250 (100.0%)
            RampDown:test_task_placement:                 passed    250/250 (100.0%)
            RampUp:test_slack:                            passed    250/250 (100.0%)
            RampUp:test_task_placement:                   passed    250/250 (100.0%)
            ThreeSmallTasks:test_slack:                   passed    250/250 (100.0%)
            ThreeSmallTasks:test_task_placement:          passed    250/250 (100.0%)
            TwoBigTasks:test_slack:                       passed    250/250 (100.0%)
            TwoBigTasks:test_task_placement:              undecided 12/250  (4.8%)
            TwoBigThreeSmall:test_slack:                  passed    250/250 (100.0%)
            TwoBigThreeSmall:test_task_placement:         undecided 77/250  (30.8%)
            TwoBigThreeSmall:test_task_placement:         FAILED    31/250  (12.4%)
            
            Error: 0/14, Failed: 1/14, Undecided: 2/14, Skipped: 0/14, Passed: 12/14
        
        test/load_tracking (LISA-test) in 0:11:57 [BAD]
            Invariance:iter_invariance_items[cpu=0@450000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_load_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@450000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_load_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_util_correctness:  passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@950000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@950000]:test_load_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@950000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@950000]:test_util_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_load_correctness:  FAILED 15/250  (6.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness:  FAILED 15/250  (6.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:test_cpu_invariance:                                        passed 250/250 (100.0%)
            Invariance:test_freq_invariance:                                       passed 250/250 (100.0%)
            Invariance:test_load_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_load_correctness:                                      FAILED 17/250  (6.8%)
            Invariance:test_util_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_util_correctness:                                      FAILED 16/250  (6.4%)
            NTasksCPUMigration:test_util_task_migration:                           passed 250/250 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          FAILED 227/250 (90.8%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed 250/250 (100.0%)
            
            Error: 0/33, Failed: 8/33, Undecided: 0/33, Skipped: 0/33, Passed: 25/33
        
        test/misfit (LISA-test) in 0:02:02 [GOOD]
            StaggeredFinishes:test_preempt_time: passed 250/250 (100.0%)
            StaggeredFinishes:test_throughput:   passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
        test/other (LISA-test) in 0:03:14 [GOOD]
            CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
            UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
        test/hotplug (LISA-test) in 0:01:44 [GOOD]
            HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
            HotplugTorture:test_target_alive: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
    (3) hikey960:
        
        testcase                                                                old%   new%  delta%       pvalue fix_iter#
        ------------------------------------------------------------------------------------------------------------------
        Invariance:iter_invariance_items[cpu=0@1709000]:test_load_correctness  13.2%   0.4%  -12.8%     1.40e-09
        Invariance:iter_invariance_items[cpu=0@1709000]:test_util_correctness  13.2%   0.4%  -12.8%     1.40e-09
        Invariance:iter_invariance_items[cpu=0@1844000]:test_load_correctness   8.4%   0.0%   -8.4%     6.15e-07
        Invariance:iter_invariance_items[cpu=0@1844000]:test_util_correctness   8.4%   0.0%   -8.4%     6.15e-07
        Invariance:iter_invariance_items[cpu=4@2112000]:test_load_behaviour    96.0%   0.0%  -96.0%    5.59e-132
        Invariance:iter_invariance_items[cpu=4@2112000]:test_util_behaviour    96.0%   0.0%  -96.0%    5.59e-132
        Invariance:test_load_behaviour                                         96.0%   0.0%  -96.0%    5.59e-132
        Invariance:test_load_correctness                                       22.4%   2.4%  -20.0%     1.79e-12
        Invariance:test_util_behaviour                                         96.0%   0.0%  -96.0%    5.59e-132
        Invariance:test_util_correctness                                       22.4%   2.4%  -20.0%     1.79e-12
        OneTaskCPUMigration:test_util_task_migration                           57.2%  46.0%  -11.2%     1.56e-02
        
        test/eas_behaviour (LISA-test) in 0:09:09 [GOOD]
            EnergyModelWakeMigration:test_slack:          passed 250/250 (100.0%)
            EnergyModelWakeMigration:test_task_placement: passed 250/250 (100.0%)
            OneSmallTask:test_slack:                      passed 250/250 (100.0%)
            OneSmallTask:test_task_placement:             passed 250/250 (100.0%)
            RampDown:test_slack:                          passed 250/250 (100.0%)
            RampDown:test_task_placement:                 passed 250/250 (100.0%)
            RampUp:test_slack:                            passed 250/250 (100.0%)
            RampUp:test_task_placement:                   passed 250/250 (100.0%)
            ThreeSmallTasks:test_slack:                   passed 250/250 (100.0%)
            ThreeSmallTasks:test_task_placement:          passed 250/250 (100.0%)
            TwoBigTasks:test_slack:                       passed 250/250 (100.0%)
            TwoBigTasks:test_task_placement:              passed 250/250 (100.0%)
            TwoBigThreeSmall:test_slack:                  passed 250/250 (100.0%)
            TwoBigThreeSmall:test_task_placement:         passed 250/250 (100.0%)

            Error: 0/14, Failed: 0/14, Undecided: 0/14, Skipped: 0/14, Passed: 14/14
        
        test/load_tracking (LISA-test) in 0:17:39 [BAD]
            Invariance:iter_invariance_items[cpu=0@533000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@533000]:test_load_correctness:  FAILED 3/250   (1.2%)
            Invariance:iter_invariance_items[cpu=0@533000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@533000]:test_util_correctness:  FAILED 3/250   (1.2%)
            Invariance:iter_invariance_items[cpu=0@999000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@999000]:test_load_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@999000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@999000]:test_util_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@1402000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1402000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1402000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1402000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1709000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1709000]:test_load_correctness: FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@1709000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1709000]:test_util_correctness: FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=0@1844000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1844000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1844000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1844000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@903000]:test_load_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@903000]:test_load_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=4@903000]:test_util_behaviour:    passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@903000]:test_util_correctness:  FAILED 1/250   (0.4%)
            Invariance:iter_invariance_items[cpu=4@1421000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1421000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1421000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1421000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1805000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1805000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1805000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1805000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2112000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2112000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2112000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2112000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2362000]:test_load_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2362000]:test_load_correctness: passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2362000]:test_util_behaviour:   passed 250/250 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2362000]:test_util_correctness: passed 250/250 (100.0%)
            Invariance:test_cpu_invariance:                                        passed 250/250 (100.0%)
            Invariance:test_freq_invariance:                                       passed 250/250 (100.0%)
            Invariance:test_load_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_load_correctness:                                      FAILED 6/250   (2.4%)
            Invariance:test_util_behaviour:                                        passed 250/250 (100.0%)
            Invariance:test_util_correctness:                                      FAILED 6/250   (2.4%)
            NTasksCPUMigration:test_util_task_migration:                           passed 250/250 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          FAILED 115/250 (46.0%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed 250/250 (100.0%)
            
            Error: 0/49, Failed: 11/49, Undecided: 0/49, Skipped: 0/49, Passed: 38/49
        
        test/misfit (LISA-test) in 0:02:32 [BAD]
            StaggeredFinishes:test_preempt_time: passed 250/250 (100.0%)
            StaggeredFinishes:test_throughput:   FAILED 4/250   (1.6%)
            
            Error: 0/2, Failed: 1/2, Undecided: 0/2, Skipped: 0/2, Passed: 1/2
        
        test/other (LISA-test) in 0:05:29 [GOOD]
            CapacitySanity:test_capacity_sanity:     passed 250/250 (100.0%)
            UserspaceSanity:test_performance_sanity: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
    (4) tc2:

        testcase                                       old%   new%  delta%       pvalue fix_iter#
        -----------------------------------------------------------------------------------------
        StaggeredFinishes:test_throughput              6.0%  16.1%   10.1%     3.22e-04        28
        TwoBigThreeSmall:test_task_placement           3.6%   0.4%   -3.2%     2.03e-02

        test/eas_behaviour (LISA-test) in 0:08:28 [BAD]
            EnergyModelWakeMigration:test_slack:          passed 250/250 (100.0%)
            EnergyModelWakeMigration:test_task_placement: passed 250/250 (100.0%)
            OneSmallTask:test_slack:                      passed 250/250 (100.0%)
            OneSmallTask:test_task_placement:             passed 250/250 (100.0%)
            RampDown:test_slack:                          passed 250/250 (100.0%)
            RampDown:test_task_placement:                 passed 250/250 (100.0%)
            RampUp:test_slack:                            passed 250/250 (100.0%)
            RampUp:test_task_placement:                   passed 250/250 (100.0%)
            ThreeSmallTasks:test_slack:                   passed 250/250 (100.0%)
            ThreeSmallTasks:test_task_placement:          FAILED 1/250   (0.4%)
            TwoBigTasks:test_slack:                       passed 250/250 (100.0%)
            TwoBigTasks:test_task_placement:              passed 250/250 (100.0%)
            TwoBigThreeSmall:test_slack:                  passed 250/250 (100.0%)
            TwoBigThreeSmall:test_task_placement:         FAILED 1/250   (0.4%)
        
            Error: 0/14, Failed: 2/14, Undecided: 0/14, Skipped: 0/14, Passed: 12/14
        
        test/load_tracking (LISA-test) in 0:23:46 [GOOD]
            Invariance:iter_invariance_items[cpu=0@350000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@350000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@350000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@350000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@400000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@400000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@400000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@400000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@500000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@500000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@500000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@500000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@600000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@600000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@600000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@600000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@700000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@800000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@900000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@900000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@900000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@900000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1000000]:test_load_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1000000]:test_load_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1000000]:test_util_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1000000]:test_util_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@500000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@500000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@500000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@500000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@600000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@700000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@700000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@700000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@700000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@800000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@900000]:test_load_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@900000]:test_load_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@900000]:test_util_behaviour:    passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@900000]:test_util_correctness:  passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_load_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_load_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_util_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1000000]:test_util_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_load_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_load_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_util_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1100000]:test_util_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_load_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_load_correctness: passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_util_behaviour:   passed 249/249 (100.0%)
            Invariance:iter_invariance_items[cpu=1@1200000]:test_util_correctness: passed 249/249 (100.0%)
            Invariance:test_cpu_invariance:                                        passed 249/249 (100.0%)
            Invariance:test_freq_invariance:                                       passed 249/249 (100.0%)
            Invariance:test_load_behaviour:                                        passed 249/249 (100.0%)
            Invariance:test_load_correctness:                                      passed 249/249 (100.0%)
            Invariance:test_util_behaviour:                                        passed 249/249 (100.0%)
            Invariance:test_util_correctness:                                      passed 249/249 (100.0%)
            NTasksCPUMigration:test_util_task_migration:                           passed 249/249 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          passed 249/249 (100.0%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed 249/249 (100.0%)
        
            Error: 0/73, Failed: 0/73, Undecided: 0/73, Skipped: 0/73, Passed: 73/73
        
        test/misfit (LISA-test) in 0:02:06 [BAD]
            StaggeredFinishes:test_preempt_time: passed 249/249 (100.0%)
            StaggeredFinishes:test_throughput:   FAILED 40/249  (16.1%)
        
            Error: 0/2, Failed: 1/2, Undecided: 0/2, Skipped: 0/2, Passed: 1/2
        
        test/other (LISA-test) in 0:05:37 [GOOD]
            CapacitySanity:test_capacity_sanity:     passed 249/249 (100.0%)
            UserspaceSanity:test_performance_sanity: passed 249/249 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
        test/hotplug (LISA-test) in 0:01:50 [GOOD]
            HotplugTorture:test_cpus_alive:   passed 250/250 (100.0%)
            HotplugTorture:test_target_alive: passed 250/250 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
    (5) pixel3:
        
        testcase                                                                old%   new%  delta%       pvalue fix_iter#
        ------------------------------------------------------------------------------------------------------------------
        Invariance:iter_invariance_items[cpu=0@1324800]:test_load_correctness  16.5%   1.7%  -14.8%     1.41e-08
        Invariance:iter_invariance_items[cpu=0@1324800]:test_util_correctness  15.5%   1.7%  -13.8%     6.23e-08
        Invariance:iter_invariance_items[cpu=0@1516800]:test_load_correctness   6.5%   0.9%   -5.6%     2.35e-03
        Invariance:iter_invariance_items[cpu=0@1516800]:test_util_correctness   6.5%   0.9%   -5.6%     2.35e-03
        Invariance:iter_invariance_items[cpu=0@300000]:test_load_correctness    9.5%   0.4%   -9.1%     2.80e-06
        Invariance:iter_invariance_items[cpu=0@576000]:test_load_correctness   50.5%  31.9%  -18.6%     8.94e-05
        Invariance:iter_invariance_items[cpu=0@576000]:test_util_correctness   49.5%  33.2%  -16.3%     6.15e-04
        Invariance:iter_invariance_items[cpu=0@825600]:test_load_correctness    0.5%   4.7%    4.2%     7.52e-03        69
        Invariance:iter_invariance_items[cpu=0@825600]:test_util_correctness    0.5%   4.3%    3.8%     1.35e-02        78
        Invariance:test_load_correctness                                       66.0%  37.0%  -29.0%     1.94e-09
        Invariance:test_util_correctness                                       62.0%  37.4%  -24.6%     3.32e-07
        OneTaskCPUMigration:test_util_task_migration                           75.5%  66.8%   -8.7%     5.67e-02
        ThreeSmallTasks:test_task_placement                                     2.5%   0.0%   -2.5%     2.03e-02
        TwoBigThreeSmall:test_task_placement                                    2.5%   0.0%   -2.5%     2.03e-02
        
        test/eas_behaviour (LISA-test) in 0:11:09 [BAD]
            EnergyModelWakeMigration:test_slack:          ERROR     1/1     (100.0%)
            EnergyModelWakeMigration:test_task_placement: ERROR     1/1     (100.0%)
            EnergyModelWakeMigration:test_slack:          passed    236/236 (100.0%)
            EnergyModelWakeMigration:test_task_placement: undecided 1/236   (0.4%)
            OneSmallTask:test_slack:                      ERROR     1/1     (100.0%)
            OneSmallTask:test_task_placement:             ERROR     1/1     (100.0%)
            OneSmallTask:test_slack:                      passed    236/236 (100.0%)
            OneSmallTask:test_task_placement:             passed    236/236 (100.0%)
            RampDown:test_slack:                          ERROR     1/1     (100.0%)
            RampDown:test_task_placement:                 ERROR     1/1     (100.0%)
            RampDown:test_slack:                          passed    236/236 (100.0%)
            RampDown:test_task_placement:                 undecided 4/236   (1.7%)
            RampUp:test_slack:                            ERROR     1/1     (100.0%)
            RampUp:test_task_placement:                   ERROR     1/1     (100.0%)
            RampUp:test_slack:                            passed    236/236 (100.0%)
            RampUp:test_task_placement:                   passed    236/236 (100.0%)
            ThreeSmallTasks:test_slack:                   ERROR     1/1     (100.0%)
            ThreeSmallTasks:test_task_placement:          ERROR     1/1     (100.0%)
            ThreeSmallTasks:test_slack:                   passed    236/236 (100.0%)
            ThreeSmallTasks:test_task_placement:          passed    236/236 (100.0%)
            TwoBigTasks:test_slack:                       ERROR     1/1     (100.0%)
            TwoBigTasks:test_task_placement:              ERROR     1/1     (100.0%)
            TwoBigTasks:test_slack:                       passed    236/236 (100.0%)
            TwoBigTasks:test_task_placement:              undecided 44/236  (18.6%)
            TwoBigThreeSmall:test_slack:                  ERROR     1/1     (100.0%)
            TwoBigThreeSmall:test_task_placement:         ERROR     1/1     (100.0%)
            TwoBigThreeSmall:test_slack:                  passed    236/236 (100.0%)
            TwoBigThreeSmall:test_task_placement:         passed    236/236 (100.0%)
            
            Error: 14/28, Failed: 0/28, Undecided: 3/28, Skipped: 0/28, Passed: 11/28
        
        test/load_tracking (LISA-test) in 0:30:41 [BAD]
            Invariance:iter_invariance_items[cpu=0@300000]:test_load_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_load_correctness:  FAILED 1/235   (0.4%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_util_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_util_correctness:  passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_load_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_load_correctness:  FAILED 75/235  (31.9%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_util_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_util_correctness:  FAILED 78/235  (33.2%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_load_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_load_correctness:  FAILED 11/235  (4.7%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_util_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_util_correctness:  FAILED 10/235  (4.3%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_load_correctness: FAILED 1/235   (0.4%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_util_correctness: FAILED 1/235   (0.4%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_load_correctness: FAILED 4/235   (1.7%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_util_correctness: FAILED 4/235   (1.7%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_load_correctness: FAILED 2/235   (0.9%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_util_correctness: FAILED 2/235   (0.9%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_load_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_load_correctness:  passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_util_behaviour:    passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_util_correctness:  passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_load_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_load_correctness: passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_util_behaviour:   passed 235/235 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_util_correctness: passed 235/235 (100.0%)
            Invariance:test_cpu_invariance:                                        passed 235/235 (100.0%)
            Invariance:test_freq_invariance:                                       passed 235/235 (100.0%)
            Invariance:test_load_behaviour:                                        passed 235/235 (100.0%)
            Invariance:test_load_correctness:                                      FAILED 87/235  (37.0%)
            Invariance:test_util_behaviour:                                        passed 235/235 (100.0%)
            Invariance:test_util_correctness:                                      FAILED 88/235  (37.4%)
            NTasksCPUMigration:test_util_task_migration:                           passed 235/235 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          FAILED 157/235 (66.8%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed 235/235 (100.0%)
            
            Error: 0/73, Failed: 14/73, Undecided: 0/73, Skipped: 0/73, Passed: 59/73
        
        test/misfit (LISA-test) in 0:02:18 [GOOD]
            StaggeredFinishes:test_preempt_time: passed    237/237 (100.0%)
            StaggeredFinishes:test_throughput:   undecided 23/237  (9.7%)
            
            Error: 0/2, Failed: 0/2, Undecided: 1/2, Skipped: 0/2, Passed: 1/2
        
        test/other (LISA-test) in 0:04:32 [BAD]
            CapacitySanity:test_capacity_sanity:     passed 236/236 (100.0%)
            UserspaceSanity:test_performance_sanity: FAILED 178/236 (75.4%)
            
            Error: 0/2, Failed: 1/2, Undecided: 0/2, Skipped: 0/2, Passed: 1/2
        
        test/hotplug (LISA-test) in 0:01:33 [GOOD]
            HotplugTorture:test_cpus_alive:   passed 237/237 (100.0%)
            HotplugTorture:test_target_alive: passed 237/237 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2
        
    (6) db845c:
        
        testcase                                                                old%   new%  delta%       pvalue fix_iter#
        ------------------------------------------------------------------------------------------------------------------
        Invariance:iter_invariance_items[cpu=0@1056000]:test_load_correctness   0.9%   3.6%    2.7%     6.64e-02        94
        Invariance:iter_invariance_items[cpu=0@1056000]:test_util_correctness   0.9%   3.6%    2.7%     6.64e-02        94
        Invariance:iter_invariance_items[cpu=0@1324800]:test_load_correctness  27.5%   2.8%  -24.6%     4.35e-15
        Invariance:iter_invariance_items[cpu=0@1324800]:test_util_correctness  27.5%   2.8%  -24.6%     4.35e-15
        Invariance:iter_invariance_items[cpu=0@1689600]:test_load_correctness   6.3%   2.0%   -4.3%     3.21e-02
        Invariance:iter_invariance_items[cpu=0@1689600]:test_util_correctness   6.3%   2.0%   -4.3%     3.21e-02
        Invariance:iter_invariance_items[cpu=0@300000]:test_load_correctness    4.5%   1.2%   -3.3%     4.55e-02
        Invariance:test_load_correctness                                       59.5%  34.0%  -25.5%     3.87e-08
        Invariance:test_util_correctness                                       58.6%  36.4%  -22.1%     2.11e-06
        UserspaceSanity:test_performance_sanity                                 4.1%   8.1%    4.0%     8.42e-02        98
        
        test/eas_behaviour (LISA-test) in 0:09:32 [BAD]
            EnergyModelWakeMigration:test_slack:          passed    247/247 (100.0%)
            EnergyModelWakeMigration:test_task_placement: undecided 1/247   (0.4%)
            OneSmallTask:test_slack:                      passed    247/247 (100.0%)
            OneSmallTask:test_task_placement:             passed    247/247 (100.0%)
            RampDown:test_slack:                          FAILED    1/247   (0.4%)
            RampDown:test_task_placement:                 undecided 43/247  (17.4%)
            RampUp:test_slack:                            passed    247/247 (100.0%)
            RampUp:test_task_placement:                   undecided 21/247  (8.5%)
            ThreeSmallTasks:test_slack:                   passed    247/247 (100.0%)
            ThreeSmallTasks:test_task_placement:          FAILED    4/247   (1.6%)
            TwoBigTasks:test_slack:                       passed    247/247 (100.0%)
            TwoBigTasks:test_task_placement:              undecided 201/247 (81.4%)
            TwoBigThreeSmall:test_slack:                  passed    247/247 (100.0%)
            TwoBigThreeSmall:test_task_placement:         undecided 2/247   (0.8%)
            
            Error: 0/14, Failed: 2/14, Undecided: 5/14, Skipped: 0/14, Passed: 7/14
        
        test/load_tracking (LISA-test) in 0:27:46 [BAD]
            Invariance:iter_invariance_items[cpu=0@300000]:test_load_behaviour:    undecided 3/247   (1.2%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_load_correctness:  FAILED    3/247   (1.2%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_util_behaviour:    undecided 3/247   (1.2%)
            Invariance:iter_invariance_items[cpu=0@300000]:test_util_correctness:  passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_load_behaviour:    undecided 2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_load_correctness:  FAILED    68/247  (27.5%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_util_behaviour:    undecided 2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=0@576000]:test_util_correctness:  FAILED    77/247  (31.2%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_load_behaviour:    passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_load_correctness:  passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_util_behaviour:    passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@825600]:test_util_correctness:  passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_load_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_load_correctness: FAILED    9/247   (3.6%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_util_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1056000]:test_util_correctness: FAILED    9/247   (3.6%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_load_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_load_correctness: FAILED    7/247   (2.8%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_util_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1324800]:test_util_correctness: FAILED    7/247   (2.8%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_load_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_load_correctness: FAILED    6/247   (2.4%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_util_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1516800]:test_util_correctness: FAILED    7/247   (2.8%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_load_behaviour:   FAILED    2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_load_correctness: FAILED    5/247   (2.0%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_util_behaviour:   FAILED    2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=0@1689600]:test_util_correctness: FAILED    5/247   (2.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_load_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_load_correctness: FAILED    2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_util_behaviour:   passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=0@1766400]:test_util_correctness: FAILED    2/247   (0.8%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_load_behaviour:    undecided 19/247  (7.7%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_load_correctness:  passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_util_behaviour:    undecided 19/247  (7.7%)
            Invariance:iter_invariance_items[cpu=4@825600]:test_util_correctness:  passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_load_behaviour:   undecided 12/247  (4.9%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_util_behaviour:   undecided 12/247  (4.9%)
            Invariance:iter_invariance_items[cpu=4@1056000]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_load_behaviour:   undecided 27/247  (10.9%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_util_behaviour:   undecided 27/247  (10.9%)
            Invariance:iter_invariance_items[cpu=4@1459200]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_load_behaviour:   undecided 20/247  (8.1%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_util_behaviour:   undecided 20/247  (8.1%)
            Invariance:iter_invariance_items[cpu=4@1766400]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_load_behaviour:   undecided 9/247   (3.6%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_util_behaviour:   undecided 9/247   (3.6%)
            Invariance:iter_invariance_items[cpu=4@1996800]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_load_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_util_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2323200]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_load_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_util_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2553600]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_load_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_load_behaviour:   FAILED    19/247  (7.7%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_load_correctness: passed    247/247 (100.0%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_util_behaviour:   undecided 18/247  (7.3%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_util_behaviour:   FAILED    19/247  (7.7%)
            Invariance:iter_invariance_items[cpu=4@2649600]:test_util_correctness: passed    247/247 (100.0%)
            Invariance:test_cpu_invariance:                                        FAILED    19/247  (7.7%)
            Invariance:test_freq_invariance:                                       passed    247/247 (100.0%)
            Invariance:test_load_behaviour:                                        FAILED    21/247  (8.5%)
            Invariance:test_load_correctness:                                      FAILED    84/247  (34.0%)
            Invariance:test_util_behaviour:                                        FAILED    21/247  (8.5%)
            Invariance:test_util_correctness:                                      FAILED    90/247  (36.4%)
            NTasksCPUMigration:test_util_task_migration:                           passed    247/247 (100.0%)
            OneTaskCPUMigration:test_util_task_migration:                          FAILED    160/247 (64.8%)
            TwoTasksCPUMigration:test_util_task_migration:                         passed    247/247 (100.0%)
            
            Error: 0/73, Failed: 23/73, Undecided: 20/73, Skipped: 0/73, Passed: 32/73
        
        test/misfit (LISA-test) in 0:02:39 [GOOD]
            StaggeredFinishes:test_preempt_time: passed    247/247 (100.0%)
            StaggeredFinishes:test_throughput:   undecided 216/247 (87.4%)
            
            Error: 0/2, Failed: 0/2, Undecided: 1/2, Skipped: 0/2, Passed: 1/2
        
        test/other (LISA-test) in 0:05:50 [BAD]
            CapacitySanity:test_capacity_sanity:     passed 247/247 (100.0%)
            UserspaceSanity:test_performance_sanity: FAILED 20/247  (8.1%)
            
            Error: 0/2, Failed: 1/2, Undecided: 0/2, Skipped: 0/2, Passed: 1/2
        
        test/hotplug (LISA-test) in 0:02:16 [GOOD]
            HotplugTorture:test_cpus_alive:   passed 247/247 (100.0%)
            HotplugTorture:test_target_alive: passed 247/247 (100.0%)
            
            Error: 0/2, Failed: 0/2, Undecided: 0/2, Skipped: 0/2, Passed: 2/2