AutoTest/RaptorEngineering: Difference between revisions

From coreboot
Jump to navigation Jump to search
No edit summary
No edit summary
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
[https://www.raptorengineeringinc.com Raptor Engineering] hosts an automated test system which checks the boards listed below for proper functionality up to every hour.  Successful test results are recorded to the board status repository, while failed tests are reported to the coreboot mailing list.
[https://www.raptorengineering.com Raptor Engineering] hosts [https://secure.raptorengineering.com/content/REACTS/intro.html an automated test system] which checks the boards listed below for proper functionality up to every hour.  Successful test results are recorded to the board status repository, while failed tests are reported to the coreboot mailing list.


Where possible, an extended test sequence is utilized.  This sequence checks cbmem, dmimdecode, and nvramtool for proper functionality and fails verification if any errors are generated by the listed tools.  As the current coreboot qemu boards do not support NVRAM, on qemu boards a basic test sequence is utilized which only verifies bootability.
Where possible, an extended test sequence is utilized.  This sequence checks cbmem, dmimdecode, and nvramtool for proper functionality and fails verification if any errors are generated by the listed tools.  Furthermore, native VGA initialization is verified by checking for correct video output.  As the current coreboot qemu boards do not support NVRAM, on qemu boards a basic test sequence is utilized which only verifies bootability.


Automated tests for patchsets on Gerrit are available by adding the "Raptor Engineering Automated Test Stand <noreply@raptorengineeringinc.com>" user as a reviewer.  If any board fails verification the test stand will set Verified-1, otherwise if all boards pass verification it will set Verified+1.
Automated tests for patchsets on Gerrit are available by adding the "Raptor Engineering Automated Test Stand <noreply@raptorengineeringinc.com>" user as a reviewer.  If any board fails verification the test stand will set Verified-1, otherwise if all boards pass verification it will set Verified+1.


All tests are run in parallel.  Coreboot serial output is disabled by default so that correct timestamps are uploaded, but if a board fails to boot the test stand will automatically retry with full serial debugging enabled.  As a result of this fallback mechanism, successful tests will take around 10 minutes while failed tests will take at least half an hour.
All tests are run in parallel.  Coreboot serial output is disabled by default so that correct timestamps are uploaded, but if a board fails to boot the test stand will automatically retry with full serial debugging enabled.  As a result of this fallback mechanism, successful tests will take around 15 minutes while failed tests will take at least half an hour.
 
Test queue status is available [https://secure.raptorengineering.com/coreboot/reacts_status.php here]


{| class="wikitable"
{| class="wikitable"
Line 13: Line 15:
|[[Board:asus/kfsn4-dre|ASUS KFSN4-DRE]]
|[[Board:asus/kfsn4-dre|ASUS KFSN4-DRE]]
|[[Board:emulation/qemu-q35|QEMU x86 q35/ich9]]
|[[Board:emulation/qemu-q35|QEMU x86 q35/ich9]]
|style="background-color:#fff0f0;" | [[Board:asus/kfsn4-dre|ASUS KFSN4-DRE (K8)]]
|[[Board:asus/kgpe-d16|ASUS KGPE-D16]]
|-
|-
|CPU
|CPU
|2x Opteron 8347 (Fam10h @ 1.9GHz / 4 cores ea.)
|2x Opteron 8347 (Fam10h @ 1.9GHz / 4 cores ea.)
|1x emulated x86_64
|1x emulated x86_64
|style="background-color:#fff0f0;" | 2x Opteron 8222 (K8 @ 3.0GHz / 2 cores ea.)
|1x Opteron 6378 (Fam15h @ 2.4GHz / 16 cores ea.)
|-
|-
|RAM
|RAM
|2GB DDR2-667 on Node 0 channel 0<BR>2GB DDR2-667 on Node 0 channel 1<BR>1GB DDR2-667 on Node 1 channel 0
|2GB DDR2-667 on Node 0 channel 0<BR>2GB DDR2-667 on Node 0 channel 1<BR>1GB DDR2-667 on Node 1 channel 0
|1GB emulated DDR
|1GB emulated DDR
|style="background-color:#fff0f0;" | 4GB DDR2-667 on Node 0 channel 0 (2x 2GB DIMMs)
|16GB DDR3-1333 on Node 0 channel 0<BR>16GB DDR3-1333 on Node 1 channel 0
|-
|-
|OS
|OS
|Debian Jessie 64-bit, kernel 3.16.0
|Debian Jessie 64-bit, kernel 3.16.0
|Debian Jessie 64-bit, kernel 3.16.0
|style="background-color:#fff0f0;" | Debian Jessie 64-bit, kernel 3.16.0
|Debian Jessie 64-bit, kernel 3.16.0
|Debian Jessie 64-bit, kernel 3.16.0
|-
|-
Line 29: Line 39:
|2x Broadcom NetXtreme BCM5721
|2x Broadcom NetXtreme BCM5721
|1x Intel E1000 82540EM
|1x Intel E1000 82540EM
|style="background-color:#fff0f0;" | 2x Broadcom NetXtreme BCM5721, 1x Intel 82571EB
|2x Intel 82574L
|-
|-
|Add-on cards
|Add-on cards
|None
|None
|None
|style="background-color:#fff0f0;" | 1x Intel 82571EB
|None
|None
|-
|-
|Peripherals
|Peripherals
|PS/2 keyboard
|PS/2 keyboard<br>Flat panel LCD
|None
|None
|style="background-color:#fff0f0;" | None
|Flat panel LCD
|-
|-
|Test type
|Test type
|Extended
|Extended with video verification
|Basic
|Basic
|}
|style="background-color:#fff0f0;" | Extended
 
|Extended with video verification
{| class="wikitable"
|+crossgcc
|-
|Package
|Version
|-
|ACPICA
|20150204
|-
|binutils
|2.25
|-
|gcc
|4.9.2
|-
|gmp
|6.0.0a
|-
|libelf
|0.8.13
|-
|-
|mpc
|ROM write method
|1.0.3
|Internal (fallback / normal)
|External (file selection)
|style="background-color:#fff0f0;" | Internal (fallback / normal)
|External (SPI)
|-
|-
|mpfr
|Status
|3.1.2
|style="background-color:#f0fff0;" | Operational
|style="background-color:#f0fff0;" | Operational
|style="background-color:#fff0f0;" | [https://www.coreboot.org/pipermail/coreboot/2016-August/081944.html Disabled since 08/22/2016]
|style="background-color:#f0fff0;" | Operational
|}
|}
CrossGCC is automatically updated when required; the latest version is used when testing.<BR>
This automatic update service is also available to REACTS support contract holders.


{| class="wikitable"
{| class="wikitable"
Line 79: Line 84:
|-
|-
|cbmem timestamps
|cbmem timestamps
|coreboot configuration
|Build log
|-
|coreboot configuration
|RS232 serial log
|RS232 serial log
|Build log
|coreboot configuration
|-
|-
|cbmem console log
|cbmem console log
Line 91: Line 100:
|-
|-
|dmidecode output
|dmidecode output
|
|
|-
|video decoding results
|
|
|-
|detected PCI devices
|
|
|
|

Latest revision as of 20:42, 13 January 2017

Raptor Engineering hosts an automated test system which checks the boards listed below for proper functionality up to every hour. Successful test results are recorded to the board status repository, while failed tests are reported to the coreboot mailing list.

Where possible, an extended test sequence is utilized. This sequence checks cbmem, dmimdecode, and nvramtool for proper functionality and fails verification if any errors are generated by the listed tools. Furthermore, native VGA initialization is verified by checking for correct video output. As the current coreboot qemu boards do not support NVRAM, on qemu boards a basic test sequence is utilized which only verifies bootability.

Automated tests for patchsets on Gerrit are available by adding the "Raptor Engineering Automated Test Stand <noreply@raptorengineeringinc.com>" user as a reviewer. If any board fails verification the test stand will set Verified-1, otherwise if all boards pass verification it will set Verified+1.

All tests are run in parallel. Coreboot serial output is disabled by default so that correct timestamps are uploaded, but if a board fails to boot the test stand will automatically retry with full serial debugging enabled. As a result of this fallback mechanism, successful tests will take around 15 minutes while failed tests will take at least half an hour.

Test queue status is available here

Test stand details
Board ASUS KFSN4-DRE QEMU x86 q35/ich9 ASUS KFSN4-DRE (K8) ASUS KGPE-D16
CPU 2x Opteron 8347 (Fam10h @ 1.9GHz / 4 cores ea.) 1x emulated x86_64 2x Opteron 8222 (K8 @ 3.0GHz / 2 cores ea.) 1x Opteron 6378 (Fam15h @ 2.4GHz / 16 cores ea.)
RAM 2GB DDR2-667 on Node 0 channel 0
2GB DDR2-667 on Node 0 channel 1
1GB DDR2-667 on Node 1 channel 0 		1GB emulated DDR 4GB DDR2-667 on Node 0 channel 0 (2x 2GB DIMMs) 16GB DDR3-1333 on Node 0 channel 0
16GB DDR3-1333 on Node 1 channel 0
OS Debian Jessie 64-bit, kernel 3.16.0 Debian Jessie 64-bit, kernel 3.16.0 Debian Jessie 64-bit, kernel 3.16.0 Debian Jessie 64-bit, kernel 3.16.0
Networking 2x Broadcom NetXtreme BCM5721 1x Intel E1000 82540EM 2x Broadcom NetXtreme BCM5721, 1x Intel 82571EB 2x Intel 82574L
Add-on cards None None 1x Intel 82571EB None
Peripherals PS/2 keyboard
Flat panel LCD 		None None Flat panel LCD
Test type Extended with video verification Basic Extended Extended with video verification
ROM write method Internal (fallback / normal) External (file selection) Internal (fallback / normal) External (SPI)
Status Operational Operational Disabled since 08/22/2016 Operational

CrossGCC is automatically updated when required; the latest version is used when testing.
This automatic update service is also available to REACTS support contract holders.

Information contained in linked log files
Successful Test Failed Boot Failed Build
cbmem timestamps coreboot configuration Build log
coreboot configuration RS232 serial log coreboot configuration
cbmem console log
dmesg output
dmidecode output
video decoding results
detected PCI devices
RS232 serial log
Retrieved from "https://www.coreboot.org/index.php?title=AutoTest/RaptorEngineering&oldid=23337"