[LinuxBIOS] [ANNOUNCE] Automated and Distributed Test System for LinuxBIOS

[Stefan Reinauer]

Dear LinuxBIOS community,

it's about time to release the automated and distributed test system for
LinuxBIOS.

The test system creates a framework to run images created by the
automatic build system (known from http://snapshots.linuxbios.org/
a.k.a http://qa.linuxbios.org/) on real hardware. 

So from now on LinuxBIOS binary images can be downloaded from
http://qa.linuxbios.org/ that are known to work on at least one
mainboard of the given type. 

I started a page gathering all the information about the test system in
the Wiki. Please help completing it:
http://linuxbios.org/Distributed_and_Automated_Testsystem

Get a short overview on the test system from the slides of the talk I
held at the LinuxBIOS symposium 2006:
http://www.coresystems.de/PDFs/LinuxBIOS-testing/Slides-LinuxBIOS-QA.pdf

Since nobody is supposed to have the 50+ mainboards supported by
LinuxBIOSv2 at a single place, the system works distributed and 
single boards can be hooked up to the test system with relatively 
cheap components.

So please do understand this announcement as a call for participation.
This test system is only of value if we get many boards tested on a
regular basis.

Are you working for a mainboard or system vendor that supports
LinuxBIOS? Are you a developer that has a spare board supported by
LinuxBIOS sitting at your office? Then please have a look at the
Test Integration Manual to find out how you can help the LinuxBIOS
project to improve its quality:
http://www.coresystems.de/PDFs/LinuxBIOS-testing/TestIntegrationManual.pdf
The Test Integration Manual also explains how you can add new tests and
test components to the test system.

Are you developing LinuxBIOS, and just want to know how to use the test
system without setting up your own dedicated test hardware? Make sure a
bios image you build works on real hardware? Then have a glance at the
Developer's Manual:
http://www.coresystems.de/PDFs/LinuxBIOS-testing/DevelopersManual.pdf

An exercise that some of you might know from safety critical and
embedded software is test and project documentation. Basically the goal
should be to have (at least) one test for each requirement of the
software system. 

There are two IEEE standards for writing specifications for such
requirements and test specifications:

IEEE Std 829-1998: IEEE Standard for Software Test Documentation
IEEE Std 830-1998: IEEE Recommended Practice for Software Requirements Specifications

I wrote a requirements specification for LinuxBIOS which is rather
incomplete and only scratches the top of the iceberg. (Not to mention
that not all of the requirements are properly tested, while some tests
dont have a requirement yet)
While I tried to keep close to the standard, there are some differences
and inaccuracies that might be fine for most projects, unless LinuxBIOS
is taking a relevant part in a mission critical application that
requires strict compliance (in which case it should be easy to fix
that)

Find the documents here:
http://www.coresystems.de/PDFs/LinuxBIOS-testing/RequirementsSpecification.pdf
http://www.coresystems.de/PDFs/LinuxBIOS-testing/TestSpecification.pdf


Any ideas, add-ons, patches, hints, flames, questions are of course welcome.

Again: If you have a chance to participate in the LinuxBIOS testing,
please have a glimpse at the Test Integration Manual and contact me to
get things going.

Best regards,
Stefan

-- 
coresystems GmbH • Brahmsstr. 16 • D-79104 Freiburg i. Br.
      Tel.: +49 761 7668825 • Fax: +49 761 7664613
Email: info at coresystems.de  • http://www.coresystems.de/