[coreboot] [RFC] Tracking run tested coreboot revisions for boards

[Thomas Gstädtner]

On Sat, Oct 8, 2011 at 10:50, Paul Menzel
<paulepanter at users.sourceforge.net> wrote:
> Dear coreboot folks,
>
>
> on IRC Rudolf mentioned that the A8V SE [1] works with the latest
> revision of coreboot and he asked if there is a way to tag that in the
> repository.
>
> There are several ways to accomplish that but all seem to have down
> sides.
>
> 1. Git tags. We could use `git tag <board name> <commit>` and interested
> folks could then do `git tag | grep <board name>` to find tested
> revisions. Peter wrote, that Git tags could slow down the repository and
> that only finitely many tags can be used. Would the last point be a
> problem for us?
>
> 2. Git notes. Peter suggested also to use Git notes. But Rudolf wrote he
> finds it difficult to handle.
>
> 3. Wiki. We could use the Wiki by either adding tested revisions to the
> corresponding board pages or by creating a new page with a table. The
> first solution is not feasible because not all boards have their own
> page. Patrick wrote that using the Wiki often it gets out of date pretty
> quickly. Although in this case I think that would not be a huge problem
> considering that the noted revision actually was tested. Additionally
> not a lot of developers are comfortable using the Wiki. OpenEmbedded
> once did something like that [2].
>
> 4. Provide tested images. In addition to specifying the revision such
> tested images could be uploaded somewhere so users would not have to
> build it themselves. This would not work though, since the
> infrastructure is not in place and we have to be careful with images
> containing option roms(?).
>
> 5. ROM-o-matic.net [3]. Idwer suggested a service similar to
> ROM-o-matic.net where know revisions get build by this server and users
> can configure an image to be built. That is an interesting idea although
> it is probably the most difficult to realize. Could this be a GSoC
> project?
>
> 6. File in repository. An other suggestion by Patrick and myself is to
> put a file in for example `Documentation/working-revisions.mdtext` and
> note the tested revision and board there or to put a file in each board
> directory and note tested revisions there. The downside is that people
> would have to register with Gerrit to submit changes.
>
> If we would manage our Wiki in our repository [4] options 3 and 6 could
> be combined.
>
> 7. Messages to the list. Thinking about it the easiest solution would be
> to create something like the script `alsa-info.sh`. This script collects
> the necessary information – in our case for example revision, boards,
> cbfs output, used build tools. Even better would be to run that script
> on the tested machine so also something like the tested distribution
> could be tested.
>
> Then a mbox or text file with an appropriate name/subject line is
> created
>
>        [Tested] ASUS M2V-MX SE works with revision <commit>
>
> which gets send to the list by the user or automatically.
>
> People then can search the archive. The only downside is that a nice
> table is missing.
>
> All in all I am quite surprised that no nice solutions seem to exist
> especially since I would imagine quality assurance (QA) folks in
> companies need to maintain similar data.
>
> Please comment and add your ideas.
>
>
> Thanks,
>
> Paul
>
>
> [1] http://www.coreboot.org/Supported_Motherboards
> [2] http://www.openembedded.org/wiki/Testing
> [3] http://rom-o-matic.net/
> [4] http://www.coreboot.org/pipermail/coreboot/2011-June/065706.html
> [5] http://alsa-project.org/main/index.php/Help_To_Debug
>
> --
> coreboot mailing list: coreboot at coreboot.org
> http://www.coreboot.org/mailman/listinfo/coreboot
>

As a "end-user" I'd really prefer a possibility to participate in this
issue; I think it is not possible for the few active devs to maintain
every mainboard all the time and make sure to note it in git. Also, I
don't think the coreboot git repo itself is the best place to keep
this information, nor is it the best tool for the job.

I think WINEs appdb (http://appdb.winehq.org) is a good example on how
it could be done. They have a overview site per application (for
coreboot this would be per board) where users can note a) with what
wine version (i.e. coreboot revision) they tested, b) how well it
works (gold, silver, ... rating e.g.) and c) what works and what not.
To not write a new web application, I'm sure something comparable can
be done in most wiki engines.

Another possibility might be using gerrit for such contributions. Many
Wiki engines have a git interface (though I think mediawiki has none)
that would allow commits, and using gerrit and a boilerplate it would
be easy for 3rd parties to send a new wiki-entry for review.
Choice 7 would also work instead.

thomasg