[coreboot] QA: Notification of newly introduced and fixed warnings

[Patrick Georgi]

Am 24.02.2013 23:35, schrieb Paul Menzel:
> True. But as we are not there yet and I just found out that Jenkins
> already checks for fixed and introduced warnings [1], maybe this could
> be added nevertheless to the Jenkins report Gerrit displays.
That data is mostly useful for the non-Gerrit builders, since history is
linearized there.

>> The main problem is that jenkins' coreboot-gerrit task is non-linear: 
>> Introduce tons of warnings with one proposed change in gerrit, and the 
>> next change build-tested by jenkins has no problems to pass...
> 
> Sorry, I did not understand. Could you give an example? If every build
> is done from master and the patches with dependencies then this should
> not be a problem, should not it?
It is a problem for a couple of reasons:

1. Jenkins has no notion of "master + patches", but of commit ids
2. Jenkins' warnings analysis works relative to the prior build.

The first issue can be worked around: Just walk back from the commit id
until you end up with a commit that is equal to, or a parent of master.
However you still have to hope that any analysis data is still around -
it's legal to push commits to gerrit that branched off against 2 year
old upstream.

The second issue goes deeper:

Consider the recent build history of the coreboot-gerrit builder:

#4857 built 2502,2 (meaning: change 2502, patchset 2)
#4856 built 2501,2
#4855 built 2501,1
#4854 built 2500,1
#4853 built 2497,2

This means that the statistics tell you how the warnings count changed
from 2497,2 to 2500,1 to 2501,1 to 2501,2 to 2502,2. That's not very useful.

The only warning analysis that's somewhat useful is for #4856, since it
can tell you if there are any changes to the warnings count compared to
the earlier patchset of the same change.

Of course, a change like you propose would be useful, but I doubt it's
possible within the current architecture of Jenkins.
The plugin integrating Gerrit into Jenkins also requires changes to be
able to push custom data into the build reports, and it might be
necessary to expose such data within the Jenkins framework first.

I guess they'd accept patches, but I won't be the one writing them.


Patrick

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://www.coreboot.org/pipermail/coreboot/attachments/20130225/63a892a0/attachment.sig>