[coreboot] [PATCH] [v3] Fix ROM caching on Via C7

[Marc Jones]

On Tue, Apr 14, 2009 at 9:56 AM, Corey Osgood <corey.osgood at gmail.com> wrote:
> On Tue, Jan 6, 2009 at 12:31 PM, Marc Jones <marcj303 at gmail.com> wrote:
>>
>> On Tue, Jan 6, 2009 at 7:39 AM, Myles Watson <mylesgw at gmail.com> wrote:
>> > On Tue, Jan 6, 2009 at 4:52 AM, Corey Osgood <corey.osgood at gmail.com>
>> > wrote:
>> >> Tested and working on hardware :D
>> >
>> > Great job!
>> >
>> > As I read your patch I wondered if there's a good reason to disable
>> > caching.  Can we just take out the #ifdefs?  Or maybe make it an
>> > #ifndef DONT_CACHE_AND_RUN_REAL_SLOW_ON_C7? :)
>> >
>> > Acked-by: Myles Watson <mylesgw at gmail.com>
>> >
>> > Thanks,
>> > Myles
>
> Ooops, this never got committed, figured that out very quickly last night.
> r1164
>
>
>>
>> I think that we need to examine this for all the CPUs. In a legacy
>> BIOS all the runtime code will be in memory, so caching the ROM is
>> only need prior to memory setup. That is not the case for coreboot
>> because we have the XIP printk and LAR code. We should use one MTRR
>> for caching the ROM. So MTRR for ROM caching is setup by the CPU
>> specific CAR code and then later in normal MTRR setup we add an MTRR
>> that will support XIP for the rest of POST and payload loading. There
>> are two minor drawbacks to doing this.
>>
>> 1. You use an MTRR that could be used by the OS. No big deal, there
>> are more than enough MTRRs for every confguration I can think of.
>>
>> 2. You might use cache space for ROM code that will fetch slowly and
>> might only run once. This is no longer a problem with large L2 and L3
>> caches. Even on Geode it should be ok. Also, the ROM shouldn't be
>> accessed once the OS loads.
>>
>> The last thing is if the ROM should be cached read-only or
>> write-through. If set RO, flashrom will need to be MTRR aware and set
>> it to WT when writing ROMs.
>
> Currently, at least with C7 CAR and I think some others, CAR uses the first
> variable MTRR, MTRR 0, and the ROM is cached at MTRR 1 by stage 0. Once the
> MTRR setup code runs, the ROM cache is killed off, and things slow down. I
> see a few solutions:
>
> 1. Give ROM caching priority, MTRR code first sets up a WB MTRR 0 to cache
> the ROM, then clears out MTRR 1-8, and sets up MTRRs for RAM. This way the
> ROM is never not cached, and the code should be easier to write.
> 2. Give memory priority, MTRR code clears all variable MTRRs, sets up
> memory, and then does the ROM cache as the very last thing. If memory eats
> up all the MTRRs, oh well, the rest of the boot will be slow. Any calls to
> XIP functions between the time MTRRs are cleared and the time the ROM MTRR
> is set up will be slow.
> 3. Give memory priority, but reserve the last MTRR for the ROM cache. At
> that point you might as well go for 1.

I think that you have to do #2 to be "safe" but there are enough MTRRs
that you shouldn't have a problem. As you pointed out the real problem
is between the end of CAR and you are still executing out of the ROM.
Copying data out of the ROM doesn't help that much since the device is
slow and narrow (you can't fetch way at a time like real memory). In
V2 MTRRs get setup 3 times.
1. CAR MTRR setup
2. End CAR basic memory MTRR setup
3. MTRR setup which happens after device setup near the end.

In v2 step 2 is very basic generic setup and could be improved. From
step 2 to 3 you want the ROM cached since it copies and decompress
from the ROM. I don't think that the ROM is left cachable at this
point for K8/FAM10 and needs to be fixed.

At step 3 the devices have been setup and MTRRs need to be setup for
the devices (UMA video memory). The only ROM access after this point
is to  copy out the payload. Caching is a minimal help there and not a
big deal if you are out of MTRRs.

The last thing is that you might want to save few MTRRs for the OS to
use. I think that in v2 it is handled in step 3.

In v3 we only do step 1 and 2 for K8/FAM10 but step 2 is more advanced
than in v2. It sets the MTRRs, fixed and variable and caches the ROM.
A step 3 needs to be added but then the late init needs to be AP CPU
aware and we didn't go down that route yet.....

I hope that was clear enough to help...

Marc


-- 
http://marcjonesconsulting.com