[LinuxBIOS] RESEND: v3 patch for console functions, and mods to lib/lar.c to support a void * parameter for running functions.

[ron minnich]

I am restarting the CAR discussion a bit. I want to get a resync.

One part of the discussion involves the idea that I am trying to avoid
return from disable_car because of some "bug" in our code. It is not
that simple. The issue is that, when the code enters disable_car,the
CPU has been using cache as ram, and using "memory" at non-memory
addresses. In disable_car, the CPU is supposed to disable cache as
ram, and continue -- should it return? The return is on the stack. The
function that called disable_car has its own stack variables. They're
actually in cache, not ram. So if the code is going to return, it will
need a working stack. It will need to restore the stack from cache to
ram. But where is the stack? at c000 in one case. Where is ram?
Somewhere else. So it is possible that variables on stack are wrong,
if they are pointers to variables on the stack, and there is the
problem of changing esp, ss, and then fixing up the stack for the RET.

Here's a useful comment:
x86/car/cache_as_ram_post.c:
/*
        FIXME: I hope we don't need to change esp and ebp value here,
so we can restore value from mmx sse back
                But the problem is the range is some io related, So
don't go back
        */

If you look at the cache as ram for the amd and x86, you will find
something interesting: it does not go back.
x86:
        leal    _iseg, %edi
        jmp     *%edi


For AMD, at the end of the mainboard cache as ram main, it calls
post_cache_as_ram(void). At the end of that function is this:
        /*copy and execute linuxbios_ram */
        copy_and_run();
        /* We will not return */

in other words, once CAR is disabled, we chain to copy_and_run, and
never return. We've also set up a new ESP.

To put it simply, the most current CAR code *does not return*
--because getting that right, for all CPUs, is tricky and not always
possible.

So, expecting a return from disable car is not really in line with
what we do now. In fact, what we do now on most of v2 -- certainly on
opteron -- is merely what I'm proposing we do on V3.

I'm pasting in some old mail now, with my comments in[[[]]]

=================================================================
Marc Jones wrote:
>
> Carl-Daniel Hailfinger wrote:
>
>> Dumb question: We know the location of the stack when we enter
>> disable_car(). We also know that all memory belongs to us. Can we copy
>> CAR stack contents to some safe location and restore them after wbinvd()?
>>
>
> This shouldn't be needed. If the stack needed to be moved you would want
> to do it before the wbinvd so you were copying from cache to memory.
> Much faster than memory to memory.
>
> Marc
>

and I am too smart for my own good.....


Here is the deal. Look in cpu/amd/model_lx/cache_as_ram.inc.
/* If you wanted to maintain the stack in memory you would need to set
the tags as dirty
[[[ Good point here! For the wbinvd to push the stack out, we need to
set cache tags!
    This is going to be tough on some CPUs -- how do we do it on LX?
]]]
         so the wbinvd would push out the old stack contents to memory */
/* Clear the cache, the following code from crt0.S.lb will setup a new
stack*/
wbinvd

For LX we didn't need to maintain the stack or any variables beyond CAR.
[[[[ But on v3 we do .... ]]]]
That meant that we didn't need to do any stack copy like the K8
post_cache_as_ram()(that never returns). The LX cache_as_ram_main
returns and the LB copy function is back in cache_as_ram.inc. Since
there is nothing on the stack and everything is running out of the ROM a
new stack is easily created.

[[[ Please note the comment: "that never returns"]]]

[[[ and, note: if you look at that code, round about line 313, you'll
notice that it does not return either]]]

Sooo, if you want to maintain the stack in v3... I left a comment in the
 v2 code. Recall that they cache is disabled so the way doesn't get
reallocated and thus the tags are never marked dirty and written back.
Since the tags are never marked dirty the wbind won't wb but it will invd.

[[[meaning: stack cache lines will be invalidated but NOT written back ...]]]

There are a couple ways to address this.
1. copy the stack to a new location.
2. Set the tags dirty with by writing the way MSRs.
[[[ This is how you set the tags ]]]
3. enable the cache and copy the stack back on it's self to dirty the tags.
[[[This might work too to set the tags BUT -- the stack really has to
be running at the right location. We would have to change this.]]]

I am least sure about number three.

I think that sums it up. I still think that it is best/fastest if the
code returned to the CAR function and it setup a new stack in memory and
then launched the next stage.

[[[ Which is pretty much what I am proposing]]]
=============================================================
I hope this additional discussion is helpful.


Now, getting back to where we were: I am still arguing for what Marc
argues in the last paragraph, stating "I still think that it is
best/fastest if the
code returned to the CAR function and it setup a new stack in memory and
then launched the next stage.".

Carl-Daniel, I believe you are hoping we can do the return from
disable_car, but as Marc points out in options 1-3 above, it's really
not simple. This is not a "bug" per se. Fixing it will be complex. It
is possible that you will need different solutions for each CPU
*implementation* in some cases, not just each CPU type -- for example,
the Intel CAR had to be changed for AMD, and different types of
Opteron may well need different CAR return code.

Thanks

ron