[coreboot] K8 HT architecture

[Carl-Daniel Hailfinger]

On 25.10.2008 02:33, Marc Jones wrote:
> Carl-Daniel Hailfinger wrote:
>> On 24.10.2008 20:14, Marc Jones wrote:
>>> Carl-Daniel Hailfinger wrote:
>>>> Hi,
>>>>
>>>> I'm trying to understand how HT is modeled into PCI space so that I
>>>> can
>>>> propose the "right" way to handle it in the dts.
>>>> Depending on whether I run lspci -t under coreboot or factory BIOS,
>>>> different topologies will be displayed. That means looking at lspci is
>>>> not going to tell me how the hardware really works.
>>>>
>>>> Given a standard setup with three HT links from the CPU, where do I
>>>> find
>>>> which device?
>>>> Is PCI device 18.0 sort of a PCI bridge which has multiple PCI
>>>> buses (HT
>>>> links) behind itself?
>>>> Or is the hardware organized in a completely different way? I'm
>>>> especially curious about the MP scenario as depicted above. Where
>>>> do the
>>>> PCI functions of 18.[0123] reside?
>>>>
>>> This is why you don't want to mix the ht links and the pci bus. The
>>> physical layout and the logical layout on pci are different. the lspci
>>> -t view is the correct view.
>>
>> Except that it sometimes differs (at least in numbering) between
>> coreboot and factory BIOS, so I'm not sure which aspects of lspci -t are
>> immutable and which are arbitrary.
> Bus numbering is up to the BIOS.

Ah thanks.


>>> That is how the devices are addressed. Several different physical
>>> layouts will get you the same logical layout because of how pci buses
>>> are scanned.
>>
>> Interesting.
>>
>>> this
>>> cpu --- pci
>>>  |
>>> cpu --- pci
>>>
>>> would be equivalent to this (although unlikely topology)
>>> cpu --- pci
>>>  | \--- pci
>>> cpu
>>>
>>> I think it would be a mistake to over complicate the dts with the
>>> physical connections.
>>
>> Is the "pci" in your graphics above a bus or a device?
>> From your answer it seems that "pci" represents a bus. Do 18.[0123],
>> 19.[0123] and other processor devices all live on PCI bus 0?
> yes
>
>
>>  Is that bus somewhat related to the buses attached via ncHT? 
> in that ncHT looks like a pcibus

Good, then we can treat it as a PCI bus in our device tree.


>> IIRC I once saw a machine where 18.[0123] was alone on bus 0 and all
>> other PCI devices
>> were on separate buses.
>>
> There has be someother device on bus0. CPUs are not pci bridges.

That's strange. I do believe you, but that would imply at least one PCI
device from each ncHT link has to be visible on bus 0.


>> Taking the "everything is PCI" model, how would I specify the virtual
>> PCI buses attached via ncHT? Are they children (secondary buses) of 18.0
>> which would act like a PCI-to-PCI bridge?
> it is another pci bus off of root. Just like bus0 and just like it
> looks  in lspci -t. Tom's email describes this in more detail.

OK, then we should model it that way in v3.


>>> They don't know (or need to know) what ht links are connecting the cpu
>>> and what ones go to pci bus.
>>
>> They need a way to specify settings for any given PCI device. Since most
>> modern machines have multiple PCI devices with the same vendor/device
>> ID, we have to be able to identify devices based on their logical path.
>> For that, we have to model the logical PCI bus/device/tree reasonably
>> well. I'm trying to do that, but no model seems to fit.
>
> I disagree. How would a two devices of the same type require
> difference initialization? I think that defining where the devices are
> at build time is wrong. You only need the ID and the functions that
> device requires.

The best example I saw was multiple PCI express bridges with the same
ID, although only some of them were used by physical PCIe slots or
onboard PCIe devices. Factory BIOSes usually hide the unconnected
bridges and I'd be happy if we could do the same. AFAICS that requires
different initialization of devices with the same type.


Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/