[coreboot] HT chains fixup

[Myles Watson]

> -----Original Message-----
> From: Marc Jones [mailto:marcj303 at gmail.com]
> Sent: Thursday, November 20, 2008 11:13 AM
> To: Myles Watson
> Cc: Coreboot
> Subject: Re: [coreboot] HT chains fixup
> 
> On Wed, Nov 19, 2008 at 9:16 PM, Myles Watson <mylesgw at gmail.com> wrote:
> >
> >
> >> -----Original Message-----
> >> From: Marc Jones [mailto:marcj303 at gmail.com]
> >> Sent: Wednesday, November 19, 2008 4:11 PM
> >> To: Myles Watson
> >> Subject: Re: [coreboot] HT chains fixup
> >>
> >> Myles,
> >> Hi, sorry to take so long to look at this. I have some possibly dumb
> >> questions.
> >
> > No problem.  I appreciate the review.
> >
> >> What is gained by knowing the ht path?
> > When a HT chain is powered on, all the devices have the same UnitID (0)
> > The subordinate busses get assigned bus numbers in the order that they
> are
> > found by PCI scan, but they are found in the order of the chain.  As you
> > assign each a device number, another device is visible at device 0 until
> the
> > end of the chain is found.
> >
> 
> But I think that the unitid scan happens before and device/dts is used
> or called. This has to be done to set the HT link speed. See
> northbridge\amd\k8\incoherent_ht.c. By the time the device code starts
> they are setup and the bus can be scanned as normal pci.

The problem is HTX slots and modules that can't be initialized without some
code.

> >> Every HT bridge/tunnel under a
> >> CPU acts like a PCI device. Linux doesn't know about HT and I am not
> >> sure that the dts needs to.
> >
> > That's why I made ht devices morph into pci devices at chain enumeration
> > time.  Once they're assigned UnitIDs it doesn't matter where they are in
> the
> > chain.
> >
> 
> Because it is a unitid bus number it is already numbered. I think that
> the device code should use any  bus number already set. That would be
> equivalent to setting the bus number with pci_x@ in the dts.

That could happen.  The problem is how you pick the "correct" early
enumeration.  It's easy to pick an enumeration that lets you see all the
busses and optimize the HT links.  It's not easy to make the busses get
numbered correctly, which is why you need the dts information.

> >> The exception being that coreboot needs to
> >> know the CPU that the bus comes off of.  I feel like I am missing
> >> something obvious that you, Ron, and others understand.
> >
> > I'm getting just as much push back from everyone else.  I'm not sure
> anyone
> > understands the code that well.  I feel silly for having to rewrite so
> much
> > of it.  It probably points to my lack of understanding.
> >
> >> What are you describing with and ht@? For example:
> >> +                     ht at 0,a {
> >> +
> > /config/("southbridge/amd/amd8132/pcix.dts");
> >> +                     };
> >> +                     pci_a at 0,1{
> >> +
> > /config/("southbridge/amd/amd8132/apic.dts");
> >
> > ht at 0,a is ht chain 0, UnitID a.  I wanted it to be explicit where the
> chains
> > end and what the numbers are.  The way the code works is that it takes
> the
> > HT device and all the PCI devices that it implements off the list at
> once.
> > Without knowing which devices are HT, that's hard to tell.  The code
> before
> > used a heuristic and a lot of #if statements.
> >
> 
> How does the code treat an ht device differently at this point?

When it sets the UnitID, it also sets the function numbers of the children,
since they are offsets from the UnitID.

> > The reason I put pci_a here is that otherwise device 0,1 of the 8132
> > collides with 0,1 of the 8111 (their dts names are the same.)
> >
> > The reason for specifying the chain number is that devices could be
> inserted
> > in a lot of places, and it is nice to be able to specify which one you
> mean.
> > Think of an 8-way box where you could replace any of the processors with
> a
> > co-processor.  It gets ugly in a hurry.
> >
> 
> So shouldn't you treat it like any bridge? ( I think this is how
> bridges should be treated.)

coprocessors aren't necessarily bridges.  Mine isn't.

> pci at 0,1{
>   /config/bridge dts
>   pci at 0,0{
>     /config/device dts
>   };
> };
> 
> Since ht bridges are getting special names should pci bridges? pcib at x,y

They don't need them unless you want to override automatic bus numbering.

> >> Another nit I have is that pci_18_0 is the top device. Physically it
> >> is but logically it isn't. The dts doesn't make sense when you compare
> >> it with the lspci. I don't know if this is really better but maybe
> >> something like this:
> >
> > My next patches will move everything from being a child of 18.0 to being
> a
> > child of the domain.  That's assuming I ever get agreement for what I'm
> > doing :)  I plan to have it look like this:
> >
> > domain {
> >        /k8/domain
> >        ht at 0,a
> >        ht at 0,6
> >        pci at 18,0
> >        pci at 18,1
> >        pci at 18,2
> >        pci at 18,3
> >        pci at 19,0
> >        ...
> > };
> >
> 
> So a ht link off of another cpu  will be a new domain? Do you know how
> the 8151 will be defined in the Serengeti dts?

No.  I'm not sure I understand domains yet, but I've been told you can
implement multiple domains in a k8 or fam10.

from the lspci: 

08:01.0 Host bridge [0600]: Advanced Micro Devices [AMD] AMD-8151 System
Controller [1022:7454] (rev 14)
08:02.0 PCI bridge [0604]: Advanced Micro Devices [AMD] AMD-8151 AGP Bridge
[1022:7455] (rev 14)
09:00.0 VGA compatible controller [0300]: ATI Technologies Inc RV280 [Radeon
9200] [1002:5961] (rev 01)
09:00.1 Display controller [0380]: ATI Technologies Inc RV280 [Radeon 9200]
(Secondary) [1002:5941] (rev 01)

The 8151 will be:
ht at 1,1 (link 1, UnitID 1)
pci_uniquifier at 1,0 (8151 AGP bridge)
 
and the dtsnames will then be:
domain_0_ht_1_1
domain_0_pci_uniquifier_1_0

> 
> >> >    domain_0_pci_18_0_ht_0_6(PCI: 00:06.0): enabled 1 have_resources 0
> >
> > This will be domain_0_ht_0_6 if I get my way.
> 
> great
> 
> >
> >> What if this were a real PCI to PCI bridge? Would it be
> >> domain_0_pci_18_0_pci_0_6?
> >
> > Yes, but there's no such thing with an Opteron, right?  It would have to
> be
> > another level down in the hierarchy like domain_0_ht_0_6_pci_6_0
> (remember
> > that HT UnitID and PCI dev are the same, so ht_0_6 becomes pci_6_0.)
> >
> 
> I meant if the ht bridge that acts like a pci bridge were a real pci
> bridge. I think then it is domain_0_pci_18_0_pci_0_6.


?  The 8111 pci is a real pci bridge.

> 
> >> >     domain_0_pci_18_0_ht_0_6_pci_0_0(PCI: 01:00.0): enabled 1
> >> have_resources
> >> > 0
> >>
> >> Why isn't this domain_0_pci_18_0_ht_0_6_pci_1_0_0?
> > Bus numbers get assigned as they're discovered, so they aren't mentioned
> in
> > the dts.  That's why the 8111 has to be at 0x6 and the 8132 has to be at
> > 0xa, otherwise the 8132's busses get numbered first, and  the
> southbridge
> > ends up on bus 3.  Actually the 8111 just has to have a lower number.
> 0x0
> > and 0x2 also work.
> >
> 
> This is what I noted above. Devices under a bridge need to be in the
> bridge brackets so you know where they are regardless of bus
> numbering.

Maybe this is related to my previous misunderstanding?
 
> >> >     domain_0_pci_18_0_ht_0_6_pci_0_1(PCI: 01:00.1): enabled 1
> >> have_resources
> >> > 0
> >> ...
> >> >    domain_0_pci_18_0_pci_6_1_0(PCI: 00:07.0): enabled 1
> have_resources 0
> >>
> >> I really don't understand this one. It is a device on bus 0 so what
> >> does 6_1_0 mean?
> >
> > It's device 1 function 0 of the pci_6 ht device.  Remember I had to add
> the
> > _6 so that it wouldn't have the same name as pci_a_1_0.  I agree it's
> > confusing.  I couldn't think of a really great way to do it.  Maybe just
> > taking out the extra underscore would help:
> >
> > domain_0_pci6_0_1
> > or naming it explicitly
> > domain_0_pciht6_0_1
> >
> > I'm open to suggestions.
> 
> This still doesn't make sense to me. It is on the silicon that happens
> to be a HT device but it is just a pci device on bus 0. Think of it in
> terms of any other motherboard device. The device is always at the
> same location in the system. This one happens to be set to unitid+1. I
> think that you still set it in the dts as pci at 0,7.

I think you meant pci at 7,0.  With ht@ it's link, unitid.  With pci it's dev,
fn (dev == unitid)

If we later have a hierarchical dts so you don't have to specify all these
numbers (since they don't change), we'll want them to be offsets.

> >> > Here are some of the next steps:
> >> > - Get video working.
> >>
> >> Do you know where the breakage is?
> > I think it's in resource allocation.  It's really hard to follow,
> though.
> > In the end the 8132 claims the address space that should be for the
> video
> > buffer, and it never gets to the card.
> 
> Is the Subtractive or VGA bits set in the 8132 bridge or is the BAR
> claiming the area?
The BAR seems to be claiming the area.

Thanks,
Myles