<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p>This problem is solved by setting the CPU internal clock as NON-SSC(spread spectrum clock).</p>
<p><br>
</p>
<p>PCIe spec says seperate ref clock should be Non-SSC. We only set device clock as Non-SSC before,</p>
<p>ignoring the clock at CPU side.</p>
<p><br>
</p>
<p>Now the system is stable. The"<span>slot clock configuration</span>" and "<span>common clock configuration</span>" are both still 1.</p>
<p>It doesn't seem to matter.</p>
<p><br>
</p>
<p>Thanks to you all.<br>
</p>
<p><br>
</p>
<p>Zheng<br>
</p>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Kyösti Mälkki <kyosti.malkki@gmail.com><br>
<b>Sent:</b> Friday, January 13, 2017 4:58 PM<br>
<b>To:</b> Zheng Bao<br>
<b>Cc:</b> Predrag Vidic; coreboot@coreboot.org; Zoran Stojsavljevic<br>
<b>Subject:</b> Re: [coreboot] Question about PCIe separate reference clock</font>
<div> </div>
</div>
<div>
<div dir="ltr"><br>
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Jan 13, 2017 at 6:01 PM, Zheng Bao <span dir="ltr">
<<a href="mailto:fishbaoz@hotmail.com" target="_blank">fishbaoz@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="m_-420259592890830305divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>About "<span>Asynchronous clock mode </span>on mainboard side", I guess.</p>
<p><br>
</p>
<p>Both the device and bridge have the fields "<span>slot clock configuration</span>" and "<span>common clock configuration</span>".</p>
</div>
</div>
</blockquote>
<div>Again assuming the mainboard side is CPU (some AMD SoC) and you are running coreboot and can bodify binaryPI / AGESA firmware.<br>
<br>
</div>
<div>You can spoof this in the firmware and leave "Common Clock Configuration" bit unset on both sides, even if they report "Slot Clock Configuration = 1". This affects the ASPM L0s and L1 latencies devices report in the Link Capabilities register, but may
also change hardware behaviour since specs require link retraining after bit is set.<br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="m_-420259592890830305divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>On our board, both the"<span>slot clock configuration</span>" of device (E8860) and bridge are 1.</p>
<p><br>
</p>
<p>Does it mean the "on mainboard" side it does not support "<span>Asynchronous clock mode</span>"?</p>
</div>
</div>
</blockquote>
<div>So, any specs from the mainboard or CPU side of the link? Ultimately your 100MHz clock stability and jitter is affected by the choice of your 25MHz crystal part and PCB layout, right?<br>
</div>
<div><br>
</div>
<div>Kyösti<br>
</div>
<div> </div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="m_-420259592890830305divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block; width:98%">
<div id="m_-420259592890830305divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Predrag Vidic <<a href="mailto:pvidic@gmail.com" target="_blank">pvidic@gmail.com</a>><br>
<b>Sent:</b> Friday, January 13, 2017 10:29 AM
<div>
<div class="h5"><br>
<b>To:</b> Zheng Bao<br>
<b>Cc:</b> Zoran Stojsavljevic; <a href="mailto:coreboot@coreboot.org" target="_blank">
coreboot@coreboot.org</a><br>
<b>Subject:</b> Re: [coreboot] Question about PCIe separate reference clock</div>
</div>
</font>
<div> </div>
</div>
<div>
<div class="h5">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>Hi Zheng,<br>
<br>
</div>
Schematic is simple and OK. I assume you are trying to run Wolf VPXxx-E8860-xxxx or similar GPU borad. You can't do much there if there is a hardware problem but it is most possibly a software setup problem.<br>
<br>
</div>
VPX does not have incomming clock so it is by default in async_clock mode. The question is did you allowed that mode on the main board side?
<br>
<br>
Asynchronous clock mode is not default mode for a motherboard. Check if the mode is allowed. Second, check if the other PCIe device can work on the motherboard PCIe slot.<br>
<br>
</div>
Regards,<br>
</div>
Predrag<br>
<br>
<div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jan 13, 2017 at 6:44 AM, Zheng Bao <span dir="ltr">
<<a href="mailto:fishbaoz@hotmail.com" target="_blank">fishbaoz@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="m_-420259592890830305m_-375174279812459283divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Here is the ref clk part. Please review.</p>
<p>No refclk routes to VPX connector.</p>
<p><br>
</p>
<p>Thanks.</p>
<p><br>
</p>
<p>Zheng<br>
</p>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block; width:98%">
<div id="m_-420259592890830305m_-375174279812459283divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Predrag Vidic <<a href="mailto:pvidic@gmail.com" target="_blank">pvidic@gmail.com</a>><br>
<b>Sent:</b> Thursday, January 12, 2017 6:53 PM<br>
<b>To:</b> Zheng Bao<br>
<b>Cc:</b> Zoran Stojsavljevic; <a href="mailto:coreboot@coreboot.org" target="_blank">
coreboot@coreboot.org</a>
<div>
<div class="m_-420259592890830305h5"><br>
<b>Subject:</b> Re: [coreboot] Question about PCIe separate reference clock</div>
</div>
</font>
<div> </div>
</div>
<div>
<div class="m_-420259592890830305h5">
<div>
<div dir="ltr">Hi Zheng,<br>
<br>
Without knowing the particular solution for the PCIe transmitter you have on your board, I'd check refclk pins on your design for the proper termination. The problem can be in irregular readings from the pins.<br>
<br>
Reragds,<br>
Predrag</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Jan 12, 2017 at 3:26 PM, Zheng Bao <span dir="ltr">
<<a href="mailto:fishbaoz@hotmail.com" target="_blank">fishbaoz@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="m_-420259592890830305m_-375174279812459283m_1263023594929631817divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<span>
<p><font size="2"><span style="font-size:10pt">Q: do you use local xtal attached to Si52111-B5 to generate local PCIe 25MHz clock?</span></font></p>
</span>
<p><font size="2"><span style="font-size:10pt">Zheng: yes<br>
</span></font></p>
<p><br>
<font size="2"><span style="font-size:10pt"></span></font></p>
<p><font size="2"><span style="font-size:10pt">Q: If you dothis, my next question is how you synchronize these two clocks: Local<span><br>
PCIe 25 MHz and common reference clock from CPU?</span></span></font></p>
<p><font size="2"><span style="font-size:10pt"><font size="2"><span style="font-size:10pt">Zheng</span></font>: We do NOT sync these two.<br>
</span></font></p>
<p><font size="2"><span style="font-size:10pt"><br>
Q: Since these two clocks, as I understand above scenario, are<br>
asynchronous to each other?!</span></font></p>
<p><font size="2"><span style="font-size:10pt"><font size="2"><span style="font-size:10pt"><font size="2"><span style="font-size:10pt">Zheng</span></font>:
</span></font>yes. Asynchronous.</span></font></p>
<p><font size="2"><span style="font-size:10pt"><br>
</span></font></p>
<p><font size="2">The VPX connector does not have a PCIe ref clock signal, so we can not send CPU PCIe ref clock to
<br>
</font></p>
<p><font size="2">device. The PCIe spec says if the separate refclk on devices should be 100MHz <span>±</span> 300PPM, with SSC</font></p>
<p><font size="2">off. We believe our board meet this requirement. So we doubt the problem lies in PCI configration space.</font></p>
<p><font size="2"><br>
</font></p>
<p><font size="2">Zheng<br>
</font></p>
<p><font size="2"></font><br>
</p>
<br>
<div style="color:rgb(0,0,0)">
<div>
<hr style="display:inline-block; width:98%">
<div id="m_-420259592890830305m_-375174279812459283m_1263023594929631817x_divRplyFwdMsg" dir="ltr">
<font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Zoran Stojsavljevic <<a href="mailto:zoran.stojsavljevic@gmail.com" target="_blank">zoran.stojsavljevic@gmail.com</a><wbr>><br>
<b>Sent:</b> Thursday, January 12, 2017 9:22 AM<br>
<b>To:</b> Zheng Bao; Predrag Vidic<br>
<b>Cc:</b> <a href="mailto:coreboot@coreboot.org" target="_blank">coreboot@coreboot.org</a><br>
<b>Subject:</b> Re: [coreboot] Question about PCIe separate reference clock</font>
<div> </div>
</div>
</div>
<div>
<div class="m_-420259592890830305m_-375174279812459283h5"><font size="2"><span style="font-size:10pt">
<div class="m_-420259592890830305m_-375174279812459283m_1263023594929631817PlainText">
Hello Zheng,<br>
<br>
For decades, I've been FW/SW engineer, but I do understand a little<br>
bit of a HW. I have looked into the Si52111-B5 data sheet for<br>
clarification.<br>
<br>
My problem here is to understand, your use case: do you use local xtal<br>
attached to Si52111-B5 to generate local PCIe 25MHz clock? If you do<br>
this, my next question is how you synchronize these two clocks: Local<br>
PCIe 25 MHz and common reference clock from CPU?<br>
<br>
Since these two clocks, as I understand above scenario, are<br>
asynchronous to each other?!<br>
<br>
Please, clarify for us your use case.<br>
<br>
Thank you,<br>
Zoran<br>
_______<br>
<br>
On 1/12/17, Zheng Bao <<a href="mailto:fishbaoz@hotmail.com" target="_blank">fishbaoz@hotmail.com</a>> wrote:<br>
> Our VPX design uses separate reference clock source, which is Si52111-B5 (No<br>
> spread), instead of common ref clock from CPU.<br>
> Now The system is unstable. Reading PCIE configuration space is unstable<br>
> too. (If we add some fly wire to make it work with common ref clock, the<br>
> system becomes stable.)<br>
><br>
> (abstracted from PCIe spec: 12 Slot Clock Configuration - This bit indicates<br>
> that the<br>
> component uses the same physical reference clock that the<br>
> platform provides on the connector. If the device uses an<br>
> independent clock irrespective of the presence of a reference<br>
> clock on the connector, this bit must be clear.<br>
> For a multi-Function device, each Function must report the<br>
> same value for this bit.)<br>
><br>
> Based on my understanding, the BIOS need to read bit "Slot Clock<br>
> Configurationclear" to see if<br>
> separate ref clock is used. BIOS then write bit "Common Clock<br>
> Configuration".<br>
><br>
> On our board, the bit "Slot Clock Configuration" is always 1, which I assume<br>
> should be 0.<br>
><br>
> My question is, how the hardware affect the bit "Slot Clock Configuration"?<br>
> How do we need to design our board to make the bit "Slot Clock<br>
> Configuration" be 0?<br>
><br>
> Thanks.<br>
><br>
> Zheng<br>
><br>
><br>
</div>
</span></font></div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
--<br>
coreboot mailing list: <a href="mailto:coreboot@coreboot.org">coreboot@coreboot.org</a><br>
<a href="https://www.coreboot.org/mailman/listinfo/coreboot" rel="noreferrer" target="_blank">https://www.coreboot.org/<wbr>mailman/listinfo/coreboot</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</body>
</html>