Expansion interface: rules

Before Christmas Kevin and I discussed the expansion header rule-set that I posted last time. I ended up giving him a little clarification. What follows is a summary. Keep in mind that this is a work in progress. Once we finish the new expansion routing for rev C we will post a summary and use case examples.

Figure 1. A View of the MEC8-RA Expansion Header on Titan.

Trace Lengths and Matching All signal trace lengths must be matched to within 20mils and may not exceed 2000mils. Additionally each differential pair in the bank will be routed as a differential pair each pair's N and P net will be length tuned within 5mils. This tuning is intentionally over-kill to reserve as much margin as possible for the routing on modules.

TX and RX pair assignments All LVDS transmit pairs must be connected to the same side of the expansion connector. Figure 2 shows and mapping that meets this rule. This rule is included because the Samtec MEC8-RA connector as longer pins on one side of the connector. Figure 3 was taken from the MEC8-RA's STEP file. My rough measurements show that the top pins are about 100mils longer than the bottom pins. We can avoid accounting for this difference by routing the ECP5's true LVDS transmitter pairs on one side (labelled as pads A and B) and the receiver pairs (labelled as pads C and D) on the other side.

Figure 2. An TX/RX Mapping Example (TX on the Long Side, RX on the Short Side).

Figure 3. A Top and Bottom Pin Pair from the MEC8-RA.

DQ Pin assignments Each DQ group must be matched to the same set of diff pairs; Within the each DQ group the DQS pair must be on a fixed pin. Figure 4 shows an example DQ mapping. This rule means that the individual DQ pair assignments are not fixed and may be routed differently to ease routing.

Figure 4. An Example Mapping of the Two DQ Groups from an ECP5 Bank.

Other Fixed Pin Assignments The bank's VREF net and PCLK on C/D pads must be routed to fixed pins.

Expansion interface

We are finalizing our plans for rev C of Titan now. Over the last week Kevin has been working on our current change list: layout adjustments to ease manufacture, tighter tuning of the expansion header traces, and the addition of a 1.2V power supply for the SerDes (PCIe) transmitter and receiver. Kevin is working in his fork of the Titan project. Once he completes his changes I'm going to make a few adjustments to the DDR3 layout. When we are ready to order we will pull the changes back into the master branch of Titan on the C-E-S organization Github page.

While Kevin is updating the layout, I've been focused on validated our changes. Initially I was working on using the Diamond's Reveal embedded logic analyzer to identify the PCIe issues. Now that I know that there isn't a PCIe core in the samples we have, I'm focusing my efforts on validating the DDR3 memory design and the external interfaces.

Figure 1. Titan Rev B with Two Expansion Modules Installed.

Our expansion headers bring out a single bank of the ECP5 FPGA to a single connector. Now that we've had more time to consider possible expansion modules it has become clearer how we should organize the connections from the FPGA's I/O banks to the connectors. We are trying to balance keeping the layout simple and clean with our desire to standardize the expansion header pinout to make our future modules interchangeable from one Titan header to another.

Each FPGA bank that we have connected to an expansion header can be configured as 32 single-ended GPIOs, 16 differential GPIOs, two DDR data groups, or 7:1 geared interfaces (ie. CameraLink). Lattice's ECP5 High-Speed I/O Interface guide describes the interface possibilities. One specific interface that we are studying that has applications with SDR is the Linear Technology LTC217x quad ADC.

This leads me to my list of features that need to be the same from one expansion header to another. These assignments are based on the expansion differential pair nomenclature used on the rev B Titan schematic (see Figure 2).

Figure 2. Titan Expansion Header (showing our simple nomenclature).

Titan Expansion Header Features

• One PCLK pair must assigned to a fixed diff pair (ie. DO_P/N)
• Each of the two DQS pairs must be assigned to a fixed diff pair (ie D8_P/N and D12_P/N)
• Each DQ group must be matched to the same set of diff pairs (ie D0/1/2/3/9/10/11_P/N)
• VREF must be assigned to a fixed net (ie. D1_P)
• All LVDS transmit pairs must be connected to the same side of the expansion connector (ie. D0/1/2/3/4/5/6/7_P/N)
• All trace lengths must be matched to within 20mils and may not exceed 2000mils.

Now we'll see if I've given Kevin an impossible set of requirements. If I have I'm sure that I'll hear about it.

Stupid mistakes

Through my career I've had the pleasure of working with some intelligent and talented engineers and scientists. Along the way I've learned to listen when I'm given advice. I might decide not to heed their advice, but I always listen. The two most memorable nuggets of advice that I've received are:

1. Don't be a candy ass
2. Don't sweat the stupid mistakes, it's the smart mistakes that should worry you

Tonight I'm trying hard to remind myself of #2. The principle is simple; We all make silly mistakes, overlook things, or just forget something. As long as we identify our mistakes quickly and move on it shouldn't be a problem. The bad mistakes are ones where you focus on something for a long time and come to the wrong conclusion. These smart mistakes often mean that you really don't understand the problem.

While debugging our two rev B boards, we've seen several successes. Our external expansion headers are working, the power rails all work, and the USB programming circuit all work. Unfortunately we were not able to get the PCI Express interface to operate. We found three issues that prevented PCIe from working:

1. From testing on the Lattice ECP5 PCI Express card, I showed that my implementation would only enumerate the PCIe interface on Diamond 3.2, not 3.3. This isn't too big a deal since I can just go back to 3.2 for now.
2. We did not include LDOs for the SerDes (PCIe) transmitter and receiver. We were able to repurpose an LDO on Titan to provide the 1.2V as required by the ECP5 SERDES/PCS Usage Guide. This LDO wire-mod can be seen in the LED blinky video (below) on the left side of the ECP5. The mod worked well, and I verified that the noise level at the decoupling caps nearest to the ECP5 was similar to that measured on the Lattice's ECP5 card.
3. The third issue keeping PCIe from working on Titan can be seen in the blinky video. Watch it again at 720p or higher as see if you can find it. I recommend watching it in full screen.

Did you see it? The engineering samples that we used on the rev B boards were part number LFE5U-85F-8BG381IES. I noticed this when I was building a DDR3 test project tonight. The correct part number is LFE5UM-85F-8BG381IES. What does the M mean? SerDes. We are testing parts without the SerDes (PCIe) interface included.

Second build of rev B

As Kevin mentioned yesterday, we were able to build a second rev B Titan using the ECP5 that we reclaimed from our rev A build. I'm still working to get the PCI Express interface to enumerate, but the build went well as the video above shows.

We've found a few clear errors in the PCB that we need to fix and we've already started some preliminary work on rev C. I'll outline what we found as soon as we have a more complete list.

New oscilloscope

If you follow both of our blogs, you may know that Kevin has recently organized a home lab and added some new equipment. I've been working on mine as well and I made a different decision regarding oscilloscopes: The Rigol DS1074Z.

I wanted a newer scope with deep memory and advanced trigger modes. I was especially interested in a scope with an Ethernet interface compatible with the VXI-11 standard. I'm hoping to use Python to unify my simulation and testing environments and this scope should help. I've already found an interesting python project, python-vxi11 and used it to communicate with the scope (see Figure 1).

I've also posted a gratuitous picture of my home office/lab to compete with Kevin (Figure 2).

Figure 1. Identify Query and Response from My DS1074Z.

Figure 2. The Scene of the Crime (see And the sparks fly...)

PCIe enumeration

Today was a day for good news and even better news. I passed the Titan PCBs to Kevin to review our options to get the board up and running, and I focused on the firmware any why the PCIe interface didn't enumerate on the Titan prototype.

Since the titan_wiggle project was a fork from ecp5dev_wiggle, I decided to validate that the Lattice ECP5 PCI Express Card enumerated properly on my test PC; It did not enumerate. I know that ecp5dev_wiggle enumerated in the past, but now I'm wondering if I failed to check PCIe enumeration at every step during development. The last two significant changes were the Diamond update from 3.2 to 3.3 and the pairing down of the project I did before I posted it to Github.

I followed IPUG112_01.2 closely and rebuilt the PCIe core in the Lattice Clarity tool in Diamond 3.3 and the PCIe interface still did not enumerate on the ECP5 dev card. I then rebuilt the project in Diamond 3.2 and the board enumerated properly. That explains why Titan didn't enumerate since it was running a direct fork of ecp5dev_wiggle in Diamond 3.3. I'm going to contact Lattice and see what might be wrong with my 3.3 project, but for now I can test Titan in Diamond 3.2 as soon as we have another working Titan.

On the PCB front, Kevin has been working working on an idea that we have to repair our Titan prototype. When I was last inspecting our rev A prototype the ECP5 popped of the board. Popping off wasn't that surprising since the rev A's pads were so small that we had virtually no ball collapse. Kevin's review of the part showed that we should be able to use it to replace the (possibly) damaged ECP5 on our rev B board. His next step is to isolate the LDOs on shorted rails from the ECP5 to determine if the failure is in the ECP5, the LDOs, or both.

And the sparks fly...

I'm still trying to wrap my head around it, but unfortunately I destroyed our only working Titan prototype while testing it. The build went very well, and I got a lot of good data from testing the card. While Kevin and I consider our next steps, I'll review how we got here.

Building

The rev B board assembly went very smoothly. I simplified the build by not loading the USB programming circuit, and I was able to place and reflow the board myself (including the two BGA parts). The only items that required manual rework after the build were two twisted capacitors and the switching power supply IC in a QFN package at U10. U10 also skewed in the same direction on the rev A build so we are going to examine this issue more deeply later.

Figure 1. U10, Skewed by a Pad.

We decided to simple remove U10 and bypass the 12V to 3.3V power supply and use the PCI Express connector's 3.3V rail instead. With U10 removed, this just required a simple mod-wire.

Testing

Prior to powering the board I measured the impedance to ground on each voltage rail:

Rail	Location Measured	Impedance
3.3V	C86	6 kOhms
2.5V_A	C60	4.8 kOhms
1.1V_A	C31	4.3 kOhms
1.1V	C59	79 Ohms
2.5V	C67	4.1 kOhms
1.5V	C27	2 kOhms

     Table 1. Impedance Measurements.

Rail	Location Measured	Voltage
3.3V	C86	3.472 V
2.5V_A	C60	2.49 V
1.1V_A	C31	1.099 V
1.1V	C59	1.096 V
2.5V	C67	2.480 V
1.5V	C27	1.499 V

     Table 2. Voltage Measurements.

Using a Lattice USB programmer, I used Diamond to validate that the ECP5 was identified with a JTAG scan. I then loaded my titan-wiggle test project and two LED expansion boards to validate That 64 I/O pins (32 to each expansion header) all work as my video shows.

Next I rebooted the Linux PC that was powering Titan and used the lspci command see if the PCI Express interface enumerated; It did not enumerate. I was able to validate the PERSTn (the PCI Express reset signal) and REFCLK were being received by the FPGA. Titan-wiggle uses PERSTn as its reset; I simply issued a warm reboot (linux reboot command) and saw that the LED counter reset during the PC's reboot. I then changed the LED counter from the 50MHz oscillator to the 125MHz output from the PCIe core and the LED counting pattern speed increased as I expected.

This is when the damage to Titan occurred. The mod-wire bypassing U10 was connected to a 3.3V via very close to the PCIe connector. With Titan installed into the PC and powered I accidentally nudged the board and the mod-wire popped off the via. The wire was under tension and swung across the card, most likely contacting the 12V rail on nearby caps. This would have shorted the ECP5's 3.3V rail directly to 12V. I saw a spark and a little smoke. As we all know, it is nearly impossible to return all of the smoke to an IC once it escapes :p  I powered off the unit and checked the voltage rails and found that the 3.3V, 2.5V, and 1.1V rails were all shorted to ground.

While this is disheartening, mistakes do happen, and hopefully we can recover quickly. For now I will be focusing on firmware development using the Lattice PCIe Development Kit and developing theories about why the PCI Express interface did not enumerate

More signs of life

I used two Dione LED boards and the titan_wiggle FPGA project to validate that all of the expansion I/O pins work. Next step, PCIe!

Titan Lives!

All of the power rails on the rev B board come up correctly and the FPGA is detected in Diamond via JTAG!

More details tonight.

Rev B, assembled

Figure 1. Titan Rev B, Assembled.

I just finished assembling a rev B board. I decided to leave of the on-board USB programmer for now to simplify the build. I'll use an external USB programmer to program the ECP5 if the board checks out.

Rev B arrives

Figure 1. Titan, Now with Bigger Pads!

Figure 2. BGA Pad Size Comparison (Left: Rev A, Right: Rev B).

Rev B arrived today. If all goes well I'll assemble a board tonight!

Another picture of rev B

Two more pictures from PCB-Pool just arrived. These show the surface finish of the PCBs being built:

Figure 1. Titan rev B Top Side (Surface Finish).

Figure 2. Titan rev B Bottom Side (Surface Finish).

Titan rev B Moves Along

Rev B of Titan is moving along nicely at PCB-Pool. I just received these plots:

Figure 1. Titan rev B Top Side.

Figure 2. Titan rev B Bottom Side.

Test setup

I've been giving some thought to the best development platform for Titan. I'm trying to optimize several criteria at the same time:

• Low cost
• While this seems self-evident, I'm not purely interested in cheap. I could always find a cheap PC with a PCI Express (PCIe) slot and use it. I'm looking for an inexpensive platform that also meets my other needs:
• Repeatability
• I'd like to find a platform that I can replicate in a hurry. The idea here is for Kevin and I to have similar setups that can be built up quickly and that we can share easily.
• Accessibility
• Accessibility is all about sharing. If I can create a common test platform that Kevin and I are both familiar with we will end up spending less time configuring test systems and more time developing HDL and hardware.
• Future Development Modeling
• I'm also interested in using test platforms that may model future FPGA/CPU pairings that show promise. This leads me to consider non-Intel platforms.
• Easy to Use with Linux

Usually I end up using one PC with my UUT (PCIe express card that I'm testing) and another PC with the JTAG programmer and other GUI based development tools. 

For Titan I'm considering moving to an environment where a small, low-cost platform is used to provide the PCIe link to Titan as well as the USB ports for any development tools that I need (JTAG programmer, logic analyzer, etc.). The plan is to use the USB/IP open source project to share the USB ports from the PCIe host platform to a my laptop (or Kevin's) where I can run the GUI based Lattice Diamond or SaleAE logic analyzer software. Figure 1 shows this idea.

Figure 1. Titan Development Configuration Idea.

I'll have to do some testing with USB/IP to see how practical this is, but I'm confident enough that I went ahead and ordered some hardware:

• SolidRun HummingBoard-i2eX
• $119.00 with AC power supply and 8GB micro-SD card.
• Bplus PM2 PCI-E / Mini PCI-E adapter
• $40. This is a little pricey and seems like a negative at first. I looked for a simple ARM-based processor board few low-cost options had PCIe slots (most had mini-PCIe like the Hummingboard). My long term plan is to build my own mini-PCIe to PCIe adapter with a GPIO input capable of power cycling Titan.

Making adjustments

Figure 1. Rev A of Titan, Assembled.

I posted a summary of our build experience with our first rev of Titan on the CES blog yesterday: Building Titan.

We completed the rev B updates to the artwork, released it on GitHub, and ordered new PCBs today. The changes are listed in the release notes on GitHub.

Building Titan

Kevin and I spend the afternoon building Titan. We placed every part except the Lattice ECP5 FPGA. Since supply of the FPGA ICs are limited we are going to have a local contract manufacturer install it and X-Ray it to validate the solder quality.

We should be ready test later next week!

Figure 1. Placing Parts on Titan

Figure 2. Titan after Placing Most of the Small Passives.

Figure 3. Ready for Relow!

Titan has arrived

We received Titan from PCB-Pool today!

Figure 1. Titan rev A.

Titan is curing

I love PCB-Pool. They send pictures of your PCB during the assembly process (including the two latest pictures below). We will have the boards in our hot little hands early next week.

Figure 1. Top View of Titan

Figure 2. Bottom View of Titan

Titan is public!

I just finished posting rev A of Titan to http://github.com/c-e-s/titan. The rev A release may be found at http://github.com/c-e-s/titan/releases. I will write a longer post for the CES blog explaining the organization of the project. Look for the post in the next few days!

First open source PCB commit

I just posted Enceladus to the CES github account: https://github.com/c-e-s/enceladus.

Enceladus is the 0.1" header break-out board for the expansion headers used on Titan. I ordered this board awhile back. Since it was so simple Enceladus was the perfect board to practice a release. Let me know if anything looks odd since I expect to post rev A of Titan in a few days!

Oh, the lights!

My heart grows three sizes every time I see an FPGA make LEDs blink! This simple project has been checked into CES's github account: https://github.com/c-e-s/ecp5dev_wiggle

I will create new repos under the c-e-s organization as I practice building simple projects using Diamond 3.2 for the Lattice ECP5 PCI Express board.

DDR3 initial route complete!

I finished routing and tuning the DDR3 circuit last night. My branch of the Titan PCB layout is now in Kevin's hands for re-integration into trunk. I'll post some screen caps of the layout when the integration is complete. My next task will be to build a test project for the Lattice's new ECP5 PCI Express Board. When I get it working I'll port the code to Titan and validate our pinout.

Until then, enjoy our latest post at the CES blog: Lattice's ECP5 PCI Express Card

DDR3 data group tuning

I moved half of the upper byte to layer 4 and tuned the tracks. I was shooting to tune each byte to within 100mils, but I ended up beating that without really trying. The lower byte is tuned within 32 mils (see Table 1) and the upper byte is tuned within 21 mils (Table 2).

Figures 1 and 2 (below) show the routes on layers 4 and 6. All of the tracks in the data groups except for the DQS differential pairs are shown; They are routed on layers 1 and 3.

	Pre-tuned length (mils)	Post-tuned length (mils)
D0	859	1205
D1	855	1185
D2	839	1178
D3	934	1210
D4	1027	1204
D5	1187	1187
D6	934	1194
D7	1016	1202
DM0	802	1202
DQS0p	1200	1200
DQS0n	1194	1194

Table 1. Pre and Post Track Lengths for the Lower Byte.

	Pre-tuned length (mils)	Post-tuned length (mils)
D8	962	962
D9	861	981
D10	877	982
D11	975	972
D12	746	961
D13	868	978
D14	803	970
D15	953	973
DM0	888	974
DQS1p	983	983
DQS1n	924	973

Table 2. Pre and Post Track Lengths for the Upper Byte.

Figure 1. Tuned Data Tracks on Layer 6 (Upper Byte and Half of Lower Byte).

Figure 2. Tuned Data Tracks on Layer 4 (Half of Lower Byte).

The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

DDR3 data routing for x16 parts

I tried to length match each byte pair last night and immediately ran into troubles. While I had left room to allow for tuning wiggles, it turns out that the mismatch is too great. Figure 1 shows the location of the lower and upper bytes in the route. My goal was to route both of the data bytes on the same layer.

Figure 1. Single Layer Data Route with Bytes Annotated.

Typically, six layer DDR3 routes require four signal layers, one ground plane, and one power plane. So far, I've assumed that if I only used a single DDR3 part I could drop the number of signal layers from four to three by avoiding the connection of control/address signals between each DDR3 IC.

Everything was going great until I started to tune the data bytes. The problem was with the lower byte (D0 to D7). Since the Lattice ECP5 requires that the same DQS group (data byte, strobe, and byte enable) go to the same DQS group on the ECP5, my route groups the bytes together. This is fine for the upper byte, but half of the lower byte (D1, D3, D5, D7, & DM0) have to swing wide to the left to escape the DDR3 IC without crossing the upper byte. This swing horribly mismatches the trace lengths within the byte:

• D0 length: 877 mils
• D1 length: 1371 mils
• D2 length: 861 mils
• D3 length: 1532 mils
• D4 length: 956 mils
• D5 length: 1619 mils
• D6 length: 892 mils
• D7 length: 1460 mils

The average mismatch between odd and even lower byte bits is 599 mils. Figures 2 and 3 show D0 before and after tuning. Since I could only remove 291 mils of mismatch, I'm doubtful that I will be able to tune the lower byte as routed.

Figure 2. Non-tuned D0 Route (Length = 877 mils)

Figure 3. Tuned D0 Route (Length = 1168 mils)

This leaves me with a few ideas about how to proceed:

1. Use a fourth signal layer to route the lower byte. While this doesn't meet my goal for such a simple DDR3 design, it looks like a pretty standard solution.
2. Try to more aggressively select ECP5 DQS groups to route the entire x16 data set. This may be possible because the DQS groups are stacked roughly in pairs of two from the top to the bottom of the ECP5 (per side). This approach might work, but it works heavily against my address/control routing method where I bring all address/control nets in on the top layer.
3. Switch to two x8 DDR3 ICs. With each byte coming from a different IC I can add more vertical space between the parts to ease routing

After taking to Kevin, I've decided to go with option 1 as long as the impact on the rest of the layout is minimal.


The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

DDR3 data lines connected

I just finished routing both DDR3 data bytes from the memory IC (left side of Figure 1) to the FPGA (right side). Using 0.125mm traces (4.92mils) made the route easy. I only had to move a few of the address/control nets. The net reassignments were required because the Lattice ECP5 requires that all data bytes be routed to the same data group (named LDQ17, LDQ41, LDQ53, or LDQ89 on the left side of the ECP5).

Next I will tune the trace lengths of the address/control and data bytes. I'm going to start with a ruleset that was developed for a 1333 Mbps DDR3 design. Since the ECP5's maximum data rate is 800 Mbps this ruleset should be overkill. If the route becomes too difficult I'll reconsider the rules.

For now my goal is to get a route that is good enough so that I have something to simulate. Here's hoping that I'll have a simulation ready design soon.

Figure 1. Data Connection (2 Bytes) from DDR3 IC to FPGA.

The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

Lattice ECP5 PCI Express Card

I received the Lattice Semiconductor ECP5 PCI Express Development Kit today! A few gratuitous unboxing photos follow:

Figure 1. The Box.

Figure 2. Lattice ECP5 PCI Express Development Card (Front Side).

Figure 3. Lattice ECP5 PCI Express Development Card (Back Side).

New address route

I finished connecting the DDR3 to the FPGA (thru the series resistors) using 0.125mm (4.92mil) traces. The route looks very clean and there is plenty of room left to tune the traces lengths. I'm going to do the initial route of the data lines next and then I'll start tuning. Figure 1 shows the new route with both routing layers shown at the same time. Figures 2 and 3 show the old, fat, route for reference.

This route was fast and I expect the data bytes to be just as quick.

Figure 1. New Top and Inner Layer DDR3 Route with Thin Traces

Figure 2. Top Layer DDR3 Route with Fat Traces.

Figure 3. Inner Layer DDR3 Route with Fat Traces.

The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

Series termination layout

While reconsidering the trace size and termination topology for the address and control bus (see Fat, fat traces), it became apparent that I needed to keep my layout as tight as possible for the series resistors. The resistors needed to be packed tightly to allow them to be placed as close as possible to the driver (the FGPA) and to limit trace length differences to each resistor.

The first solution that came to mind was the termination part used on Lattice's ECP3 Versa development card. They used a CTS Corp RT2402B7-TR7 (Figures 1-3). They are designed to be used for common termination applications such as DDR3, VME64, PCIX, etc. I always try to limit the number of new components that I have to include in CES' database, but this part certainly fits the bill as a compact termination solution.

As I studied it a bit more, I made this list:

• For series termination, the C Series looks like the right option. Figure 2 shows an RT2402 part used in a series termination application.
• 18 series resistors are in the RT2402 (C Series), this is four short of the 22 resistors that I need.
• In in smallest (1.0mm) package, the RT2402 is 4.0mm by 9.0mm. When I tried to replicate the arrangement and trace pattern in Figure 3 using discrete 0402 resistors, my pattern was approximately 4.0mm by 10.0mm (Figure 4).

While the RT2402 looks like a fantastic part for many applications, I think that well-placed discrete resistors will work well for me.

Figure 1. CTS Corp. RT2402 (from the RT2402 datasheet)

Figure 2. Series Termination with the RT2402 (from the RT2402 datasheet)

Figure 3. Escape Routing with the 1.0mm RT2402 (from the RT2402 datasheet)

Figure 4. My Series Resistor Routing Inspired by the RT2402.

The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

Fat, fat traces

As I mentioned in the last post, I decided to route the DDR3 memory using a series R tree architecture. My initial analysis showed that I needed to use very thick traces to get a 50 Ohm impedance using PCB-Pool's default stack-up. I didn't complete the route, but I got close enough that it looked possible to connect everything. Several problems became apparent:

• In Figure 1, the control traces on the lower left hand corner connecting the DDR3 IC and the series resistors had to fan out in a wide pattern to the left of the DDR3 IC. This makes the overall memory layout large given the size of the components. Since the control traces have to be length matched, the large trace distance difference between the traces connecting to the top versus the bottom resistors implies that the top traces will need a lot of tuning 'wiggles'.
• Normally the control nets (left side of the DDR3 IC in Figures 1 and 2) are routed vertically between the DDR3's fanout vias. The wide fanout that the large traces required will make re-use of this layout in designs with multiple DDR3 ICs unlikely
• The data bus routes on the right side of the DDR3 IC on the inner layer and the bottom layer (Figures 2 and 3) will be even more challenging to tune. I met my goal of routing a byte per layer, but the traces are so close that there is no tuning room left.

When I re-read Micron's design guide for point to point designs (TN-41-13) this morning, I realized that I overlooked a critical note on pages 15-16: "The tree with series R provides acceptable signal quality without having to provide VTT power when there is a mismatch between the driver and transmission line. The tree without series R option generally provides acceptable signal quality without having to provide VTT power when there is a minimal mismatch between the driver and transmission line".

My route was trying to both match the transmission line impedance (thick traces) and use series resistors. According to TN-41-13 this is overkill. I should pick one strategy or the other, not both. Since PCB-Pool's pricing is so attractive, I'm going to stick with the series resistors and use much smaller trace widths. I should be able to simulate the layout and validate if this approach will work.

Figure 1. Top Layer DDR3 Route

Figure 2. Inner Layer DDR3 Route.

Figure 3. Bottom Layer DDR3 Route.

The PCB layout shown in this post by Custom Embedded Solutions, LLC. is licensed under a Creative Commons Attribution 4.0 International License.

DDR3 Control Architectures

With Kevin working on the main Titan layout, I've been focusing on the DDR3 layout and building a simple FPGA project to validate our pin selection. I expect to post the FPGA project to my github account: https://github.com/jsloan256. Be warned, I'm still learning to use git and posting testing code; I don't guarantee that anything works. When I reach a milestone I'll post about it here and link to the github commit.

The DDR3 design on Titan will use a simple point to point topology. Unlike more traditional module DDR3 designs like one would find on a PC motherboard, point to point designs have both the DDR3 memory IC and the controller (CPU, FPGA, etc.) on the same PCB. The lack of connectors and shorter traces in point to point design allow some simplification of the DDR3 routing rules. I'm still finalizing my DDR3 rules, but I've found the Micron design guide for point to point designs (TN-41-13) to be very helpful.

Figures 1 thru 4 below show Micron's recommended address/control routing topologies. Since I'm only planning to use a single DRAM, the difference between fly-by and tree architectures is negligible. I settled on the topology on Figure 1 (tree architecture with series R) mostly because it doesn't require a VTT power rail and the signal quality should be better with the series R than without (Figure 2).

I've started routing and have hit some hurdles. I'll post some pictures of the route next time and talk about what I'm trying to fix.

Figure 1. Tree Architecture with series R and Vtt pull-up (from TN-41-13)

Figure 2. Tree Architecture with no R (from TN-41-13)

Figure 3. Tree Architecture with series R (from TN-41-13)

Figure 4. Fly-by Architecture with Vtt pull-up (from TN-41-13)