Expansion interface: the tuning begins

I finished connecting all of the expansion nets tonight. The current route is committed on my Titan Github page. The principle trade-off was that I had to use some long traces. Figure 1 shows the current state of the route. I've also listed the minimum and maximum trace lengths for each DDR data group in Table 1. As I start tuning the trace lengths, I may re-route some nets to make the design easier to tune (both by balancing layer usage and shortening the longest routes).

Figure 1. Expansion Route (All Nets Connected, Pre-tuning).

Expansion Header P3			Expansion Header P5
A_DQ0 min	1239.646		B_DQ0 min	1718.488
A_DQ0 max	3388.985		B_DQ0 max	2966.341
A_DQ1 min	1059.547		B_DQ1 min	1248.02
A_DQ1 max	1604.663		B_DQ1 max	3499.98

Table 1. Net Minimum and Maximums.

Expansion interface: the depature

My attempts to adjust the expansion header routing by changing pin assignments minimally to make the two headers pinout compatible has turned into an unbounded problem. I decided to change tack. I wiped the current routing, assigned the nets manually, and defined a few groupings to direct the routing.

Within each group I've allowed myself varying level of routing flexibility depending on how critical the net locations are. So far it is going well, hopefully I'll have time to complete the routing this weekend analyze the results. A summary of the working groupings are listed below in priority order:

DQ Group 0 & 1   Each expansion header is connected to a bank that contains two DSQ groups. One of these DQ groups contains the PCLK inputs for the bank as well as the VREF net; This DQS is designated as DQ0 and routed to the long side of the expansion header

PCLK, DQS0, & DQS1   The PCLK input and data strobe (DQ0 & DQ1) differential pairs are assigned to fixed locations that cannot be changed. These nets are routed first and are only allowed layer transitions at the ECP5 and at the expansion header (never more than two). Figure 1 shows my initial route of these nets.

DQ0 pairs 0 & 1 / DQ1 pairs 0 & 1   The first two data pairs adjacent to the data strobes must be "true LVDS TX" pairs from the ECP5 (identified as A/B pairs). They are labeled DQ0-0, DQ0-1, DQ1-0, and DQ1-1. Any A/B pair from the same DQ group may be used. I was able to route these pairs only using vias at the ECP5 and/or the expansion header, like the highest priority pairs (see Figure 2). I'm not sure if we should allow more or not, but since the routing went well I don't have to worry about that today.

DQ0 pairs 2, 3, & 4   The last three differential pairs for each data group (DQ0-2, DQ0-3, DQ0-4, DQ1-2, DQ1-3, DQ1-4) are all routed to ECP5 C/D pairs. Each set of three pairs within the same DQ group (ie. DQ0-2, DQ0-3, & DQ0-4) may be switched to each routing. I have not yet routed these pairs on Titan, but from my examination of the current state of the route (Figure 2) I've concluded that I will have to allow additional via transitions to route all of these pairs (hopefully no more than one additional via).

VREF & D0   The final two nets in the DQ0 data group will be routed as single ended nets (not differential). This is because the VREF net is on a diff pair true net on one bank and on a compliment net on the other bank. There is no common routing that will allow these nets to be differential and have the VREF pin on the same expansion header pin.

D1, D2, D3, & D4   Since the DQ1 data group does not contain a PCLK input or VREF, it has four "extra" nets to route. These will all be routed as singled ended nets.

Figure 1. Top Priority Nets Routed.

Figure 2. Second Priority Nets Routed.

Expansion interface: the re-route

Over the holiday I've been considering how painful it will be to re-route the expansions headers on Titan to meet (or approximate) the rules that I defined last week. The answer: Really, really, painful.

The current route is layer efficient and has very few layer transitions (both fantastic qualities). Unfortunately, this natural matching of an ECP5 bank to an expansion connector places DQ groups/strobes, PCLK inputs, and the VREF all over the place. Table 1 shows my analysis of the pinout comparison between expansion headers P3 and P5. I realize that this table is a little busy, but I was trying to look at several different factors at the same time. I considered posting the color version, but it is a rather terrifying to behold.

P5 Function			P3 Function							P3 Function			P5 Function
DQ	Pair	Oth	DQ	Pair	Oth	Top Side Net	Pin #		Bottom Side Net	DQ	Pair	Oth	DQ	Pair	Oth
						3.3V	2	1	V_I/O
						3.3V	4	3	V_I/O
						GND	6	5	GND
89	C		41	C	PCLK	D0_P (S0)	8	7	D8_P (S16)	41	A		89	A
89	D		41	D	PCLK	D0_N (S1)	10	9	D8_N (S17)	41	B		89	B
						GND	12	11	GND
89	A		41	A	DQS	D1_P (S2)	14	13	D9_P (S18)	41	C		89	C
89	B		41	B	DQS	D1_N (S3)	16	15	D9_N (S19)	41	D		89	D
						GND	18	17	GND
89	A		17	A	DQS	D2_P (S4)	20	19	D10_P (S20)	17	A		89	A	DQS
89	B		17	B	DQS	D2_N (S5)	22	21	D10_N (S21)	17	B		89	B	DQS
						GND	24	23	GND
53	C	PCLK	41	C		D3_P (S6)	26	25	D11_P (S22)	17	C		89	C
53	D	PCLK	41	D		D3_N (S7)	28	27	D11_P (S22)	17	D		89	D
						GND	30	29	GND
						GND	32	31	GND
53	C		41	A		D4_P (S8)	34	33	D12_P (S24)	17	C		53	C
53	D		41	B		D4_N (S9)	36	35	D12_N (S25)	17	D		53	D
						GND	38	37	GND
53	C		17	A		D5_P (S10)	40	39	D13_P (S26)	17	C		89	C
53	D		17	B		D5_N (S11)	42	41	D13_N (S27)	17	D		89	D
						GND	44	43	GND
53	A		41	C	VREF	D6_P (S12)	46	45	D14_P (S28)	17	A		53	A
53	B		41	D		D6_N (S13)	48	47	D14_N (S29)	17	B		53	B
						GND	50	49	GND
53	A	DQS	41	A		D7_P (S14)	52	51	D15_P (S30)	17	C		53	A
53	B	DQS	41	B		D7_N (S15)	54	53	D15_N (S31)	17	D		53	B	VREF
						GND	56	55	GND
						n/c	58	57	n/c
						n/c	60	59	n/c

Table 1. P3 & P5 Pin Mappings.

Figures 1 and 2 show the scale of the problem. Figure 2 especially shows just how clean the current route is. I spent quite awhile looking at different pinouts where where only the most critical nets were routed to the same connector pin. The scale of the change quickly becomes so severe that it doesn't look possible without adding quite a few layer transitions (bad for signal quality) and possibly using more PCB layers (bad for PCB cost).

Figure 1. The Spaghetti Monster.

Figure 2. Routing Layers for the Expansion Interface.

I only have one knob left to twist: function. I have a hunch that only routing about half of the nets as differential pairs will make it much easier to route the rest as single-ended nets. I have a few use-cases in mind. Hopefully I'll find a way to map them to a rational pin mapping.

Reading this you might wonder why I'm taking so much time with this re-planning of the expansion interface. The first is simple: I believe that an intelligent pin assignment will ease the design of modules. The second is all in the schedule: We have our first order of ECP5 FPGA's on the way, but the lead time is long enough for us to give a careful review of the entire design before placing our next PCB order. We intentionally side-stepped the tricky problem of the expansion header design in our first two PCB revs. Now that the rest of the design is stable (or nearly stable), it's time to reconsider this part of the design.