The requirements with respect to signal routing are:

• Route all differential signals at 100 Ω differential track impedance
• Use internally sink-terminated devices throughout. Any non-terminated high-speed devices need to be documented in a separate list.
• Use LVDS signalling on all connections into the FPGA fabric, for both data and clocks
• Design all LVDS interconnect for 1Gb/s signalling rate
• Use DC coupling on all LVDS lines
• Use CML signalling on all MGT lines, for both data and clocks
• Design all MGT data links for 10Gb/s signalling rate
• Use AC coupling on all MGT differential inputs and outputs, for both data and clocks
• Use CML on all common clock trees; rather than using AC coupling, observe the signalling voltage and input compatibility rules as outlined by the device manufacturers
• Use source terminated single ended CMOS signalling for JTAG and configuration lines, where required. Avoid impedance discontinuities on clock lines.
• Observe the requirements on overshoot and undershoot limitation, in particular for System ACE and FPGA JTAG and configuration lines. Use slew rate limited signals.

The requirements with respect to the opto mezzanine are:

• Provide four FMC connectors on the GOLD for connection of the opto transceivers to the input processor FPGAs
• Use identical pinout on all FMC connectors to allow for variable size mezzanine modules

- Provide 72 differential input links

- Provide 12 differential output links for test purposes

- Provide 13 single-ended 3.3V CMOS control lines (incl. I2C and JTAG),
power 2V5/3V3, ground

- Use XILINX ML605 FMC pinout for reference

• Deal with any required input signal conditioning at mezzanine level:

- Duplication

- AC coupling

The requirements with respect to data reception and conditioning are:

• Provide five MPO/MTP compatible blind-mate fibre-optical backplane connectors in ATCA zone 3
• Route bare fibre bundles to 12-channel opto receivers
• Allow for up to 3 standard sized opto receivers per FMC connector (i.e. total of 12 on a full-size mezzanine module)
• Fan out CML level electrical signals
• Supply optical and electrical components with appropriately conditioned supply voltages
• Provide suitable coupling capacitors
• Run the signal paths into the input processors
• Allow for footprint compatible medium- and high-end FPGAs with up to 36 MGT links per chip

The requirements with respect to the processors are:

• Provide an aggregate input bandwidth of 92 GB/s (payload) into the processors

- 144 channels of up to 6.4Gb/s line rate

• Process the real-time data in a 2-stage (maximum) processing scheme (input and main processor)
• Minimise latency on chip-to-chip data transmission
• Maximise bandwidth between the two processing stages

- Send output data to CTP on MGT links, so as to maximise low-latency inter-processor communication bandwidth

- Use higher latency channels for non-RTDP links where possible

- Use CPLDs as “port expanders” for low-speed signals

• Use spare resources on the main processor for board level control functionality
• Provide an aggregate bandwidth of up to 8 GB/s (payload) on MGT outputs towards the CTP

- 12 channels of up to 6.4Gb/s line rate

- Additional 12-channel low latency electrical (LVDS) port

The requirements with respect to the clock distribution on the main board are:

• Provide the FPGAs with clocks of multiples of either a 40.08MHz crystal clock, or the LHC bunch clock
• Provide a common MGT clock to all FPGAs.
• Provide a second MGT clock to the main processor
• Provide a separate MGT clock to all HXT FPGA devices
• Connect the MGT clocks to the LXT FPGAs such that the central quad of 3 quads is supplied. This rule must be followed such that the requirements are met, whether the smaller or the larger device is mounted.
• Provide two common fabric clocks to all FPGAs
• Provide a separate crystal based MGT clock to the main processor for use on the control link
• Provide a separate crystal based MGT clock to the main processor for use on the DAQ and ROI link outputs (40.00 MHz or multiple)
• Provide a separate crystal based 40.08 MHz (or multiple) MGT receive reference clock to the main processor on the receive section of the DAQ and ROI links, for use on future LHC bunch clock recovery circuitry (TTCDec replacement), and thus:
• Allow for the input portion of the DAQ and ROI transceivers to be used for optical reception of LHC clock and data

The requirements with respect to the clock generation on the mezzanine board are:

• Receive a TTC signal from backplane zone 2
• Receive a TTC bunch clock through an external connection to a TTCDec module, at LVDS signalling standards
• Provide for single ended connectivity to the TTCDec to receive TTC data (broadcasts, counter resets) and control the TTCRx chip via I2C
• Allow for insertion of clock conditioning hardware into the clock path
• Provide for a fall-back to a 40.08MHz (or multiple) crystal reference

The clock mezzanine module provides some connectivity and real estate for general control purposes. The requirements with respect to auxiliary controls on the clock mezzanine board are:

• Receive a CAN signal from backplane zone 2
• Receive a single USB signal pair from backplane zone 2
• Route the USB signal on to the JTAG mezzanine, if it isn’t handled on the clock mezzanine
• Receive a 4-pair Ethernet signal from backplane zone 2
• Connect the clock mezzanine to the main processor via a single MGT for purpose of module control
• Connect the clock mezzanine to the main processor via 12 LVDS pairs
• Communicate to the control CPLD on the main board

The requirements with respect to boundary scan and CPLD configuration are:

• Allow for the connection of a boundary scan system to all scannable components of the GOLD: FPGAs, CPLDs, System ACE, and mezzanine modules via standard JTAG headers, following the standard rules on pull-up, series termination, and level translation between supply voltage domains.
• Allow for CPLD (re)configuration, following the boundary scan tests, and occasionally on the completed system.
• There is currently no requirement known regarding integration of devices sourcing or sinking MGT signals externally, into the boundary scan scheme

Pre-configuration access and device configuration of the FPGAs is required at any time. USB access and the System ACE / CompactFlash configuration scheme are employed.

The requirements with respect to USB access and FPGA configuration are:

• Employ the standard System ACE configuration scheme to configure the FPGAs upon power-up
• Connect the System ACE external JTAG port into an USB-to-JTAG converter according to Xilinx ML605 configuration scheme
• Connect the USB processor to a front panel mini USB socket
• Alternatively connect the USB (data lines only) via the clock/control mezzanine module to the zone2 backplane connector
• Allow for static control of the FPGA configuration port settings and read-back of the status via the control CPLD.

The requirements with respect to general board control are:

• Provide an optical bidirectional channel from the front panel to a the main processor MGT (control link)
• Provide four-lane access from zone 2 to the clock/control mezzanine, compatible to 10/100/1000 Ethernet, so as to allow for a Ethernet Phy to be mounted on the mezzanine module
• Provide two-lane access from the mezzanine on to the main processor (one MGT, bidirectional)
• Provide bi-directional connectivity between input processors and main processor via a subset of the general routing resources otherwise used for the real-time data path
• Provide unidirectional transmission from each input processor to the main processor via four MGT lanes for additional monitoring bandwidth
• Provide a 12-wide single-ended 2.5V-CMOS bus between main processor and control CPLD
• Have the CPLD act as a port expander for low-speed and static control

The CPLD is in charge of mainly static controls and low speed I2C fanout.

The requirements with respect to the CPLD are:

• Communicate to the general board control system via a 12-wide bus to the main processor.
• Communicate to the opto transceivers located on the mezzanine module, via I2C and static lines
• Communicate to the on-board optical transceivers (12-channel Avago and SFP)
• Communicate to any CPLD located on the clock mezzanine module
• Communicate to the IPMB-A port via I2C protocol
• Communicate to the System ACE sub-system so as to control FPGA configuration and allow for in-situ update of the CompactFlash card
• Control the static FPGA configuration control lines

• Provide an optical ROI output channel to the front panel
• Provide an optical DAQ output channel to the front panel
• Use standard SFP modules
• Allow for wiring to either MGT or fabric resources of the main processor via the clock/control mezzanine module
• Provide a separate 40 MHz (or multiple) clock to the MGT quads driving DAQ and ROI fibres

The rules with respect to signal integrity are:

· Use low-noise local step-down regulators on the module, placed far from susceptible components.

· Observe power ramp and sequence requirements for FPGAs

· Run all supply voltages on power planes, facing a ground plane where possible, to provide sufficient distributed capacitance

· Provide at least one local decoupling capacitor for each active component

· For FPGAs, follow the manufacturer’s guidelines on staged decoupling capacitors (low ESR) in a range of nF to mF

· Observe the capacitance limitations imposed by the voltage convertors

· Minimise the number of different V_CCO voltages per FPGA to avoid fragmentation of power planes

· avoid large numbers of vias perforating power and ground planes near critical components

· avoid impedance discontinuities on single ended clock lines

· source-terminate single ended clocks with approximate impedance

· Route all differential signals on properly terminated, controlled-impedance lines:

o Have all micro strip lines face a ground plane

o Have all strip lines face two ground planes or one ground plane and one non-segmented power plane

o avoid sharply bending signal tracks

o minimise cross talk by running buses as widely spread as possible

o Avoid in-pair skew, in particular for MGT links and clocks

o Avoid impedance discontinuities and stubs, in particular on MGT links and clocks

o Use LVDS on all general-purpose FPGA-FPGA links

o Use LVDS on all GCK clock lines

o Use PECL or CML on all MGT clock lines

o Use CML on all MGT data lines

o Use AC coupling on all MGT clock inputs

o Use AC coupling on all MGT data inputs

o For all off-board links assume capacitors off-board

o For on-board links place capacitors close to source

o Use AC coupling when crossing PECL/CML domains

o Use AC coupling or suitable receivers when crossing 2.5V/3.3V domains, except on LVDS

o Use bias networks on AC coupled inputs where required

o Run FPGA configuration and FPGA JTAG clock lines on approximately 50 W point-to-point source terminated lines

· Connect all Vccaux and all Vcco pins of a given FPGA to one 2.5V plane

· Connect all Vccint pins of a given FPGA to one power plane of 1.0+/-0.05V

· Ramp-up requirement on all voltages 0.2ms to 50ms

The rules with respect to general I/O connectivity are:

· Tie Vccaux and all bank supplies to 2.5V. A given FPGA is supplied by only one 2.5 V plane.

· Use all FPGA banks for LVDS and low-current, point-to-point 2V5 CMOS only

· On any external single ended links that might generate ringing at the FPGA inputs, use series resistors or slew-rate limited, or low-current outputs

· No reference voltages nor DCI termination are required for single ended general I/O. Use respective dual-use pins for I/O purposes

The rules with respect to pre-configuration and configuration control pins are:

· The VFS pin is the fuse programming supply voltage. Allow for wiring to GND (0Ω) or external 2.5V supply. Wire to GND.

· Allow HSWAPEN to be jumpered to either Vcc or GND

· Allow mode lines M0, M2 to be jumpered to either Vcc or GND. Pre-wire to Vcc

· Connect M1 to the CPLD (GND=JTAG mode, Vcc=slave serial)

· Connect PROGRAM, INIT and DONE lines to the CPLD

· Pullup DONE 330Ω, INIT 4k7 PROGRAM 4k7

· Connect Vbatt to GND

· Wire DIN, DOUT and CCLK (series terminated) configuration lines to the CPLD

· Allow for wiring RDWR and CSI to either Vcco or GND

The rules with respect to system monitor pins are:

· Do not use temperature sense lines DXN,DXP and short them to GND

· Decouple analog power and GND according to UG370 with ferrite beads and wire the system monitor for internal reference (both Vref pins to analog GND)

· Do not use analog sense lines Vn and Vp and connect to analog GND

Further remarks

· Check 3.3/2.5V domain interfaces

o clock trees

o JTAG

o I2C

o Parallel I/O in general

· Check in-pair skew

o MGT data 5ps ?

o MGT clocks ?

o GCK clocks ?

o General LVDS resources ?

·