Comments by the reviewers and the GOLD designers
answers - May 4, 2011
===========================================================
By Sam, on the requirements:
1 Concept and use
If this is to be a test bench for the TP, GOLD
should also be able to function as a data source for other connected
subsystems, including CTP and ROD.
------------> yes
2.2 USB connectivity for pre-configuration access
Why is this on the backplane in Zone 2? Would we
ever expect to have USB on the backplane in a "real" environment?
Proposal: front-panel USB access from laptop
------------> added a note on
pre-configuration access. Needs to be discussed separately wrt future L1Calo modules, I would suggest. I might try and
start a discussion on that soon.
2.3 Clock pair
Do you mean TTC/GBT?
------------> yes
2.5.2 Signal duplication at CML level
Is there a possibility to rearrange the
distribution of duplicated signals to the different FPGAs? (Here I am thinking
about testing quadrant/global architectures with full backplane readout vs.
single-FPGA global merging with reduced-bandwidth CMM++ output)
------------> yes. opto mezzanine board will be cheap and rather passive (apart
from fanout chips) allow for several daughter
versions during GOLD lifetime
2.6 FMC sockets
2.12 Where do the LXT electrical outputs go?
------------> front
panel 12 pairs, LVDS, via clock daughter. note:
connectivity shared with other potential
use of differential pairs on clock daughter. In case DAQ/ROI links would
fail to operate with MGTs, two of these pairs would be taken for that purpose,
using de-serialisation in fabric/IOB rather than MGTs
Do the FMC sockets also need to provide power to
the input mezzanine modules, or are there other arrangements for this?
------------>yes, they do
2.17, 2.18 Two independent pairs of MGT and GCK
clock trees
Are these two independent clock trees are generated
on-board from the single "clock" pair in Zone 2? (I see this is
answered in 2.19)
------------>yes
2.23 Location of external connectivity? (Might be
answered later)
It would be nice if we can connect an LVDS cable
that could be brought to a DSS or the CTP.
------------> yes. connector type to be chosen later as "final" clock
daughter is produced
3 ATCA compliance
3.6 DAQ/ROI issues
The TP will almost certainly need to be able to
read out data correctly to the existing RODs, if only for the R&D phase. So
we need to be fully compatible, meaning a 40.00 MHz crystal clock should be
brought to the FPGA that produces the Glink-compatible
outputs.
So I see that a 40.08 MHz crystal clock is already
connected to E-U65. If there is space I would also like a 40.00 MHz crystal
added there as well (or keep the option to swap the 40.08 MHz crystal with
another one)
------------>yes
--------------------------------------------
By Yuri:
on the requirements:
http://www.staff.uni-mainz.de/uschaefe/browsable/L1Calo/GOLD/GoldURD1.htm
The following use cases apply to CMM++ and I've
tried to formulate a possible requirements (relevant to
CMM++) to the GOLD for these use cases:
4. Test bench for topological algorithms
-
Demonstrate upgraded L1Calo data processing in a single TP module in a single
FPGA
-
Demonstrate upgraded L1Calo data processing in multiple TP modules (in single FPFAs)
6. Optical data sink for L1Calo modules
-
Demonstrate suitable parallel fiber modules and
transceiver chipsets
-
Demonstrate data synchronization from multiple data sources
7. Optical data source for purpose of self-test and
stand-alone link tests
-
Demonstrate electrical data replication in FPGA and optical distribution
-
Demonstrate passive optical data regrouping
-----> I would think that this is not immediate
requirements on the capabilities of the hardware to be built. It is rather an
expansion of the use cases. The tests that will actually be performed on this
piece of hardware are not in the scope of the review. But we will happily
expand the use cases by these ones. I would believe that the current
requirements and specifications describe a system that actually allows for
above use cases.
further clarification: 2nd line of 4. seems
to refer to data sharing between TP modules. This will be possible by optical
splitters for the LXT based sub-system, and by sharing electrical backplane
signals for the HXT sub-system.
Specification:
http://www.staff.uni-mainz.de/uschaefe/browsable/L1Calo/GOLD/GOLDspecs1.htm
1.2.1
Real-time data path
The module incorporates two distinct processing schemes.
One is based on currently available FPGAs (XC6VLXT series); the other one tries
to make use of FPGAs (XC6VHXT)...
>>> How FPGA can define the processing
scheme? I didn't find two mentioned schemes description...
>>> Are they local-global and all-in-one
schemes?
------> symmetric vs.
hierarchical (input processors plus main processor)
Figure 1 shows a conceptual drawing of the GOLD
(main components only). The processors are labelled A to D for the input
processors, M for the main processor,... More detailed
diagrams are shown in section 3.
>>> Where is M on the Fig.1? Didn't find
it in section 3 either...
------> the labels are
inconsistent in text and drawings. Corrected.
1.2.1.1 Input processing
As the number of electrical input channels may be
higher than the number of optical inputs, the electrical signals are routed
from the o/e converters into CML fan-outs, and then onto the mezzanine
connectors. This will allow for evaluation of electrical duplication schemes
that future algorithms will require.
>>> In order to run algorithms in several
FPGAs on the same TP module?
------> yes, algorithms
might require some data sharing
Such a concept allows for several instances of a
topological processor to be run with different signal routing, optimized for
the respective algorithms.
>>> Would it be easier to route signals
inside FPGA rather than on different flavors of the
mezzanine modules?
------>
for a processor with more than a single FPGA we'd have to care about data
routing at PCB level, unless cross bar switches were used
1.2.1.2 Main processor
This theoretical limit cannot probably be reached,
since a fraction of lanes might be required for control purposes, as the main
processor dubs as a module controller. Bandwidth can be increased when larger,
foot print compatible FPGAs are mounted on the PCB.
>>> Will foot print compatible FPGA have
the same number of IO? How then the bandwidth can be increased?
------> XC6VLX550T has in fact an additional 120
lines parallel I/O wrt LX240. However, that statement
has been removed meanwhile, since the main purpose of foot print compatible
devices is maximisation of MGT input aggregate bandwidth.
>>> It looks like the local-global
partitioning with small FPGA artificially creates bandwidth limitations...
------> unfortunately
yes. For the prototype/production module we'd have to decide what number of
inputs we really need. The less the
input bandwidth is divided up in individual chips the better.
1.2.3.1 Pre-configuration access
The microcontroller is interfaced to the FPGAs via
JTAG protocol. The microcontroller design is fully compatible to the Xilinx
platform USB solution.
>>> Therefore you can just directly
connect the Xilinx platform cable USB to the JTAG port (without the
microcontroller on a mezzanine module)?
------> yes. Using a passive mezzanine. We'd equip future modules with
direct USB access, if the scheme works, unless someone comes up with a better
approach. We would not want to buy a separate Xilinx USB box for every module
manufactured though.
1.3 Opto module
At a later stage opto modules with higher input
link counts and higher optical fibre densities will be built, targeted at
setups for system level tests with CMM++ modules.
>>> Comment: for the CMM++ tests a sink
with six 12-channel opto receivers and source with six 12-channel opto
transmitters may be required
------> The previous should not pose any
problem. We have currently got mounting space for 5 MTP/MPO connectors, and 24-way
connectors seem to be rather common by now, therefore no problems expected when
connecting 72 fibres. That number of fibres would actually just fit a single
MTP/MPO if the glossy paper tells the truth. Though, that's just the input
lines. Please note that the GOLD is mainly a demonstrator for data concentrator
purposes many-in, few-out. The optical output from the input processors is
intended for self test only (see use cases). Just as you would probably use the CMM++ inputs in conjunction with
the CMM++ outputs for self tests.
1.5 HXT processor
The HXT devices suffer from a split bit rate range
of up to 6.5 Gb/s for the slower MGTs, and a non-overlapping 10Gb/s only for
the faster transceivers on the same chip.
>>> Is it a real problem?
------> probably, since
you need to know beforehand what signals to route where. Flexibility lost.
Whether this is considered a serious issue is up to the reader. It is
definitely inconvenient.
The HXT subsystem is not within the scope of this
specifications document
>>> Does it mean that there will be no HXT
based version of GOLD?
------> Remains to be seen. Comparing the
currently known projections for V6H and V7 one might possibly not be bothered
to dig into V6H. If we were to go for V6H we would definitely give the current
GOLD 1.0 a try, but would then possibly have (to have) a further iteration on
that. GOLD has grown beyond what one would probably want to have on a topo processor module and we'd rather shrink it a bit in
complexity on the way towards topo prototype and
production.
-----------------------------------------------------------------
By Ian :
Requirements
------------
Section 1, paragraph 2:
I think the term
"demonstrator-for-everything" is misleading. I think GOLD is better described
as a multi-purpose demonstrator or a module intended to demonstrate
technologies used potentially throughout the trigger.
----------> accepted
Section 1, use case 9:
I don't think this "others" clause adds
any information. I'd rather see potential uses named explicitly or not
mentioned at all.
----------> well, that
was just a placeholder for what we might have forgotten about. Since noone asked for additional uses we can just as well remove
that.
Section 2, requirement 19.1:
I'm concerned that this requirement could be
interpreted as GOLD providing an interface for the TTC _or_ the GBT, when I
think you intend to provide an interface for the TTC _and_ GBT. To clarify
this, and as these clock schemes are quite different, I'd like to see this requirement
split into two, one for the TTC and one for the GBT.
---------->what we meant to say is, that for a
given module version just one of the options will be available. We will have a
clock mezzanine module with an interface to the TTCrx
system, and we will at a later stage have a mezzanine with a GBT
interface.
Section 2:
Should there be a requirement to provide access to
the System ACE micro-controller interface?
----------> We do in fact plan to attach the SystemACE micro controller interface to the Main processor
via a CPLD. Therefore nothing speaks against adding a requirement on that.
Specification
-------------
Section 1.1:
To clarify, I think you should say explicitly it is
the Phase-1 upgrade scenario you are describing here.
---------->accepted
Section 1.2, paragraph 1:
Rather than describe GOLD as a demonstrator for the
"topological processor and other future L1Calo modules", I think it
is truer to say it demonstrates technologies that potentially could be used
throughout L1Calo.
---------->we could well change the wording that
way.
Section 1.2.3.3:
This is not really an issue for the demonstrator,
where I can put up with it, but in future I'd be happier if the monitoring
interface wasn't implemented in the main FPGA. It looks too much like a single
point of failure.
----------> accepted. In fact, in case someone were
planning to implement somthing similar to the current
Fujitsu CAN scheme for future L1Calo modules, we would happily take such a
solution on board. We just want to avoid descending into that
ourselves. On future clock mezzanines for the current GOLD we might well add
such circuitry.
Section 1.4:
I infer the use of the GBT here, incoming via the
SFPs; should it be mentioned explicitly? It would also be nice if the L1A,
incoming from the TTCdec, was mentioned explicitly. I
can't visualise how the TTCdec is connected
mechanically to GOLD and the clock module, but perhaps that's just my problem.
----------> we could
well mention the GBT here. It isn't actually clear to the GOLDers
whether we can expect anything GBTish during GOLD
lifetime. I believe that somewhere we are saying that we are providing a
sub-set of the TTC data. We should perhaps mention a few signals there,
including L1A. As for the TTCdec-on-GOLD we thought
we could possibly hide that from the reviewers ;-) We were originally concerned about the space
taken and therefore assumed to actually have it on some external module with a
cable connection to the front panel. Though by now we rather
envisage the following scenario: Initially just the upper half of GOLD will be
assembled with FPGAs due to unavailability of HXT devices. Therefore
there is plenty of space for a large clock mezzanine. At a later stage we would
rather experiment with optically supplied LHC clock/data, be it GBT (if
available) or some GBTish home grown scheme.
===================================================
The documents have already been updated to reflect
Sam's and Yuri's comments. Ian's comments will be dealt with at a later date.