18.05.
I am adding comments here at the end of this text file. Please check back occasionally and refresh your browser. That avoids me writing many separate mails or delaying transmission of comments. Comments are in order of appearance, not by chapters.
---------------------------------------------------------------------------------------------------
We have actually decided for slightly different crystal frequencies: 40.07x and multiples. That was based on information from MSU. Please ask Bruno on Tuesday and update.Do not update each occurence. Just say somewhere that 40.08 means 40.07...whatever
1.2.3
"Configuration and module control are achieved via Ethernet"
FPGA configuration is *not* via ethernet, but from local memories.
"Monitoring"
should we write here that we intend to send those data into DCS ? We believe that Makis is working at such a scheme for the Muons and we might be able to take that over. But please confirm with Makis !
Fixed
DAQ and ROI: do we refer here to the Cracow activities or does that come later ?
I put a reference to the Marek Talk in Birmingham.
2.4
This rule must be followed such
that the requirements are met, whether a smaller or larger device is
mounted."
remove this phrase, there are no different sizes of devices on the production modules ! Please check also in remaining document"
Fixed
---------------
There were comments by Weiming on certain rules regarding signal integrity on the PDR. Has that part been removed from the document altogether ? Or was it in the Checklist ? have you checked that against PDR comments ?
It's the SFF-8431 standard. Quite of a large document. I put a reference and mention it on Sec.4.2 where we talk about
link tests.
That’s not what I meant. He was commenting on eg. Positions of capacitors that we specified in our rules and he thought unnecessary. There wer e more of such comments.
-----------
further to 2.7: refer to cracow project here ? Or do we better refer to that in just one place ? We should at least say that ROD firmware is under development. In the firmware section ? Hardware wise we need to say somewhere that the output links of the embedded ROD via opto fibre is bi-directional, and that we additionally have an electrical busy output. Also need to say that for any ROI links we provide a duplicated signal. Will forward that mail to you ! We have agreed to provide some small amount of ROI bandwidth (single link, ie. 2 fibres due to duplication).
However, easiest way to send ROI data would be via the SFPs through L1Calo R-ROD and have those RODs care for duplication.
I added a sub-section in the Firmware chapter, there we mention and link the Krakow activities.
Add bi-directional for ROD.
Add the lemo out.
Add the requirements of two linkd for future ROI builder as requested by Jinlong Zang.
3.1 remove reference to VX485 T ! production module is for 690T only. Please check generally that text referring to the prototype is taken out !
with standard backplane connectors only 4 of them will fit the available space and that's 192 fibres, maximum is 288 fibres (72-way MPO)
Removed any reference to the small device.
Reworked the text on the max. fibers number.
3.2 since the reviewers did ask that previousla, shouldn't we say: in case of issues with clock phase an alternative jitter cleaner device can be mounted on the extension mezzanine ? Or is that covered further down ?
Added here.
3.3 "IPbus will be adopted if L1Calo decides in favor of that
approach." we will definitely go for IPBus and Mainz has started to work on firmware and software implementation as you are rightly saying in the next phrase.
Fixed
3.1 should not read scalable design since there are no more FPGA options ! modular design ?
Fixed
Fig.4 what is this mmcx thing ?
There are two RX lines from one AVAGO device which goes straight into mmcx, which is a diff. connector.
We can remove that evenctually. It was meant as debug thing, considering that those RX lines are in excess
compared to the number of MGT-RX.
Ok. Thanks. Good to learn what mmcx is
-------------------------------
https://indico.cern.ch/getFile.py/access?contribId=1&sessionId=0&resId=1&materialId=slides&confId=252265 slide 25: the UK suggested future configuration method !!
Link referenced.
Well, but that was mainly for your information. If we mention that we should think up a scheme on how we could possibly use that. I had asked Bruno about moving both the compact flash and the ACE chip onto common daughter to be able to replace that with something else. So that might be a place to put such a firmware SysAce thing…
----------------------------------
19.05.
----Please read through the whole document again. We have too often written that something can be added if required.It was ok for the PDR of the prototype. That's not a good thing to write for a FDR of the final design !!!. Either something is there or isn't only in exceptional cases something can still be undefined at that stage. For example we could explicitly say that "this functionality is implemented on the extension module and might therefore be replaced by ... at a later stage...
We should make sure we and the reviewers understand what's on and what isn't
I cleaned up statements like those.
------------------------------------------------------------
3.3 2nd line: why initially ? It will always be ethernet except that we have more than 1 ethernet port to choose from.
Fixed
fig.6: ATC250 is the correct name.
Fixed
fig. 2 why are the input links from miniPOD to FPGAs not shown, if that's adrawing of the RTDP ?
I planned to mention in the text, but I just re-draw it.
fig.4 that's only the non-real time links. That's not clear from the caption
Better explained
sect.4 :"As far
as control software is concerned a likely candidate for module control is the"
replace with : "Module control will be based on the"
Fixed
Fig. 8: what is the meaning of "Delays added where needed" ? Why is there an OR at the output ? That's not realistic, we have separate trigger bits per algorithm !
Some algorithms use 3BX some 4BX. Stefan told me he add a delay register to align all of them. The OR is
what he is doing right now, but I should probably just let that empty or people would get confused indeed. I update
the block scheme: remove the comment and the OR.
The caption is too long. What's the meaning of "In this example, muons sorted/selected
TOB still need to be included in the generic R(lep,jet) algorithm." ?
The DR(jet,lep) should be connected with the muon too. It's not because he is still working
on that. I can leave that comment out and expect a question on that.
check for typos throughout (here: "includin")
4.2 what is "6. Input links time calibration." ?
To measure time difference in data arriving to different input. I comment it out.
I told would be done. In VC707 the path are adjusted to be time aligned and can be
used as source. We can talk later on this.
last phrase: remove "initially" or remove the whole phrase
Fixed.
Table 1: the optical outputs to CTP are 6.4G only ! For the inputs we are somewhat inconsistent. sometimes we say 10G, here 13G.
Fixed. I read thorugh and clean up where necessary.
The plan is actually to run at 12.8 G and that means 13G speed grade of the FPGAs, 14G speed grade of the miniPOD receivers. Can we make a note of this here ?
Fixed. And I add a footnote.
further down that page: we are now at 4 slices of 32-bit wide at 160 MHz clock. That's what Patric is doing ! ----------
Fixed.
The baseline deserialization scheme is relying on a de-multiplexing the
6.4Gb/s data stream to 4sub-ticks (word slices) of 32 bits wide, transmitted
on a 160MHz clock.
remove:
This de-multiplex factor will most likely yield lowest
latency. It should be noted, however, that alternatively the data could be demultiplexed
into four sub-ticks of 32 bits wide. At marginally higher latency
this scheme might simplify further de-multiplexing to the bunch clock rate.
Fixed.
5.3 I thought the muons are now sending 2 fibres to each topo module. But that's to be discussed.Since that violates the protocol, see below.
Jos said that 320 MHz it's hard to achieve.
I do not understand :The base frequency is 160 MHz but 320 MHz could
be possible, reducing the number of
ber to L1Topo down to two. ??
As above, I quote his sentence "320 MHz could be a problem for the electronics".
We do insist on them using same protocol as CMX. Same comma characters etc.
do the maths again:
they have 16 octants with 2*8 bits each. That's 256 bits total. That would exactly fit on two fibres of 128 bit each. On the CMXes we are rather sure that there are enough zeroes to generate plenty of commas. Is that true for the muons as well ?
can you try to polish that section a bit ? we do not need to make assumptions on what they build, just what the data formats are !
btw who makes sure that QSFPs are able to talk to miniPODs ? is the optical power budget similar ? I would personally not mention any QSFPs. We receive miniPOD compatible data and expect a minimum of 8 fibres from them. *they* would have to make sure the transmitters are compatible.
If they overclock at 160 MHz -> 8 bit per octant -> 1 fiber?
If they overclock at 320 MHz -> 16 bit per octant -> 2 fibers at 6.4
That is also reported on their twiki.
I should make sure if they can get to 320 MHz. For now we could stick the text to this
second preferable option.
I Fixed all the rest.
---------------
I do not understand the hierarchy. Try to find a structure:
Realtime--> CMX and Muon
non-real-time.
Fixed.
The comma character stuff on zero data is true for all real-time links.
5.4 as of now the miniPODs are 14Gb/s. Please do correct. Our costng is for fast FPGAs and fast MiniPOD receivers (you might remember).
Fixed.
table 1 : the miniPODs for DAQ/ROI are not mentioned. call it DAQ/ROI S-link, it's 12 fibres
Fixed.
19.05.2013 10:03
5.4.1 no need to talk about strict need for early L1Calo ROD operation. Is just an option. Also, why limit ourselves to 6.6Gb/s ? Just need to buy the right device:
Fixed. The 6.6 was a left over, I removed it.
so why not :
"
There are two types of non-real-time optical links. Their data rates and encoding schemes
need to be kept within the capabilities of the control FPGA (Kintex-7 / GTX transceiver, up
to 10.3 Gb/s). Three SFP links are envisaged for legacy DAQ and ROI links into L1Calo RODs, and for module control. The latter is available for 1000BASE-SX interconnect. The otherwise unused receive path of the DAQ and ROI SFPs can be used for reception of an external clock (multimode, 850nm, if used concurrently with DAQ/ROI interface).
The MiniPOD devices are used by the embedded ROD firmware and are compatible to the S-Link specifications. They offer 12 bi-directional fibre links and are accompanied by an electrical busy output (LEMO). In accordance with the new ROIB input scheme any ROI signals would be sent out duplicated on a pair of fibres.
"
or similar ... hope that's true ?!?
That's exact. I integrated the text above.
5.5 did you cross-check that there are two 12-way optical inputs on the CTP ? My memory rather says it's a single MTP ! We do have two outputs, but that's a lot of spare only and we might just make a fibre bundle that combines input from those two ports... Or maybe after discussion with the reviewers we even do this step of merging on the PCB already and have the 2nd port as a spare port ? Who knows...
With Stefan H. we mostly talk on the elctrical interface. Indee should be one miniPOD RX only, but I mail him to be sure
about that.
5.6 doe we really want to say that the connector mechanics are not within the scope of review ? I would guess that the review does in fact cover the baseline mezzanine, so we should make sure we put sensible stuff on. Though the mezzanine is obviously a place where easily changeable stuff can reside.
maybe we just rempve that phrase...
Removed.
"Therefore the CTP will
provide a maximum of 192 input signals." what does that mean ?
3 plug x 32 signal at 40 MHz = 96 input signals
overclocking at 80 MHz, twice as much. and that's the maximum the CTP can input.
I rework the text to make it more clear.
5.7 is it just three ? I count so far two outputs to CTP, one input, one output to/from ROD
I did not count an input for the ROD. Added. I'll check with Bruno if it can fit.
Appendix A: you have reformatted that, did you check after that ? Any modifications due to earlier comments from reviewers ? I am skipping over that for now.
figure 9 wrong caption
also: it is a rather old slide. No more ROI but TOB ! can you correct ? Source is at http://epweb2.ph.bham.ac.uk/user/hillier/tmp/homework/TopoFormats.xlsx
No, still to fix. All here it's old.
I stop here now. Please note that I have not read the front sections except those you flagged ! Please let me know if anything els need to be read. Please point Stephan at my comments on the firmware stuff.
13:22