18.05. I am adding comments here at the end of this text file. Please check back occasionally and refresh your browser. That avoids me writing many separate mails or delaying transmission of comments. Comments are in order of appearance, not by chapters. --------------------------------------------------------------------------------------------------- We have actually decided for slightly different crystal frequencies: 40.07x and multiples. That was based on information from MSU. Please ask Bruno on Tuesday and update.Do not update each occurence. Just say somewhere that 40.08 means 40.07...whatever 1.2.3 "Configuration and module control are achieved via Ethernet" FPGA configuration is *not* via ethernet, but from local memories. "Monitoring" should we write here that we intend to send those data into DCS ? We believe that Makis is working at such a scheme for the Muons and we might be able to take that over. But please confirm with Makis ! DAQ and ROI: do we refer here to the Cracow activities or does that come later ? 2.4 This rule must be followed such that the requirements are met, whether a smaller or larger device is mounted." remove this phrase, there are no different sizes of devices on the production modules ! Please check also in remaining document" --------------- There were comments by Weiming on certain rules regarding signal integrity on the PDR. Has that part been removed from the document altogether ? Or was it in the Checklist ? have you checked that against PDR comments ? ----------- further to 2.7: refer to cracow project here ? Or do we better refer to that in just one place ? We should at least say that ROD firmware is under development. In the firmware section ? Hardware wise we need to say somewhere that the output links of the embedded ROD via opto fibre is bi-directional, and that we additionally have an electrical busy output. Also need to say that for any ROI links we provide a duplicated signal. Will forward that mail to you ! We have agreed to provide some small amount of ROI bandwidth (single link, ie. 2 fibres due to duplication). However, easiest way to send ROI data would be via the SFPs through L1Calo R-ROD and have those RODs care for duplication. 3.1 remove reference to VX485 T ! production module is for 690T only. Please check generally that text referring to the prototype is taken out ! with standard backplane connectors only 4 of them will fit the available space and that's 192 fibres, maximum is 288 fibres (72-way MPO) 3.2 since the reviewers did ask that previousla, shouldn't we say: in case of issues with clock phase an alternative jitter cleaner device can be mounted on the extension mezzanine ? Or is that covered further down ? 3.3 "IPbus will be adopted if L1Calo decides in favor of that approach." we will definitely go for IPBus and Mainz has started to work on firmware and software implementation as you are rightly saying in the next phrase. 3.1 should not read scalable design since there are no more FPGA options ! modular design ? Fig.4 what is this mmcx thing ? ------------------------------- https://indico.cern.ch/getFile.py/access?contribId=1&sessionId=0&resId=1&materialId=slides&confId=252265 slide 25: the UK suggested future configuration method !! ---------------------------------- 19.05. ------------------------------- Please read through the whole document again. We have too often written that something can be added if required.It was ok for the PDR of the prototype. That's not a good thing to write for a FDR of the final design !!!. Either something is there or isn't only in exceptional cases something can still be undefined at that stage. For example we could explicitly say that "this functionality is implemented on the extension module and might therefore be replaced by ... at a later stage... We should make sure we and the reviewers understand what's on and what isn't ------------------------------------------------------------ 3.3 2nd line: why initially ? It will always be ethernet except that we have more than 1 ethernet port to choose from. fig.6: ATC250 is the correct name. fig. 2 why are the input links from miniPOD to FPGAs not shown, if that's adrawing of the RTDP ? fig.4 that's only the non-real time links. That's not clear from the caption sect.4 :"As far as control software is concerned a likely candidate for module control is the" replace with : "Module control will be based on the" Fig. 8: what is the meaning of "Delays added where needed" ? Why is there an OR at the output ? That's not realistic, we have separate trigger bits per algorithm ! The caption is too long. What's the meaning of "In this example, muons sorted/selected TOB still need to be included in the generic R(lep,jet) algorithm." ? check for typos throughout (here: "includin") 4.2 what is "6. Input links time calibration." ? last phrase: remove "initially" or remove the whole phrase Table 1: the optical outputs to CTP are 6.4G only ! For the inputs we are somewhat inconsistent. sometimes we say 10G, here 13G. The plan is actually to run at 12.8 G and that means 13G speed grade of the FPGAs, 14G speed grade of the miniPOD receivers. Can we make a note of this here ? further down that page: we are now at 4 slices of 32-bit wide at 160 MHz clock. That's what Patric is doing ! The baseline deserialization scheme is relying on a de-multiplexing the 6.4Gb/s data stream to 4sub-ticks (word slices) of 32 bits wide, transmitted on a 160MHz clock. remove: This de-multiplex factor will most likely yield lowest latency. It should be noted, however, that alternatively the data could be demultiplexed into four sub-ticks of 32 bits wide. At marginally higher latency this scheme might simplify further de-multiplexing to the bunch clock rate. 5.3 I thought the muons are now sending 2 fibres to each topo module. But that's to be discussed.Since that violates the protocol, see below. I do not understand :The base frequency is 160 MHz but 320 MHz could be possible, reducing the number of ber to L1Topo down to two. ?? We do insist on them using same protocol as CMX. Same comma characters etc. do the maths again: they have 16 octants with 2*8 bits each. That's 256 bits total. That would exactly fit on two fibres of 128 bit each. On the CMXes we are rather sure that there are enough zeroes to generate plenty of commas. Is that true for the muons as well ? can you try to polish that section a bit ? we do not need to make assumptions on what they build, just what the data formats are ! btw who makes sure that QSFPs are able to talk to miniPODs ? is the optical power budget similar ? I would personally not mention any QSFPs. We receive miniPOD compatible data and expect a minimum of 8 fibres from them. *they* would have to make sure the transmitters are compatible. --------------- I do not understand the hierarchy. Try to find a structure: Realtime--> CMX and Muon non-real-time. The comma character stuff on zero data is true for all real-time links. 5.4 as of now the miniPODs are 14Gb/s. Please do correct. Our costng is for fast FPGAs and fast MiniPOD receivers (you might remember). table 1 : the miniPODs for DAQ/ROI are not mentioned. call it DAQ/ROI S-link, it's 12 fibres 19.05.2013 10:03 5.4.1 no need to talk about strict need for early L1Calo ROD operation. Is just an option. Also, why limit ourselves to 6.6Gb/s ? Just need to buy the right device: so why not : " There are two types of non-real-time optical links. Their data rates and encoding schemes need to be kept within the capabilities of the control FPGA (Kintex-7 / GTX transceiver, up to 10.3 Gb/s). Three SFP links are envisaged for legacy DAQ and ROI links into L1Calo RODs, and for module control. The latter is available for 1000BASE-SX interconnect. The otherwise unused receive path of the DAQ and ROI SFPs can be used for reception of an external clock (multimode, 850nm, if used concurrently with DAQ/ROI interface). The MiniPOD devices are used by the embedded ROD firmware and are compatible to the S-Link specifications. They offer 12 bi-directional fibre links and are accompanied by an electrical busy output (LEMO). In accordance with the new ROIB input scheme any ROI signals would be sent out duplicated on a pair of fibres. " or similar ... hope that's true ?!? 5.5 did you cross-check that there are two 12-way optical inputs on the CTP ? My memory rather says it's a single MTP ! We do have two outputs, but that's a lot of spare only and we might just make a fibre bundle that combines input from those two ports... Or maybe after discussion with the reviewers we even do this step of merging on the PCB already and have the 2nd port as a spare port ? Who knows... 5.6 doe we really want to say that the connector mechanics are not within the scope of review ? I would guess that the review does in fact cover the baseline mezzanine, so we should make sure we put sensible stuff on. Though the mezzanine is obviously a place where easily changeable stuff can reside. maybe we just rempve that phrase... "Therefore the CTP will provide a maximum of 192 input signals." what does that mean ? 5.7 is it just three ? I count so far two outputs to CTP, one input, one output to/from ROD Appendix A: you have reformatted that, did you check after that ? Any modifications due to earlier comments from reviewers ? I am skipping over that for now. figure 9 wrong caption also: it is a rather old slide. No more ROI but TOB ! can you correct ? Source is at http://epweb2.ph.bham.ac.uk/user/hillier/tmp/homework/TopoFormats.xlsx I stop here now. Please note that I have not read the front sections except those you flagged ! Please let me know if anything els need to be read. Please point Stephan at my comments on the firmware stuff. 13:22 h 1.2.2 "Future LHC bunch clock distribution might di er from this scheme" seems to be left over from my text. what does this phrase relate to now? p12 "Allow for footprint compatible medium- and high-end FPGAs for scalability of processing resources." can be removed, we are using the largest one anyway! "Provide an aggregate bandwidth of up to 24 GB/s on MGT outputs towards the CTP: { 24 channels, 6.4 Gb/s (capable of 10Gb/s) line rate." we can send a maximum of 12 channels, that's half that... fig 12 why called RTDP pairs ? how about "electrical links" btw you have drawn the outputs to CTP for bottom FPGA only. Well, perhaps that's better readable. But reviewers might ask... table 1 why not write 6.4/12.8 Gb/s in this table ? Here people will find that info ! I do not know what the bit rates for the S-links is, but it is certainly above 1Gb/s. Why not say "up to 8Gb/s" here ? FYI: on the GTX there is this infamous gap between 8 and ~9.8Gb/s so we have to run either below or above... 5.2.2 "The L1Topo therefore receive two sets of 16 bits at 160 MHz" is that understandable ? What do you mean ? "The base frequency is 320 MHz" what does this mean ? we should add : The muon input links will use the same data encoding of zero data as being defined for the calorimeter inputs (see ...) 5.2.3 "The remaining 2 CMXs on JEM need in total 14 bres." I do not understand. The format in fig.9 shows a total of 128 bits for the energies. that's just one fibre worth of data. Lets split that to two to get enough zeroes and that's it... typo XC7VX980T is the correct spelling 5.3 "This will be a rmware plugin on the control FPGA." not clear what this is referring to. How about "The ROD functionality ... Appendix A : again, I will not go through it. Please ask Bruno on Tuesday (send him an e-mail now) to go through that and check whether that's sensible / whether that's the rules he applied so far. And check whether Weimings comments have been worked in (not from the report, but from his comments) If required post an update uf the document early next week... fig.9 please check with Stephan that this is the real-time input formats he assumes. I am still worried about the acronym ROI Should be TOB