HCAL TPG Status
Tullio Grassi
University of Maryland
May 2004
CMS ESR. May, 2004
HCAL TriDAS
1
Clock & BC0 Distribution
• “RX_CLK” and “RX_BC0” signals for all CAL TPGs transmission
• Implement this requirement using 2 or 3-stage fanout scheme
• Details to be defined with ECAL.
TTC Minicrate
Rack-to-Rack CAT 6/7
HHHHD
T T T T C
RRRRC
F
A
N
O
U
T
F
A
N
O
U
T
HHHHD
T T T T C
RRRRC
F
A
N
O
U
T
F
A
N
O
U
T
F
A
N
O
U
T
ECAL
(18 crates)
16 HCAL VME Crates
CMS ESR. May, 2004
HCAL TriDAS
2
Fanout board - features
• Built by J. Mans and C. Tully at Princeton
• Fanout 4 differential LVPECL signals over
cat6 with RJ45 connectors:
–
–
–
–
RX_CLK,
RX_BC0
TTC serial stream
80MHz Ref_Clk
• Board operates in 2 modes
– Global (unique board for ECAL and HCAL)
– Crate
• Successfully used in 2003 testbeam and
all current HCAL R&D test stands
• J.C. da Silva received one board for
evaluation and “blessing”
CMS ESR. May, 2004
HCAL TriDAS
3
Fanout Board - diagram
TTC Broadcast
TTC fiber
TTCrx
QPLL implemented this
way allows RX_CLK to
be “cleaned” at each
board if needed
Clk80
40MHz TTC_CLK
QPLL can run
stand-alone
G
C
P1
G
QPLL
P2
18 Identical
Outputs
G
RX_CLK
C
(also as timing)
INT_BC0
RX_BC0
C
RX_CLK
Input from
GLOBAL Fanout
CMS ESR. May, 2004
G
Decoding
EXT_BC0
FPGA
Progr.
Delay
80MHz
RX_BC0
HCAL TriDAS
4
Complete path of a 40MHz RX_CLK
Fanout board in Global-mode
TTC fiber
TTCrx
CLK40_Des1
3.3V CMOS
• Path is 3.3V differential PECL unless
otherwise stated.
• Path of RX_BC0 is similar but comes from
the FPGA rather the QPLL
• In the Global-mode card, do not mount the
buffers for CLK80 and TTC
QPLL
FPGA
CAT7
(RX_CLK,
RX_BC0)
CAT7 (RX_CLK,
RX_BC0, TTC,
CLK80)
HTR
TTCrx
SLB
QPLL
TPG spec is:
Skew <  6 ns
across HCAL and
ECAL
FPGA
SLB
SLB
Max skew on
HTR traces is
0.7 ns.
SLB
SLB
SLB
Fanout board in Crate-mode
CMS ESR. May, 2004
HCAL TriDAS
5
TTC-PMC mezzanine (UMD)
– Carry TTCrx onto HCAL boards (HTR, DCC, Fanout)
– Money-saver as it eliminated a lot of optical components
and Fine-Line BGAs on 9U boards.
Production is done (~350 boards)
• The assembly house mounted all
LEDs backward
• Test in progress (100% yield so far)
CMS ESR. May, 2004
HCAL TriDAS
6
HTR Schematic
Fiber Data
Serial FE-Data
LC
CLK80
TTC
RX_BC0
RX_CLK40
Ref
Clk
TTCrx
Recovered
Clk
20
Crystal
mezzanine
TTC 40 Clk
PLL
x2
SYS80 Clk
Fanout Card
(1/VME crate)
Deserializers (8)
SLB
TTC Broadcast
SYS40 Clk
SLB
Async Fifo
SLB
TPG
Path
SLB
XILINX
SLB
SLB
CMS ESR. May, 2004
HCAL TriDAS
7
HTR Rev4 Status
• Current (Rev4) board – 3 in lab now
– Testing:
•  Links/Clocks (same as Rev3, no problems, no mystery at UMD)
•  DAQ path exactly same as Rev3
•  Trigger Primitive Generation (basic features)
•  Integration with SLB (next slide)
CMS ESR. May, 2004
HCAL TriDAS
8
Trigger link and SLB testing
– Control of SLB (“Local Bus”) seems ok
– JTAG working (connector on P3 area of motherboard)
– Link tests between SLB and Wisconsin receiver board (STC)
•
•
•
•
HP signal generator  120MHz clock for Receiver-STC
120MHz divided by 3 and injected into TTC system
SLB runs with RX_CLK extracted from TTC+QPLL
1.2Gbaud copper link: verified
– Some tests of Cat6 cable showed BER increases by ~10
compared to skew-clear cables.
•
CMS ESR. May, 2004
This could be fine, depending on the final cable length
HCAL TriDAS
9
Trigger link and SLB testing
Problems
In our setup we have seen 2 problems, but we had not much
time to investigate:
1. Two pairs seem swapped (we see A,B,D,C instead of
A,B,C,D)
2. Data are mis-aligned from pair-to-pair. The misalignement changes every time we restart.
We think that these are not a hardware or PCB problems, but
cabling or firmware.
CMS ESR. May, 2004
HCAL TriDAS
10
Trigger Link testing
• So far, used Wisconsin STC boards. For commissioning
and mass-production we need something different:
– More channels
– Programmable to optimize the test setup
• Layout underway for an SLB/Vitesse mezzanine transition
– Allows to plug a Vitesse-Rx-mezzanine on each SLB post.
– Run backwards into motherboard
• Board is now under design
– Delays as an engineer left.
UW Vitesse receiver
mezzanine card
SLB site
HTR
CMS ESR. May, 2004
HCAL TriDAS
11
Production Issues
Fiber Optics
• Some problems not fully understood.
• Seem more related to optical cables, connectors, laser
• Changing the receiver (HTR) is not considered
Yield (excluding optics)
• Last year batch (Rev3) had 5 bad boards out of 30
• Only 1 was fixed (assembler mounted wrong part)
• Not much time to find out the problems
• Assembler seems the best in the area.
CMS ESR. May, 2004
HCAL TriDAS
12
HTR Plans
Testbeam04 in CERN
– At some point we want to use the last version (Rev4)
– Issue: we only have 3 now.
– HF needs 2 HTRs to run
HTR-RCT Integration test in Wisconsin
– Start on May 24th
– Will try to put energy on a given bunch and generate an L1A
– Main goal is to validate all hardware
If Wisconsin and Maryland groups are satisfied
with the Integration Tests, we will launch the HTR
production.
CMS ESR. May, 2004
HCAL TriDAS
13
Level 1 Latency
• Nothing has changed (46 clock tick
budget)
– FPGA logic does not include summing!
– Estimates: probably 1 more clock cycle
in HF
– Eliminating summing in overlap might be
ok for MET/Jet triggers
• But would still have the 1→ 6 summing in
HF
Item
HB
BX to QIE input
2
2
4
QIE to GOL (FE)
9
9
8
Optical Link (7590m@1clock/5m)
15
15-18
15-18
HTR → SLB
12
12
12
SLB ?
4
4
4
TPG Cables (15m)
3
3
3
45
45-48
46-49
TOTAL
HE
HF
46 clocks = 1,147.7ns
HPD or
PMT (HF)
BX
TOF
CMS ESR. May, 2004
HCAL
To RBX
RBX
12-13
O-E QIE CCA GOL
HCAL TriDAS
Data
HTR
SLB
To RCT
RCT
14
TPG Alignment
• Align so that all ECAL and HCAL data from same bucket
reaches RCT inputs at same time
– Achieved by delaying each channel individually
Ch N
Ch N+1
Ch N+2
RX_BC0 (Global)
– Method for establishing this delay implemented inside SLB
• Histogram data over threshold, look for LHC structure pattern
– Issue: For some detectors, occupancy is very low (HO, especially
at low lumens)
CMS ESR. May, 2004
HCAL TriDAS
15
Absolute (and Relative) Timing
• Relative timing within HCAL via:
– Laser and LEDs
• Have to consider random latency
variation after resynching optical links
– Cross-check with BC0 from FE
• Absolute synchronization for Level 1
– SLB histograms ET
– Looks for LHC beam structure
– Does this work? (esp at low luminosity?)
• Salavat has been working on this
– Min bias events, some with Orca and
some with fast sim
• Occupancy at 1034 shown here
– 250MeV per ADC count
• Question: how many orbits to establish the LHC beam structure per
detector?
CMS ESR. May, 2004
HCAL TriDAS
16
Summary (from Salavat)
• At 2x1033
HCAL Element
HB, h~0
HB, h~1.4
HE, h~1.6
HE, h~2.8
HF, 13, h~2.9
HF, 13, h~3.4
HO, ring 2
CMS ESR. May, 2004
Cut (ADC counts)
# LHC Orbits Required (88ms/orbit)
≥6
≥6
≥5
≥5
≥5
≥5
≥8
5x105
105
104
102
104
103
>107
HCAL TriDAS
17
Summary (from Salavat)
• For HB, HE, and HF probably easy to remeasure “on the
fly”
– Fill SLB histograms, read out over VME, calculate offset…
• For HO, probably will take few hours to maybe even a day
– OK as long as absolute (non random) latency is stable over long
periods
– Need more simulation, checks…in progress
CMS ESR. May, 2004
HCAL TriDAS
18
Occupancy Implications and Alignment
• Bottom line: Will try to develop the following algorithm:
– Whatever it takes, we measure the absolute alignment
• If it takes hours, then so be it...
– We hope that this absolute alignment will not change over time
– Keep track of the relative alignment by:
• Sending up a BC0 signal from the FE
– Keep track of this. If the links go down and we reset, then we measure the
relative alignment and adjust accordingly
• Problems:
– If we are off by an order of magnitude or more...
• Needs some more simulation, computing resources, etc.
– Assumes long term stabilities which will have to be tracked
CMS ESR. May, 2004
HCAL TriDAS
19
Another Latency Issue
• TTCrx chip has a chip-to-chip variation in latency of ~20ns
– Affect only the DAQ-path
• How well known is this?
• Need a scheme for insitu calibration of the TTCrx latency
– Probably can come up with something for HTR/DCC/Fanout
– Not sure what to do about TTCrx in FE...
CMS ESR. May, 2004
HCAL TriDAS
20