Download The ATLAS ROBIN – A High-Performance Data

Chapter 3 - ATLAS DAQ
bypassing the PCI-bus. The ROBIN – the only custom component of the ATLS dataflow apart from
the RoIB – was designed to handle and buffer the input data with the flexibility to interact with the
ROS-PC as well as directly with the dataflow system via a private network interface. In addition to the
tasks above, which are related to the transport of detector data, the ROS interfaces to the configuration
database and run control, the monitoring system and the detector control system (DCS) [ATLDCS].
The requirements on the ROS are summarised in a ROS user requirements document [ROSURD].
3.3.1
Event handling
In ATLAS, events are identified by a 32 bit event number composed from a 24 bit identifier generated
by the L1 trigger – the primary L1ID – plus an 8 bit event-counter-reset (ECR) value, incremented by
the RODs upon the wrapping of the event identifier to 0. For simplicity the event number is normally
and in this thesis referred to as L1ID. A design constraint limits the frequency of the ECR to the range
from 0.1Hz to 1.0Hz. As a result, the maximum time between two zero-crossings of the event number
is 256s, equivalent to 25.6 million events at a L1-rate of 100kHz. The minimum time is 25.6s. As a
typical ATLAS run (a period of continuous operation) can extend to several hours the L1ID is not
necessarily unique for all events and an additional mechanism needs to be put in place for event
identification. As the minimum time covers the range of L2 and EF latencies the additional
information is inserted at the event building stage by adding appropriate time-stamp information. At
the ROS level the limited amount of buffering space requires to delete events as soon as possible. This
is done by explicit delete messages distributed by the DFM, after the event has been rejected by the L2
or processed by the EF. To reduce the rate for delete messages they are sent out via a multicast
mechanism typically in groups of 100.
As stated in chapter 3.2 the dataflow system is designed to minimise packet loss on the network
switches, however losses are not fully prevented. While data requests are point-to-point interactions
and can be protected by a reliable protocol this is not the case for delete messages. Lost delete message
lead to orphaned fragments in the ROS and reduce the amount of buffer space available. While the
event numbers at the ROS level restart at 0 after a maximum of 256s it cannot be guaranteed that all
orphaned events will be replaced, as the L1ID does not have to be strictly sequential. Therefore, a
“garbage collection” mechanism is required to clean up the buffer. The implementation of this
mechanism requires to distribute the “oldest” valid L1ID in the dataflow system, which is piggybacked to the delete messages. The loss of a message is detected by a jump in the message sequence
numbers. Once a lost delete message is detected, the ROS compares the oldest valid L1ID to the most
recent L1ID received from the ROLs, creates a range of valid L1IDs and deletes all fragments with
event numbers outside of this range. The actual garbage collection procedure is executed on the
ROBINs, which need to do a time-consuming scan of their entire buffer in order to build the list of
stored fragments. To avoid excessive load the garbage collection is executed only when the buffers on
the ROBIN have reached a certain filling level.
3.3.2
Configuration and monitoring
Every ATLAS run is associated with a set of configuration parameters, stored in the global ATLAS
configuration database. The range of parameters is very broad and includes calibration values for
The ATLAS ROBIN – A High-Performance Data-Acquisition Module
31