Advanced Error Reporting Mechanisms

Advanced Error Reporting requires implementation of the Advanced Error Reporting registers illustrated in Figure 10-14 on page 382. (Note that the lighter fields at the bottom of the Capability register diagram are used only for root ports, discussed later.) These registers provide several additional error reporting features:

• finer granularity in defining the actual type of error that has occurred within each classification.

• ability to specify the severity of each uncorrectable error type to determine whether it should be reported as a fatal or non-fatal error.

• support for logging errors.

• enable/disable Root Complex to report errors to the system.

• identify source of the error.

• ability to mask reporting individual types of errors.

ECRC Generation and Checking

End-to-End CRC (ECRC) generation and checking can be enabled only if the Advanced Error Reporting Capability registers are implemented. Specifically, the Advanced Error Capability and Control register provides control over ECRC generation and checking as illustrated in Figure 10-15 on page 383.

This register reports whether this device supports ECRC generation and checking. If so, configuration software can enable one or both features.

In some cases, multiple uncorrectable errors may be detected prior to software reading and clearing the register. The First Error Pointer field identifies the bit position within the Advanced Uncorrectable Status register corresponding to the error that occurred first. (See Figure 10-18 on page 387.)

The First Error Pointer and the ECRC Check and Generation Enable bits must be implemented as sticky bits.

Handling Sticky Bits

Several of the Advanced Configuration Error Registers employ sticky fields. Many of these fields are single bits. The designations of sticky fields are as follows:

• ROS — Read Only/Sticky

• RWS — Read/Write/Sticky

• RW1CS — Read/Write 1 to Clear/Sticky

Sticky error register fields behave differently in that a Hot Reset has no affect on the contents of these fields. For all other register fields, Hot Reset forces default values into the fields. Sticky bits are important in error handling to ensure that Error-related control and status information is not lost due to a Hot Reset. Software may initiate a Hot Reset in an attempt to clear errors.

Advanced Correctable Error Handling

Advanced error reporting provides the ability to pinpoint specific correctable errors. These errors can selectively cause the generation of a Correctable Error Message being sent to the host system:

• Receiver Errors (optional) — caused when the Physical Layer detects an error in the incoming packet (TLP or DLLP). The Physical Layer discards the packet, frees buffer space allocated to the packet, and signals the Link Layer that a receive error has occurred. This error is reported by setting a status bit and must not result in the Link Layer also reporting an error for this packet (e.g., the Link Layer must not report a Bad TLP or Bad DLLP).

• Bad TLPs — caused when the Link Layer detects a packet with a bad CRC check, an incorrectly nullified packet, or an incorrect Packet Sequence Number (not a duplicate). In each case, the Link Layer discards the packet and reports a NAK DLLP to the transmitter, which triggers a transaction retry.

• Bad DLLPs — caused by a CRC check failure. This type of error may be corrected by a subsequent DLLP or a time-out that results ultimately in DLLP retry. The exact corrective action depends on the type of DLLP that has failed and the circumstances associated with packet transmission.

• REPLAY_NUM Rollover — the REPLAY_NUM is a count maintained within the transmitting side of the link layer that keeps track of the number of times that a transaction had been retransmitted without successful delivery. When the count rolls over, hardware automatically retrains the link in an attempt to clear the fault condition.

• Replay Timer Time-out — this timer is maintained within the transmitting side of the Link Layer and is intended to trigger a retry when forward progress in sending TLPs has stopped. A time-out occurs when unacknowledged TLPs have not received an acknowledgement within the time-out period. A time-out results in a retry of all unacknowledged TLPs.

Knowledge of which error has occurred can help system software to make better predictions of components that are likely to fail completely in the future. Software may also choose to mask recognition of some correctable errors while reporting others.

Advanced Correctable Error Status

When a correctable error occurs the corresponding bit within the Advanced Correctable Error Status register is set. (See Figure 10-16.) These bits are automatically set by hardware and are cleared by software when writing a "1" to the bit position. These bits are set whether or not the error is reported via an Error Message. Each status bit in this register is designated RW1CS.

Advanced Correctable Error Reporting

Whether a particular correctable error is reported to the host is specified by the Correctable Mask register illustrated in Figure 10-17. The default state of the mask bits are cleared (0), thereby causing a Correctable Error message to be delivered when any of the correctable errors are detected. Software may choose to set one or more bits to prevent a Correctable Error Message from being sent when the selected error is detected. Each bit in this register is designated RWS.

Advanced Uncorrectable Error Handling

Advanced error reporting provides the ability to pinpoint which uncorrectable error has occurred. Furthermore software can specify the severity of each error and select which errors will result in an Error Message being sent to the host system (Root Complex).

• Training Errors (optional) — caused by failure in the link training sequence at the Physical Layer.

• Data Link Protocol Errors — caused by Link Layer protocol errors including the ACK/NAK retry mechanism. [Note: The specification states that "Violations of Flow Control initialization protocol are Data Link Layer Protocol Errors" and that checking these errors is optional. It seems likely that these error should be treated as Flow Control Protocol errors (which are optional) and not Data Link Layer Protocol errors (which are required)].

• Poisoned TLP Errors — caused by data corruption storage error within memory or data buffers.

• Flow Control Protocol Errors (optional) — errors associated with failures of the flow control mechanism within the transaction layer.

• Completion Time-out Errors — caused by excessive delays in the return of the expected completion. This error is detected by the Requester.

• Completer Abort Errors (optional) — occurs when the Completer cannot fulfill the transaction request due to a variety of possible problems with the request or failure of the completer device.

• Unexpected Completion Errors — occurs when a requester receives a completion transaction that does not match an request pending completion.

• Receiver Overflow Errors (optional) — caused by a flow-control buffer overflow condition.

• Malformed TLPs — caused by processing errors associated with the transaction header. This error is detected within the Transaction Layer of the device receiving the TLP.

• ECRC Errors (optional) — caused by an End-to-End CRC (ECRC) check failure within the transaction layer of the receiving device.

• Unsupported Request Errors — occurs when the Completer does not support the request. The request is correctly formed and has not incurred any detected error during transport, however the transaction cannot be fulfilled by the completer due to a variety of reasons including illegal access and invalid command for this device.

The errors noted as optional in the list above may not be implemented in the Advanced Uncorrectable register set.

Advanced Uncorrectable Error Status

When an uncorrectable error occurs the corresponding bit within the Advanced Uncorrectable Error Status register is set. (See Figure 10-18 on page 387.) These bits are automatically set by hardware and are cleared by software when writing a "1" to the bit position. These bits are set whether or not the error is reported via an Error Message. Each status bit in this register is designated RW1CS.

Selecting the Severity of Each Uncorrectable Error

Advanced error handling permits software to select the severity of each error within the Advanced Uncorrectable Error Severity register. This gives software the opportunity to treat errors according to the severity associated with a given application. For example, Poisoned TLP data associated with audio data being sent to a speaker, while not correctable has no serious side effects relative to the integrity of the system. However, if real-time status information is being retrieved that will help make critical decisions, any errors in this data can be very serious. Figure 10-19 illustrates the Error Severity register. The default values are illustrated in the individual bit fields. These represent the default severity levels for each type of error (1 = Fatal, 0 = Non-Fatal).

Those uncorrectable errors that are selected to be non fatal will result in a NonFatal Error Message being delivered and those selected as Fatal errors will result in a Fatal Error Message delivered. However, whether or not an Error Message is generated for a given error is specified in the Advanced Uncorrectable Mask register.

Uncorrectable Error Reporting

Software can mask out specific errors so that they never cause an Error Message to be generated. The default condition is to generate Error Messages for each type of error (all bits are cleared). Figure 10-20 on page 389 depicts the Advanced Uncorrectable Error Mask register.

Error Logging

A four DWord portion of the Advanced Error Registers block is reserved for storing the header of the transaction that has incurred a failure. Only a select group of Transaction Layer errors result in the transaction header being logged. Table 10-6 lists the transactions that are logged.

Table 10-6. Transaction Layer Errors That are Logged

Name of Error	Default Classification
Poisoned TLP Received	Uncorrectable - Non Fatal
ECRC Check Failed	Uncorrectable - Non Fatal
Unsupported Request	Uncorrectable - Non Fatal
Completion Abort	Uncorrectable - Non Fatal
Unexpected Completion	Uncorrectable - Non Fatal
Malformed TLP	Uncorrectable - Fatal

The format of the header is preserved when captured and placed into the register. That is, the illustration of header format in this book is exactly how the headers will appear within the Error Logging register. Note also that the contents of this register are designated ROS.

Root Complex Error Tracking and Reporting

The Root Complex is the target of all Error Messages issued by devices within the PCI Express fabric. Errors received by the Root Complex result in status registers being updated and the error being conditionally reported to the appropriate software handler or handlers.

Root Complex Error Status Registers

When the Root Complex receives an Error Message, it sets status bits within the Root Error Status register (Figure 10-21 on page 391). This register indicates the types of errors received and also indicates when multiple errors of the same type have been received. Note that an error detected at the root port is treated as if the port sent itself an Error Message.

The Advanced Root Error Status register tracks the occurrence of errors as follows:

Correctable Errors

• Sets the "Received Correctable Error" bit upon receipt of the first ERR_COR Message, or detection of a root port correctable error.

• Sets the "Multiple Correctable Error Message Received" bit upon receipt of an ERR_COR Message, or detection of a root port correctable error when the "Received Correctable Error" bit is already set.

Uncorrectable Errors

• Sets the "Received Uncorrectable Error" bit upon receipt of the first ERR_FATAL or ERR_NONFATAL Error Message, or detection of a root port uncorrectable error.

• Set the "Multiple Uncorrectable Error Message Received" bit upon receipt of an ERR_FATAL or ERR_NONFATAL Message, or detection of a root port correctable error when the "Received Uncorrectable Error" bit is already set.

Detecting and Reporting First Uncorrectable Fatal versus Non-Fatal Errors

If the system wishes to implement separate software error handlers for Correctable, Non-Fatal, and Fatal errors. The Root Error Status register includes bits to differentiate Correctable from Uncorrectable Errors but needs additional bits to also determine whether Uncorrectable errors are fatal or non-fatal:

• If the first Uncorrectable Error Message received is FATAL the "First Uncorrectable Fatal" bit is also set along with the "Fatal Error Message Received" bit.

• If the first Uncorrectable Error Message received is NONFATAL the "Non-Fatal Error Message Received" bit is set. (If a subsequent Uncorrectable Error is Fatal, the "Fatal Error Message Received" bit will be set, but because the "First Uncorrectable Fatal" remains cleared, software knows that the first Uncorrectable Error received was Non-Fatal.)

Advanced Source ID Register

Software error handlers may need to read and clear error status registers within the device that detected and reported the error. The Error Messages contain the ID of the device reporting the error. The Source ID register captures the Error Message ID associated with the first Fatal and first Non-Fatal Error message received by the Root Complex. The format of this register is shown in Figure 10-22 on page 391.

Root Error Command Register

The Root Error Status register sets status bits that determines whether a Correctable, Fatal, or Non-Fatal error has occurred. In conjunction with these status bits the Root Complex can also generate separate interrupts that call handlers for each of the error categories. The Root Error Command register enables interrupt generation for all three categories as pictured in Figure 10-23 on page 392.

Reporting Errors to the Host System

Software error handlers will initially read Root Complex status registers to determine the nature of the error, and may also need to read device-specific error registers of the device that reported the error.