Modified Ordering Rules Improve Performance

This section describes how temporary transaction blocking can occur when the strong ordering rules listed in Table 8-2 are rigorously enforced. Modification of strong ordering between transactions that do not violate the Produce/Consumer programming model can eliminate many blocking conditions and improve link efficiency.

Strong Ordering Can Result in Transaction Blocking

Maintaining the strong ordering relationship between transactions would likely result in instances where all transactions would be blocked due to a single receive buffer being full. The strong ordering requirements to support the Producer/Consumer model cannot be modified (except in the case of relaxed ordering described previously). However, transaction sequences that do not occur within the Producer/Consumer programming model can be modified to a weakly ordered scheme that can lead to improved performance.

The Problem

Consider the following example illustrated in Figure 8-1 on page 323 when strong ordering is maintained for all transaction sequences. This example depicts transmitter and receiver buffers associated with the delivery of transactions in a single direction (from left to right) for a single Virtual Channel (VC), and the transmit and receive buffers are organized in the same way. Also, recall that each of the transaction types (Posted, Non-Posted, and Completions) have independent flow control within the same VC. The numbers within the transmit buffers show the order in which these transactions were issued to the transmitter. In addition, the non-posted receive buffer is currently full. Consider the following sequence.

The Weakly Ordered Solution

As discussed previously, strong ordering is required to support the Producer/Consumer model. This requirement is satisfied entirely by the shaded area in Table 8-2. The non-shaded area deals with transaction sequences that do not occur in the Producer/Consumer programming model, and therefore can be modified. Table 8-3 on page 326 lists these entries as weakly ordered. The modified entries are defined as:

Y/N (Yes/No) — The transaction in column 1 is allowed to proceed ahead of the previously enqueued transaction because the entry is not related to the Producer/Consumer strong ordering requirements and can be weakly ordered to improve performance.

Consider the scenario in Figure 8-1 with weak ordering employed:

In summary, these examples illustrate how strong ordering can temporarily block all transactions pending delivery, and that weak ordering rules can be used to improve link efficiency without violating the Producer/Consumer model.

Order Management Accomplished with VC Buffers

As the previous example illustrated, transaction ordering is managed within the Virtual Channel buffers. These buffers are grouped into Posted, Non-Posted, and Completion transactions and flow control is managed independently for each group. This makes it much easier to implement the modified (weak) ordering described in the previous example. See Chapter 7, entitled "Flow Control," on page 285 for details.

Recall that transactions are mapped to Virtual Channels using the transaction's TC. If each TC is mapped to a separate VC, then each VC buffer will contain transactions with a single TC assignment. In this situation, VC flow control permits optimum flow of transactions.

Summary of Modified Ordering Rules

Table 8-3 on page 326 lists and highlights the modified ordering rules that allow switches to move some transactions ahead of others that may be stalled due to a receive buffer full condition. Definition of the entries are the same as the previous table:

No— The transaction in column 1 must not be permitted to proceed ahead of the previously enqueued transaction in the corresponding columns (2-6).

Y/N (Yes/No)— The transaction in column 1 is allowed to proceed ahead of the previously enqueued transaction because:

• its Relaxed Ordering bit is set (1), but it is not required to do so.

• the entry is not subject to the Producer/Consumer strong ordering requirements and is weakly ordered to improve performance.

Note: The "No" entry in Row D, Column 5 (D5) applies to cases where a Completer returns multiple Completions in response to a single read request. These Completions must return in order (i.e., Completions with the same transaction ID).

Upon examination of the table, some readers may question whether programs will operate correctly when weak ordering is employed. For example, note that a write transaction is permitted to be reordered ahead of a previously latched read request (entry A3). A programmer who reads from a location followed by a write to the same location must not expect these operations to complete in program order. Note that the ordering rules only guarantee proper operation of the Producer/Consumer programming model. If a programmer requires a read operation to complete ahead of a write transaction, then the write must not be issued until the read transaction completes.