[ Team LiB ] Previous Section Next Section

Operation of the Flow Control Model - An Example

The purpose of this example is to explain the operation of the Flow Control mechanism based on the conceptual model presented by the specification. The example uses the non-posted header Flow Control buffer type, and spans four stages to capture the nuances of the flow control implementation:

Stage One — Immediately following initialization, the several transactions are tracked to explain the basic operation of the counters and registers as they track transactions as they are sent across the link. In this stage, data is accumulating within the Flow Control buffer, but no transactions are being removed.

Stage Two — If the transmitter sends non-posted transactions at a rate such that the Flow Control buffer is filled faster than the receiver can forward transactions from the buffer, the buffer will fill. Stage two describes this circumstance.

Stage Three — The modulo counters are designed to roll over and continue counting from zero. This stage describes the flow control operation at the point of the CREDITS_ALLOCATED count rolling over to zero.

Stage Four — The specification describes the optional error check that can be made by the receiver in the event of a Flow Control buffer overflow. This error check is described in this section.

Stage 1 — Flow Control Following Initialization

The assumption made in this example is that flow control initialization has just completed and the devices are ready for normal operation. The Flow Control buffer is presumed to be 2KB in size, which represents 102d (66h) Flow Control units with 20 bytes/header. Figure 7-5 on page 295 illustrates the elements involved with the values that would be in each counter and register following flow control initialization.

Figure 7-5. Flow Control Elements Following Initialization

graphics/07fig05.jpg

The transmitter must check Flow Control credit prior to sending a transaction. In the case of headers the number of Flow Control units required is always one. The transmitter takes the following steps to determine if the transaction can be sent. For simplicity, this example ignores the possibility of data being included in the transaction.

The credit check is made using unsigned arithmetic (2's complement) in order to satisfy the following formula:

CL – (CC + PTLP )mod2[FieldSize] 2[FieldSize ]/2

Substituting values from Figure 7-5 yields:

66h–(00h + 01h)mod28 28/2

66h–01h mod256 80h

  1. The current CREDITS_CONSUMED count (CC) is added to the PTLP credits required, to determine the CUMULATIVE_CREDITS_REQUIRED (CR), or 00h + 01h =01h. Sufficient credits exist if this value is equal to or less than the credit limit.

  2. The CUMULATIVE_CREDITS_REQUIRED count is subtracted from the CREDIT_LIMIT count (CL) to determine if sufficient credits are available. The following description incorporates a brief review of 2's complement subtraction. When performing subtraction using 2's complement the number to be subtracted is complemented (1's complement) and 1 is added (2's complement). This value is then added to the number being subtracted from. Any carry due to the addition is simply ignored.

    The numbers to subtract are:

    CL 01100110b (66h) - CR 00000001b (01h) = n

    Number to be subtracted is converted to 2's complement:


    00000001b > 11 111110b (1's complement)

    11111110b +1 = 11111111b (1's complement +1 = 2's complement)

    2's complement is added.


    01100110
    11111111 (add)
    01100101 = 01100101b = 65h

    Is result <= 80h?


    Yes, 65h <= 80h (send transaction)

    The result of the subtraction must be equal to or less than 1/2 the maximum value that can be tracked with a modulo 256 counter (128). This approach is taken to ensure unique results from the unsigned arithmetic. For example, unsigned 2's-complement subtraction yields the same results for both 0-128 and 255-127, as shown below.

      00h(0)  -  80h(128)= -80h(128)
00000000b - 1000000b = n
00000000b + 01111111+1b (add 2's complement)
00000000b + 10000000b = 10000000b(10h)

 FFh(255) -  7Fh(127) = +80h(128)
11111111b - 01111111b = n
11111111b + 10000000+1 (add 2's complement)
11111111b + 10000001b = 10000000b(10h)

    To ensure that conflicts such as the one above do not occur, the maximum number of unused credits that can be reported is limited to 28/2 (128) credits for headers and 212/2 (2048) credits for data. This means that the CREDITS_ALLOCATED count must never exceed the CREDITS_CONSUMED count by more than 128 for headers and 2048 for data. This ensures that any result < 1/2 the maximum register count is a positive number and represents credits available, and results > 1/2 the maximum count are negative numbers that indicate credits not available.

  3. The CREDITS_CONSUMED count increments by one when the transaction is forwarded to the link layer.

  4. When the transaction arrives at the receiver, the transaction header is placed into the Flow Control buffer and the CREDITS_RECEIVED counter (optional) increments by one. Note that CREDIT_ALLOCATED does not change.

Figure 7-6 on page 297 illustrates the Flow Control elements following transfer of the first transaction.

Figure 7-6. Flow Control Elements Following Delivery of First Transaction

graphics/07fig06.jpg

Stage 2 — Flow Control Buffer Fills Up

This example presumes that the receiving device has been unable to move transactions from the Flow Control buffer since initialization. This could be caused if the device core was temporarily busy and unable to process transactions. Consequently, the Flow Control buffer has completely filled. Figure 7-7 on page 299 illustrates this scenario.

Figure 7-7. Flow Control Elements with Flow Control Buffer Filled

graphics/07fig07.jpg

Again the transmitter checks Flow Control credits to determine if the next pending TLP can be sent. The unsigned arithmetic is performed to subtract the Credits Required from the CREDIT_LIMIT:


 66h(CL) - 67h (CR) <= 80h
01100110b - 01100111b <= 10000000b (if yes, send transaction)

CL 01100110(66)
CR 10011001 (add 2's complement of 67h)
   11111111 = FFh<=80h (not true, don't send packet)

Not until the receiver moves one or more transactions from the Flow Control buffer can the pending transaction be sent. When the first transaction is moved from the Flow Control buffer, the CREDIT_ALLOCATED count is increased to 67h. When the Update Flow Control packet is delivered to the transmitter, the new CREDIT_LIMIT will be loaded into the CL register. The resulting check will pass the test, thereby permitting the packet to be sent.


CL 01100111 (67)
CR 10011001 add 2's complement of 67
   00000000 = 00h<=80h (send transaction)

Stage 3 — The Credit Limit count Rolls Over

The receiver's CREDIT_LIMIT (CL) always runs ahead of (or is equal to) the CREDITS_CONSUMED (CC) count. Each time the transmitter performs a credit check, it adds the credits required (CR) for a TLP to the current CREDITS_CONSUMED count and subtracts the result from the current CREDIT_LIMIT to determine if enough credits are available to send the TLP.

Because both the CL count and the CC count only index up, they are allowed to roll over from maximum count back to 0. A problem appears to arise when the CL count (which, again, is running ahead) has rolled over and the CC has not. Figure 7-8 shows the CL and CR counts before and after CL rollover.

Figure 7-8. Flow Control Rollover Problem

graphics/07fig08.jpg

If a simple subtraction is performed in the rollover case, the result is negative. This indicates that credits are not available. However, because unsigned arithmetic is used the problem does not arise. See below:


CL 00001000 (08h)
CR 11111000 (F8h) > 00000111+1 = 2's complement

CL 00001000 (08h)
CR 00001000 (add 2's complement)
   00010000 or 10h

Stage 4 — FC Buffer Overflow Error Check

The specification recommends implementation of the optional FC buffer overflow error checking mechanism. These optional elements include:

  • CREDITS_RECEIVED counter

  • Error Check Logic

These elements permit the receiver to track Flow Control credits in the same manner as the transmitter. That is, the transmitter CREDIT_LIMIT count should be the same as the receiver's CREDITS_ALLOCATED count (after an Update DLLP is sent) and the receiver's CREDITS_RECEIVED count should be the same as the transmitter's CREDITS_CONSUMED count. If flow control is working correctly the following will be true:

  • the transmitter's CREDITS_CONSUMED count should always be its CREDIT_LIMIT

  • the receiver's CREDITS_RECEIVED count (CR) should always be its CREDITS_ALLOCATED count (CA)

An overflow condition is detected when the following formula is satisfied. Note that the field size is either 8 (headers) or 12 (data):

(CA – CR)mod2[FieldSize] > 2[FieldSize ]/2

If the formula is true, then the result is negative; thus, more credits have been sent to the FC buffer than were available and an overflow has occurred. Note that the 1.0a version of the specification defines the equation as rather than > as shown above. This appears to be an error, because when CA=CR no overflow condition exists. For example, for the case right after initialization where the receiver advertises that it has 128 credits for the transmitter to use, CA = 128, and CR = 0 because it hasn't received anything yet, then this equation evaluates true. Which means it has overflowed, when actually all we have done is advertise our max allowed number of credits. If the equation evaluates for only > and not , then everything seems to work.
    [ Team LiB ] Previous Section Next Section