• Switching via a bus. In this approach, an input port transfers a packet directly to the
output port over a shared bus, without intervention by the routing processor. This is
typically done by having the input port pre-pend a switch-internal label (header) to
the packet indicating the local output port to which this packet is being transferred
and transmitting the packet onto the bus. The packet is received by all output ports,
but only the port that matches the label will keep the packet. The label is then
removed at the output port, as this label is only used within the switch to cross the
bus. If multiple packets arrive to the router at the same time, each at a different input
port, all but one must wait since only one packet can cross the bus at a time. Because
every packet must cross the single bus, the switching speed of the router is limited
to the bus speed; in our roundabout analogy, this is as if the roundabout could only
contain one car at a time. Nonetheless, switching via a bus is often sufficient for
routers that operate in small local area and enterprise networks. The Cisco 5600
[Cisco Switches 2012] switches packets over a 32 Gbps backplane bus.
• Switching via an interconnection network. One way to overcome the bandwidth
limitation of a single, shared bus is to use a more sophisticated interconnection network,
such as those that have been used in the past to interconnect processors in a
multiprocessor computer architecture. A crossbar switch is an interconnection network
consisting of 2N buses that connect N input ports to N output ports, as shown
in Figure 4.8. Each vertical bus intersects each horizontal bus at a crosspoint, which
can be opened or closed at any time by the switch fabric controller (whose logic is
part of the switching fabric itself). When a packet arrives from port A and needs to
be forwarded to port Y, the switch controller closes the crosspoint at the intersection
of busses Aand Y, and port Athen sends the packet onto its bus, which is picked up
(only) by bus Y. Note that a packet from port B can be forwarded to port X at the
same time, since the A-to-Y and B-to-X packets use different input and output
busses. Thus, unlike the previous two switching approaches, crossbar networks are
capable of forwarding multiple packets in parallel. However, if two packets from
two different input ports are destined to the same output port, then one will have to
wait at the input, since only one packet can be sent over any given bus at a time.
More sophisticated interconnection networks use multiple stages of switching
elements to allow packets from different input ports to proceed towards the same
output port at the same time through the switching fabric. See [Tobagi 1990] for
a survey of switch architectures. Cisco 12000 family switches [Cisco 12000
2012] use an interconnection network.
4.3.3 Output Processing
Output port processing, shown in Figure 4.9, takes packets that have been stored in
the output port’s memory and transmits them over the output link. This includes
selecting and de-queueing packets for transmission, and performing the needed linklayer
and physical-layer transmission functions.
326 CHAPTER 4 • THE NETWORK LAYER
4.3.4 Where Does Queueing Occur?
If we consider input and output port functionality and the configurations shown in
Figure 4.8, it’s clear that packet queues may form at both the input ports and the output
ports, just as we identified cases where cars may wait at the inputs and outputs of
the traffic intersection in our roundabout analogy. The location and extent of queueing
(either at the input port queues or the output port queues) will depend on the traffic
load, the relative speed of the switching fabric, and the line speed. Let’s now consider
these queues in a bit more detail, since as these queues grow large, the router’s memory
can eventually be exhausted and packet loss will occur when no memory is available
to store arriving packets. Recall that in our earlier discussions, we said that
packets were “lost within the network” or “dropped at a router.” It is here, at these
queues within a router, where such packets are actually dropped and lost.
Suppose that the input and output line speeds (transmission rates) all have an
identical transmission rate of Rline packets per second, and that there are N input
ports and N output ports. To further simplify the discussion, let’s assume that all
packets have the same fixed length, and the packets arrive to input ports in a synchronous
manner. That is, the time to send a packet on any link is equal to the time
to receive a packet on any link, and during such an interval of time, either zero or
one packet can arrive on an input link. Define the switching fabric transfer rate
Rswitch as the rate at which packets can be moved from input port to output port. If
Rswitch is N times faster than Rline, then only negligible queuing will occur at the
input ports. This is because even in the worst case, where all N input lines are
receiving packets, and all packets are to be forwarded to the same output port, each
batch of N packets (one packet per input port) can be cleared through the switch fabric
before the next batch arrives.
But what can happen at the output ports? Let’s suppose that Rswitch is still N
times faster than Rline. Once again, packets arriving at each of the N input ports
are destined to the same output port. In this case, in the time it takes to send a single
packet onto the outgoing link, N new packets will arrive at this output port. Since
the output port can transmit only a single packet in a unit of time (the packet transmission
time), the N arriving packets will have to queue (wait) for transmission over
the outgoing link. Then N more packets can possibly arrive in the time it takes to
4.3 • WHAT’S INSIDE A ROUTER? 327
Line
termination
Data link
processing
(protocol,
encapsulation)
Queuing (buffer
Switch management)
fabric
Figure 4.9 Output port processing
transmit just one of the N packets that had just previously been queued. And so on.
Eventually, the number of queued packets can grow large enough to exhaust available
memory at the output port, in which case packets are dropped.
Output port queuing is illustrated in Figure 4.10. At time t, a packet has arrived at
each of the incoming input ports, each destined for the uppermost outgoing port.
Assuming identical line speeds and a switch operating at three times the line speed,
one time unit later (that is, in the time needed to receive or send a packet), all three
original packets have been transferred to the outgoing port and are queued awaiting
transmission. In the next time unit, one of these three packets will have been transmitted
over the outgoing link. In our example, two new packets have arrived at the incoming
side of the switch; one of these packets is destined for this uppermost output port.
Given that router buffers are needed to absorb the fluctuations in traffic load, the
natural question to ask is how much buffering is required. For many years, the rule of
thumb [RFC 3439] for buffer sizing was that the amount of buffering (B) should be
equal to an average round-trip time (RTT, say 250 msec) times the link capacity (C).
This result is based on an analysis of the queueing dynamics of a relatively small number
of TCP flows [Villamizar 1994]. Thus, a 10 Gbps link with an RTT of 250 msec
would need an amount of buffering equal to B = RTT • C = 2.5 Gbits of buffers. Recent
328 CHAPTER 4 • THE NETWORK LAYER
Switch
fabric
Output port contention at time t
One packet time later
Switch
fabric
Figure 4.10 Output port queuing
theoretical and experimental efforts [Appenzeller 2004], however, suggest that when
there are a large number of TCP flows (N) passing through a link, the amount of buffering
needed is B = RTT C/√N
—
. With a large number of flows typically passing through
large backbone router links (see, e.g., [Fraleigh 2003]), the value of N can be large, with
the decrease in needed buffer size becoming quite significant. [Appenzellar 2004; Wischik
2005; Beheshti 2008] provide very readable discussions of the buffer sizing problem
from a theoretical, implementation, and operational standpoint.
A consequence of output port queuing is that a packet scheduler at the output
port must choose one packet among those queued for transmission. This selection
might be done on a simple basis, such as first-come-first-served (FCFS) scheduling,
or a more sophisticated scheduling discipline such as weighted fair queuing (WFQ),
which shares the outgoing link fairly among the different end-to-end connections
that have packets queued for transmission. Packet scheduling plays a crucial role in
providing quality-of-service guarantees. We’ll thus cover packet scheduling extensively
in Chapter 7. A discussion of output port packet scheduling disciplines is
[Cisco Queue 2012].
Similarly, if there is not enough memory to buffer an incoming packet, a decision
must be made to either drop the arriving packet (a policy known as drop-tail) or
remove one or more already-queued packets to make room for the newly arrived
packet. In some cases, it may be advantageous to drop (or mark the header of) a packet