The Quadrics Network

Figure 4: Quaternary fat tree of dimension 1 (left) and Elite switch of Quadrics networks (right).

• The Quadrics network (QsNet) consists of two hardware building blocks:
  1. a programmable network interface called Elan and
  2. a high-bandwidth, low-latency communication switch called Elite.

• The Elan network interface connects the Quadrics network to a processing node containing one or more CPUs.
• QsNet connects Elite switches in a quaternary fat-tree topology.
• A quaternary fat tree of dimension $n$ is composed of $4^n$ processing nodes and $n \times 4^{n-1}$ switches interconnected as a delta network.
• It can be recursively built by connecting four quaternary fat trees of dimension $n-1$.
• Figure 3left and -right shows quaternary fat trees of dimensions 1 and 2, respectively.
• When $n=1$, the network consists of one switch and four processing nodes.
• When $n=2$, the network consists of eight switches and 16 processing nodes.

• Elite networks are source routed.
• The Elan network interface attaches route information to the packet header before transmitting the packet into the network.
• The route information is a sequence of Elite link tags.
• As the packet moves inside the network, each switch removes the first route tag from the header and forwards the packet to the next switch in the route or the final destination.
• Packets are routed using wormhole routing flow control (each packet is divided into flow control digits (flits)).
• In QsNet, the size of each flit is 16 bits. Network nodes can send packets to multiple destinations using the network's broadcast capability.

Table 1: List of local area networks device bandwidths.

Interconnection	Data Rate	Data Rate	Year
Technology	(bit/s)	(byte/s)
Ethernet (10BASE-X)	10 Mbit/s	1.25 MB/s	1990
Fast Ethernet (100BASE-X)	100 Mbit/s	12.5 MB/s	1995
FDDI	100 Mbit/s	12.5 MB/s
Token Ring IEEE 802.5v	1 Gbit/s	125 MB/s	2001
Gigabit Ethernet (1000BASE-X)	1 Gbit/s	125 MB/s	1998
Myrinet 2000	2 Gbit/s	250 MB/s
Infiniband SDR 1X[24]	2 Gbit/s	250 MB/s
Quadrics QsNetI	3.6 Gbit/s	450 MB/s
Infiniband DDR 1X[24]	4 Gbit/s	500 MB/s
Infiniband QDR 1X[24]	8 Gbit/s	1 GB/s
Infiniband SDR 4X[24]	8 Gbit/s	1 GB/s

Table 2: List of local area networks device bandwidths, Contnd.

Interconnection	Data Rate	Data Rate	Year
Technology	(bit/s)	(byte/s)
Quadrics QsNetII	8 Gbit/s	1 GB/s
10 Gigabit Ethernet (10GBASE-X)	10 Gbit/s	1.25 GB/s
Myri 10G	10 Gbit/s	1.25 GB/s
Infiniband DDR 4X[24]	16 Gbit/s	2 GB/s
Scalable Coherent Interface
(SCI) Dual Channel SCI, x8 PCIe	20 Gbit/s	2.5 GB/s
Infiniband SDR 12X[24]	24 Gbit/s	3 GB/s
Infiniband QDR 4X[24]	32 Gbit/s	4 GB/s
40 Gigabit Ethernet (40GBASE-X)	40 Gbit/s	5 GB/s
Infiniband DDR 12X[24]	48 Gbit/s	6 GB/s
Infiniband QDR 12X[24]	96 Gbit/s	12 GB/s
100 Gigabit Ethernet (100GBASE-X)	100 Gbit/s	12.5 GB/s