Myrinet Clos Network

• Myrinet is a high-performance, packet-communication and switching technology.
• It was produced by Myricom as a high-performance alternative to conventional Ethernet networks.
• Myrinet switches are multiple-port (4, 8, 12, 16) components that route a packet entering on an input channel of a port to the output channel of the port selected by the packet.
• For an $n$-port switch, the ports are addressed $0, 1, 2,\ldots, n-1$.
• For any switching permutation, there may be as many packets traversing a switch concurrently as the switch has ports.

• These switches are implemented using two types of chips: crossbar-switch chips and the Myrinet-interface chip.
• The basic building block of the Myrinet-2000 network is a 16-port Myrinet crossbar switch, implemented on a single chip designated as Xbar16.
• It can be interconnected to build various topologies of varying sizes.
• The most common topology is the Clos network.

Figure 2: A 64-host Clos network using 16-port Myrinet switch (each line represents two links).

• A network of 64 hosts or fewer would require eight-port switches for the spine.
• In the figure, an Xbar16 switch can serve the purpose of two 8-port switches.
• The thick line connecting a spine switch to a leaf switch represents two links.

Figure 3: A 128-host Clos network using 16-port Myrinet switch, which includes 24 Xbar16s.

• Each Xbar16 switch is represented using a circle.
• The eight switches forming the upper row is the Clos network spine, which is connected through a Clos spreader network to the 16 leaf switches forming the lower row.
• The Clos network provides routes from any host to any other host.
• Routes between hosts connected to different Xbar16s traverse three Xbar16 switches.

• The routing of Myrinet packets is based on the source routing approach.
• Each Myrinet packet has a variable length header with complete routing information.
• When a packet enters a switch, the leading byte of the header determines the outgoing port before being stripped off the packet header.
• At the host interface, a control program is executed to perform source-route translation.