The MU, BG/Q’s network interface integrated on-chip, is operated using injection and ejection FIFOs. Each communi-cation (that can be either a mailbox send to another FIFO or an RDMA put) is initiated by inserting a pointer to a commu-nication descriptor into a FIFO. Each receiving FIFO can be polled during program execution, with a pre-defined user han-dler. There are 544 FIFOs on each node shared across 64 threads, providing ample opportunities to implement quality of service across multiple user-defined communication chan-nels.