Two-Hop Relay Probing in WiGig Device-to-Device Networks Using Sleeping Contextual Bandits

Millimeter wave (mmWave) relaying has been introduced recently as a solution to extend the coverage of mmWave communication systems and to deal with the blockage problem as well. In order for the relay probing process to identify the most suitable relays, we should maintain an intelligent trade-off between the number of probed relays and the overhead due to beamforming training (BT). This paper leverages an online learning tool, namely sleeping contextual multi-armed bandits (S-CMAB), to effectively address this problem. Thanks to the multi-band capability of WiGig devices that supports both mmWave and WiFi, the characteristics of the WiFi signal centered at 5.25 GHz are used as contexts for the candidate WiGig relays operating at 60 GHz. Moreover, the sleeping relays that are unable to construct a WiGig link, due to blockages for instance, could be identified during the online learning process and accordingly excluded. Extensive simulations prove that the proposed S-CMAB approach integrated with the proposed sleeping linear upper confidence bound (S-LinUCB) algorithm outperform the legacy approaches and the context-free UCB algorithm in terms of both the average throughput and energy efficiency.