Abstract
In this paper, we consider the multichannel rendezvous problem in cognitiveradio networks (CRNs) where the probability that two users hopping on the samechannel have a successful rendezvous is a function of channel states. Thechannel states are modeled by two-state Markov chains that have a good stateand a bad state. These channel states are not observable by the users. For sucha multichannel rendezvous problem, we are interested in finding the optimalpolicy to minimize the expected time-to-rendezvous (ETTR) among the class of{\em dynamic blind rendezvous policies}, i.e., at the $t^{th}$ time slot eachuser selects channel $i$ independently with probability $p_i(t)$, $i=1,2,\ldots, N$. By formulating such a multichannel rendezvous problem as anadversarial bandit problem, we propose using a reinforcement learning approachto learn the channel selection probabilities $p_i(t)$, $i=1,2, \ldots, N$. Ourexperimental results show that the reinforcement learning approach is veryeffective and yields comparable ETTRs when comparing to various approximationpolicies in the literature.