Abstract
Conventional retransmission (ARQ) protocols are designed with the goal ofensuring the correct reception of all the individual transmitter's packets atthe receiver. When the transmitter is a learner communicating with a teacher,this goal is at odds with the actual aim of the learner, which is that ofeliciting the most relevant label information from the teacher. Taking anactive learning perspective, this paper addresses the following key protocoldesign questions: (i) Active batch selection: Which batch of inputs should besent to the teacher to acquire the most useful information and thus reduce thenumber of required communication rounds? (ii) Batch encoding: Can batches ofdata points be combined to reduce the communication resources required at eachcommunication round? Specifically, this work introducesCommunication-Constrained Bayesian Active Knowledge Distillation (CC-BAKD), anovel protocol that integrates Bayesian active learning with compression via alinear mix-up mechanism. Comparisons with existing active learning protocolsdemonstrate the advantages of the proposed approach.