The increasing number of different, incompatible congestion controlalgorithms has led to an increased deployment of fair queuing. Fair queuingisolates each network flow and can thus guarantee fairness for each flow evenif the flows' congestion controls are not inherently fair. So far, each queuein the fair queuing system either has a fixed, static maximum size or ismanaged by an Active Queue Management (AQM) algorithm like CoDel. In this paperwe design an AQM mechanism (Learning Fair Qdisc (LFQ)) that dynamically learnsthe optimal buffer size for each flow according to a specified reward functiononline. We show that our Deep Learning based algorithm can dynamically assignthe optimal queue size to each flow depending on its congestion control, delayand bandwidth. Comparing to competing fair AQM schedulers, it providessignificantly smaller queues while achieving the same or higher throughput.