Trajectory prediction is a safety-critical tool for autonomous vehicles toplan and execute actions. Our work addresses two key challenges in trajectoryprediction, learning multimodal outputs, and better predictions by imposingconstraints using driving knowledge. Recent methods have achieved strongperformances using Multi-Choice Learning objectives like winner-takes-all (WTA)or best-of-many. But the impact of those methods in learning diverse hypothesesis under-studied as such objectives highly depend on their initialization fordiversity. As our first contribution, we propose a novel Divide-And-Conquer(DAC) approach that acts as a better initialization technique to WTA objective,resulting in diverse outputs without any spurious modes. Our secondcontribution is a novel trajectory prediction framework called ALAN that usesexisting lane centerlines as anchors to provide trajectories constrained to theinput lanes. Our framework provides multi-agent trajectory outputs in a forwardpass by capturing interactions through hypercolumn descriptors andincorporating scene information in the form of rasterized images and per-agentlane anchors. Experiments on synthetic and real data show that the proposed DACcaptures the data distribution better compare to other WTA family ofobjectives. Further, we show that our ALAN approach provides on par or betterperformance with SOTA methods evaluated on Nuscenes urban driving benchmark.