LAVIS: A Library for Language-Vision Intelligence

  • 2022-09-15 19:04:10
  • Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, Steven C. H. Hoi
  • 25

Abstract

We introduce LAVIS, an open-source deep learning library for LAnguage-VISionresearch and applications. LAVIS aims to serve as a one-stop comprehensivelibrary that brings recent advancements in the language-vision field accessiblefor researchers and practitioners, as well as fertilizing future research anddevelopment. It features a unified interface to easily access state-of-the-artimage-language, video-language models and common datasets. LAVIS supportstraining, evaluation and benchmarking on a rich variety of tasks, includingmultimodal classification, retrieval, captioning, visual question answering,dialogue and pre-training. In the meantime, the library is also highlyextensible and configurable, facilitating future development and customization.In this technical report, we describe design principles, key components andfunctionalities of the library, and also present benchmarking results acrosscommon language-vision tasks. The library is available at:https://github.com/salesforce/LAVIS.

 

Quick Read (beta)

loading the full paper ...