We present the first end-to-end, transformer-based table question answering(QA) system that takes natural language questions and massive table corpus asinputs to retrieve the most relevant tables and locate the correct table cellsto answer the question. Our system, CLTR, extends the current state-of-the-artQA over tables model to build an end-to-end table QA architecture. This systemhas successfully tackled many real-world table QA problems with a simple,unified pipeline. Our proposed system can also generate a heatmap of candidatecolumns and rows over complex tables and allow users to quickly identify thecorrect cells to answer questions. In addition, we introduce two newopen-domain benchmarks, E2E_WTQ and E2E_GNQ, consisting of 2,005 naturallanguage questions over 76,242 tables. The benchmarks are designed to validateCLTR as well as accommodate future table retrieval and end-to-end table QAresearch and experiments. Our experiments demonstrate that our system is thecurrent state-of-the-art model on the table retrieval task and producespromising results for end-to-end table QA.