Kotaemon: Open-Source RAG Chat for Documents

3 min readSep 1, 2024

Cloudways — The Best Managed Cloud Hosting | Web Hosting

Kotaemon is an open source tool based on Retrieval Augmentation Generation (RAG) that aims to enable the ability to talk to documents. The tool provides a clean and customizable UI for end users and developers, enabling users to conduct Q&A on their own documents and allowing developers to build their own RAG pipelines.

🔍 Open source RAG UI for document QA

🛠️ Support for local LLMs and API providers

📊 Hybrid RAG pipeline with full-text and vector search capabilities

🖼️ Multimodal QA with graph and table support

📄 Advanced citations with in-browser PDF preview

🧠 Complex reasoning for problem decomposition

⚙️ Configurable settings UI

🔧 Extensible architecture based on Gradio

Main Features OF Kotaemon

Documentation QA Web-UI

Kotaemon provides a Web-UI that supports multi-user login, where users can organize files, create public or private collections, and share chat history with others. This makes it easy for users to conduct document Q&A and manage and share their usage experience.

Multimode and Hybrid RAG Pipeline

Supports multiple large language models (LLMs) and embedding models, including local models and popular API providers such as OpenAI, Azure, Ollama, etc. Kotaemon uses a hybrid (full-text search and vector) retriever and re-ranking technology to ensure the best retrieval quality, supports multi-modal QA, and is able to parse documents containing charts and tables.

Advanced citation and document preview

By default, detailed citations are provided to ensure the correctness of your LLM answers. Users can view citations in the browser’s built-in PDF viewer, and the document highlights relevant content and issues warnings when retrieved articles are less relevant.

Complex reasoning support

Kotaemon supports answering complex or multi-hop questions using question decomposition, and supports agent-based reasoning methods such as ReAct and ReWOO, etc. These features enable Kotaemon to handle more complex question-answering scenarios.

Configurable UI and scalability

Users can adjust various key parameters of the retrieval and generation process in the UI, and since Kotaemon is built on Gradio, users can freely customize or add any UI elements. The project also supports multiple document indexing and retrieval strategies, and provides a GraphRAG indexing pipeline as an example.

Installation and Deployment

The project provides a simple installation script. Users can quickly deploy the server through Docker, or clone the project locally and configure environment variables to start the service. The default username and password are admin/admin, and users can set other users directly in the UI.

Scenarios and Development Prospects of Kotaemon

Kotaemon is suitable for scenarios that require complex question-and-answer sessions on documents, such as the construction of internal knowledge bases in enterprises, analysis of research literature, and management of learning resources in the field of education. Its open source nature and high customizability allow developers to further expand and optimize system functionality based on specific needs.