ChatGPT Outclassed By OpenChat-3.5–7B
OpenChat-3.5–7B: Surpassing ChatGPT on various benchmarks
OpenChat uses C-RLFT, a strategy inspired by offline reinforcement learning, for fine-tuning.
It improves model performance by analyzing existing conversation data and feedback. You can also learn from your mistakes.
After testing it, although it is only 7B in size, the effect is indeed as good as GPT.
What’s awesome is that it runs on a consumer-grade GPU with 24GB RAM.
OpenChat also provides a Web UI interface to facilitate users to interact with the model.
Performance and evaluation of OpenChat-3.5–7B:
In practical applications, OpenChat demonstrates excellent performance. It performs well in multiple benchmarks, especially in following instructions and generalizing capabilities, outperforming other similar open source language models.
In terms of benchmarking, OpenChat-3.5’s 7B model achieved an average score of 61.6 across multiple tests, surpassing ChatGPT (March version)’s 61.5.
OpenChat-3.5–7B in the competition of https://t.co/NdVm3fj0CM Grok with 33 billion parameters
How OpenChat-3.5–7B works:
1. Pre-trained language model: The core of OpenChat is a large-scale pre-trained language model. These models master the structure, syntax, and semantics of language by analyzing and learning large amounts of text data. This enables OpenChat to understand user input and generate smooth, coherent responses.
2. Fine-tuning method (C-RLFT): OpenChat adopts a method called conditional reinforcement learning fine-tuning (Conditioned-RLFT, C-RLFT). This approach is particularly suitable for processing data of mixed quality. In traditional fine-tuning methods, all training data are considered equally important, which may cause the model to perform poorly when dealing with data of varying quality. C-RLFT enables the model to learn from different data sources more efficiently by treating these data as different reward labels.
3. Class-conditional policy learning: In C-RLFT, OpenChat learns a class-conditional policy, which means it can adjust its response based on the type of input data (e.g., different data sources or quality). This strategy makes OpenChat more flexible and efficient in handling various different types of input.
4. Single-stage supervised learning: OpenChat uses a single-stage supervised learning method. This approach does not rely on traditional reinforcement learning techniques, but instead optimizes the model by maximizing rewards and reducing differences from a reference policy. This approach improves learning efficiency and helps reduce errors during training.
Details
GitHub
Paper
Try online
