How to Install Vicuna (Local ChatGPT)

Introduction to Vicuna

Vicuna is a free internet model trained on shared GPT and a database of conversations by other users extracted from ChatGPT. Vicuna can achieve up to 90% of the capabilities of Chat GPT.

Installing Vicuna

To use Vicuna, there are three ways: using it on your own computer with your GPU, using your CPU, or simply by using an online solution.

The best way to use a large language model on your own computer is by using the Ubabooga text generation web UI, an open-source web UI for running large language models like llama alpaca, GPT Galactica and, of course, Vicuna.

The interface is very similar to the stable diffusion automatic 11 web UI.

To install UbaBooga

1. Click the link in the description below
2. Scroll down and click on “One-click installer for Windows”
3. Extract the downloaded zip file
4. Run “install.bat” file and follow instructions
5. Run “downloadmodel.bat” file to download models
6. Choose which model you want to download (e.g., opt 6.7 billion parameters)
7. Wait for the download to complete
8. Once everything is installed, you should have two new folders: installer files and text generation web UI.

Using Vicuna

To use Vicuna:

1. Double click on “start with ui.bat” file inside the text generation web UI folder.
2. Select which model you want to use (e.g., Facebook opt 6.7 billion parameters or Vicuna 13 billion parameters).
3. Start generating text!

If you encounter any issues while installing or running Vicuna, refer to the instructions file inside the text generation web UI folder.

Setting up the Text Generation Web UI

This section covers how to set up the text generation web UI.

Downloading and Saving Tensor File

• Download the tensor file from the Hugging Face page repository.
• Cut and paste it inside afolder so that all files in the folder match those in the Hugging Face repository.

Editing ui.bat File

• (t=0:04:37s) Right-click on ui.bat file and select “Edit with Notepad.”
• Add two arguments, “–W bits 4” and “–group size 128,” after “–chat”.
• Save changes.

Running Text Generation Web UI

1. Double-click on the start with ui.bat file.
2. Choose the vacuum model when prompted.
3. Hold Ctrl and left-click to open the local URL.
4. Use the input box to type text, press enter or click generate to send text.
5. Use the stop button to halt response generation.
6. Use the impersonate button for responses in the chatbot’s personality style.
7. Use the regenerate button to get a new response from the chatbot.
8. Use the copy last reply button to automatically copy the previous response.
9. Use the remove last reply button to delete the previous answer.
10. Use the clear history button to delete the entire conversation.

Interface Modes

Chat Interface Mode

This mode resembles the Cha GPT interface.

Default Mode

This mode has an interface where you can input a question and click generate for a response.

Customizable Modes

Other customizable modes are also available.

Introduction to the Interface

This section introduces the interface used for the chatbot and explains how it works.

Choosing the Chat Interface

• The chatbot will not have memory from previous conversations if you don’t choose the chat interface.
• This means that you won’t have the same experience as if you were using Chat GPT.
• However, this interface is interesting because it allows you to nudge the conversation towards something that you want to hear.

Using Vicuna Prompts

• You can use Vicuna prompts to generate responses from the chatbot.
• If you ask something illegal, it will tell you that it cannot answer your question.
• By writing a sentence for the chatbot, you can nudge the conversation towards something that you want to hear.

Using Vacuna Prompt Syntax

• The official syntax for Vacuna prompts is recommended for better results.
• A preset file is available in the description below.
• To use this syntax, download and extract the file into your prompts folder and refresh your web UI.

Installing Lava.cpp

This section explains how to install and use Lava.cpp, which allows users to run Vicuna on their own computer using their CPU.

Installation Process

Download the avx.zip and 13B 4-bit rev1.bin files from the links provided in the description below.

• Extract both files and paste them into a new folder.
• Download the run with aquino.bat file from the last Mega link provided in the description below.
• Double-click on the run with aquino.bat file to start running Vicuna on your computer using your CPU.

Features of Vicuna

• You can use the demo website anywhere you want – on your computer, tablet, or phone. It works exactly the same way as if it was installed on your computer.
• This is probably the most powerful model that I ever tried to run on my computer, and you can use it for absolutely free.
• The speaker thanks his supporters for making these videos possible and encourages viewers to subscribe and smash the like button.

Overall, Vicunia is a powerful and easy-to-use NLM model that can be used for free. It has many features such as being able to run on any device with an internet connection.

Vicuna Resources:

Vicuna: https://vicuna.lmsys.org/

Official Github

Online demo

Oobabooga Webui

Arguments to put inside the start-webui.bat file: –wbits 4 –groupsize 128

LLAMA.cpp

Git: https://git-scm.com/downloads

Vicuna prompts preset

Vicuna.bat file

ABOUT THE AUTHOR

Dave Halmai

Founder of AI Sashimi. I write about AI, ChatGPT, business acceleration, SEO and content marketing. My hobbies are blogging, investing, hiking and reading.

Follow my posts on LinkedIN

Keep Reading