Enhance your AI experience by running a private chatbot on your PC

You can use Ollama, ChatRTX, and GPT4ALL

3 min. read

Published onMay 17, 2024

published onMay 17, 2024

Share this article

Read our disclosure page to find out how can you help Windows Report sustain the editorial teamRead more

The PC industry works on bringing AI chatbots that run directlyon your deviceinstead of a faraway center. The process will take time, and you will likely need to buy anAI PC. Also, there is a chance that it will come with a popular AI installed like Copilot, especially if it is a laptop. However, if you don’t want to wait for AI PCs, you can start running private AI chatbots on your PC with the right tool.

What are the benefits of running a private AI chatbot on your PC?

Running a private AI chatbot on your PC brings many benefits. For example, they don’t need an internet connection, and you know exactly where your data goes. Also, they might have fewer limitations than the popular ones. So, you might see fewer messages like “I’m sorry, but I can’t continue this conversation” or “Let’s try a different topic.”

The future of AI could consist of private chatbots running on your PC. After all, they might perform tasks better than their public counterparts. Yet, before we get there, you can use Ollama.

Ollama is an open-source project named after Llama LLM from Meta. It allows you to run private AI chatbots on your PC. The tool is easy to install and use and works on Windows 10, 11, Linux, and macOS. In addition, it uses your hardware to accelerate the AI models. With it, you won’t need a PC with a40 TOPS NPUperformance because it runs on various PCs.

Once installed, to useOllama, open a command-line window like Terminal, Command Prompt, or PowerShell. To access the command line, pressWin + X. Then, to use an AI chatbot, type:ollama run llama2. Afterward, the tool will download the AI model and its required files. When the installation is over, you will see the following: »> Send a message (/? for help), and you can start using the chatbot. Then, to close the session type/byeand pressEnter.

There are plenty of AI chatbots available on the Ollama site. For example, you can useLlama 3from Meta, Mistral, Gema from Google, Phi-3 Mini from Microsoft, and more. Also, if you don’t want to use Ollama, tryOpen WebUI.

Additional tools

In addition, if you want to use a private AI model on your PC, you can getNvidia ChatRTX. However, it requires a PC with a 30-series or 40-series GPU with at least 8 GB of VRAM. Nvidia ChatRTX has a web-based interface and can run LLama 2, Mistral, and Gemma.

Another tool you could run on your PC isGPT4ALL, but it has fewer private AI chatbots available. Yet, it doesn’t require high-end hardware and works on various systems.

Ultimately, there are a few ways in which you can install and run private AI chatbots on your PC. Most of them don’t require high-end devices and they can keep your data safe. If you decide to get one for your PC, make sure that you scan it for malware. Also, don’t forget that they have limited resources compared to public chatbots.

Are you going to use a private AI model? Let us know in the comments.

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low#

Copilot in Outlook will generate personalized themes for you to customize the app#

Microsoft will raise the price of its 365 Suite to include AI capabilities#

Death Stranding Director’s Cut is now Xbox X|S at a huge discount#

Outlook will let users create custom account icons so they can tell their accounts apart easier#

Enhance your AI experience by running a private chatbot on your PC#

What are the benefits of running a private AI chatbot on your PC?#

Additional tools#

With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low

Copilot in Outlook will generate personalized themes for you to customize the app

Microsoft will raise the price of its 365 Suite to include AI capabilities

Death Stranding Director’s Cut is now Xbox X|S at a huge discount

Outlook will let users create custom account icons so they can tell their accounts apart easier

Enhance your AI experience by running a private chatbot on your PC

What are the benefits of running a private AI chatbot on your PC?

Additional tools