Nvidia's RTX-Powered Chat with AI: A Game-Changing Local Chatbot for Your PC
Nvidia is releasing an early version of "Chat with RTX" today, a demo app that lets you run a personal AI chat bot on your PC. You can feed it YouTube videos and your own documents to create summaries and get relevant answers based on your own data. This innovative feature runs locally on a PC and all you need is an RTX 30 or 40 series GPU with at least 8 GB of VRAM.
Comprehensive Description and Analysis
In my brief testing of "Chat with RTX" over the past day, I've found that although the app is a little rough around the edges, it has the potential to be a valuable tool for data research, especially for journalists or anyone who needs to analyze a collection of documents.
"Chat with RTX" has the capability to handle YouTube videos, allowing users to input a URL and search for specific mentions or summarize an entire video. This feature is particularly useful for searching through video podcasts, making it easier to find specific mentions within episodes. However, there are still some bugs to be ironed out in this early demo.
When searching through the transcript of a Verge YouTube video, "Chat with RTX" downloaded the transcript for a completely different video that wasn't even related to the query. Despite these early issues, it's evident that the app has the potential to revolutionize video searching and analysis.
/cdn.vox-cdn.com/uploads/chorus_asset/file/3482/slack_response.png)
On the other hand, "Chat with RTX" is exceptional at searching local documents. When it worked properly, it was able to find references and videos within seconds. Personally, I created a dataset of Microsoft documents for "Chat with RTX" to analyze when covering a court case last year. It was often overwhelming to search through a large number of documents at speed, but with "Chat with RTX", I was able to query them nearly instantly on my PC.
For example, the chatbot did a commendable job summarizing Microsoft's entire Xbox Game Pass strategy from internal documents revealed during the trial. It provided valuable information, explaining that Xbox Game Pass is a content subscription service in gaming that offers access to a library of games for a single monthly fee. The service aims to empower players to play their games anywhere and allows publishers to reach players everywhere. With a potential subscriber base of 750 million, scaling Xbox Game Pass is a primary strategic objective for Microsoft.
Additionally, "Chat with RTX" proved to be useful in quickly skimming through PDFs and fact-checking data. While Microsoft's co-pilot system struggles with PDFs within Word, "Chat with RTX" had no trouble extracting key information. The app's responses are nearly instant, without the lag typically experienced with cloud-based chat GPT or co-pilot chat bots.
However, it is important to note that "Chat with RTX" is still in the early stages and feels more like a developer demo. During installation, it sets up a web server and python instance on your PC, utilizing MSTR or Llamas 2 models to query the data you feed it. Furthermore, it leverages Nvidia's tensor cores on an RTX GPU to expedite your queries.
Installing "Chat with RTX" took approximately 30 minutes on my PC, which is powered by an Intel Core i9-114900K processor with an RTX 490 GPU. The app consumes nearly 40 GB of storage space, and the python instance occupies around 3 GB of the available 64 GB of RAM on my system. Once it's up and running, you can access "Chat with RTX" through a browser while a command prompt displays what's being processed, including any error codes that may appear.
Nvidia acknowledges that "Chat with RTX" is not a polished app for all RTX owners to download and install immediately. There are several known issues and limitations, such as source attribution not always being accurate. Additionally, attempting to index a large number of documents can cause the app to crash, requiring you to clear the preferences to resume using it. The app also lacks the ability to remember context, so follow-up questions cannot be based on the previous query. As a cautionary note, it creates JSON files within the folders you ask it to index, so it is advisable not to use it on your entire documents folder in Windows.
Conclusion
In conclusion, "Chat with RTX" is a promising tech demo showcasing the potential of an AI chatbot that operates locally on your PC. It offers valuable features for data research, particularly for journalists and individuals who need to analyze extensive collections of documents. While there are still some issues to be addressed and improvements to be made, the app demonstrates the power and capabilities of the RTX GPU series combined with AI technology.
FAQs
-
Q: Is "Chat with RTX" available for all GPUs?
A: No, it requires an RTX 30 or 40 series GPU with at least 8 GB of VRAM.
-
Q: Can "Chat with RTX" search through YouTube transcripts?
A: Yes, it can search through YouTube transcripts and provide specific mentions or summaries.
-
Q: Does "Chat with RTX" handle PDFs well?
A: Yes, it can extract key information from PDFs quickly and efficiently.
-
Q: What are the limitations of "Chat with RTX"?
A: Known limitations include inaccurate source attribution, difficulty indexing a large number of documents, and the inability to remember context for follow-up questions.
-
Q: Is "Chat with RTX" recommended for analyzing personal files?
A: While it can be useful, it is advisable not to use it on your entire documents folder in Windows due to the creation of JSON files.
