Nvidia has introduced a generative AI (genAI) chatbot that can operate on Windows PCs, providing businesses with the opportunity to utilize AI on employees’ local environments to enhance productivity without relying on genAI tools hosted by providers such as OpenAI.
The company has launched Chat with RTX, a demo app that is now available for free download. It allows users to customize the chatbot with their own content, enabling them to personalize the data sources of the bot’s large language models (LLMs) while keeping their private data on their PC. This ensures fast results and data privacy, as the chatbot runs locally on Windows RTX PCs and workstations without the need for cloud-based LLM services.
Chat with RTX offers a choice of two open-source LLMs, Mistral or Llama 2, and requires Nvidia GeForce RTX 30 Series GPU or higher with at least 8GB of video RAM, running on Windows 10 or 11 with the latest NVIDIA GPU drivers. The chatbot utilizes retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software, and Nvidia RTX acceleration to provide quick and accurate responses to user queries.
Nvidia is positioning itself as a leading supplier of hardware and software to power and “democratize” AI technology. The chatbot supports various file formats, including text, pdf, doc/docx, and xml, and allows users to add data to the chatbot’s library by pointing the application at a folder containing files. Additionally, users can provide the URL of a YouTube playlist, and Chat with RTX will load the transcriptions of the videos in the playlist, enabling people to query the content they cover.
Furthermore, developers can build their own RAG-based apps for the platform, as Chat with RTX is built from the TensorRT-LLM RAG developer reference project available from GitHub, according to NVIDIA.
The adoption of genAI-based chatbots like Open-AI’s ChatGPT is on the rise, and Nvidia’s Chat with RTX appears to be a strategic move to address this trend by providing a local, personalized AI solution that prioritizes data privacy and user control.
2024-02-16 09:00:04
Post from www.computerworld.com