Introducing GPUStack 0.2: heterogeneous distributed inference, CPU inference and scheduling strategies
Introducing GPUStack 0.2 GPUStack is an open-source GPU cluster manager for runn
What is GPUStack?
A GPU cluster manager
for running Large Language Models.
GPUStack allows you to create a unified cluster from any brand of GPUs in Apple MacBooks, Windows PCs, and Linux servers. Administrators can deploy LLMs from popular repositories such as Hugging Face. Developers can then access LLMs just as easily as accessing public LLM services from vendors like OpenAI or Microsoft Azure.
What can GPUStack
help you?
For AI Developers
- Access any supported LLM in GPUStack using API keys
- LLM is ready for use after setting a few parameters
- View your token usage and API calls by LLM and date
For Admins
- Deploy, update and remove LLMs as needed
- Add various GPUs into GPUStack from anywhere
- Gain comprehensive visibility for all resources in GPUStack
- Get detailed reports of token usage and operational status of the LLMs
Integration & Serving
GPUStack aggregates all GPU resources within a cluster. It is designed to support all GPU vendors, including Nvidia, Apple, AMD, Intel, Qualcomm, and others. GPUStack is compatible with a laptops, desktops, workstations, and servers running MacOS, Windows, and Linux.
GPUStack can host any LLMs (on the cloud or on premises) and to make their functions available so that applications can incorporate AI into their systems.
Latest Seal News
Read the latest news from our blog.
GPUStack 0.1.2: Enhanced Experience for Deploying Models from Hugging Face
GPUstack 0.1.2 GPUStack 0.1.2 has been released. This update introduces enhancem
Building Free GitHub Copilot Alternative with Continue + GPUStack
Continue Continue is an open-source alternative to GitHub Copilot, this is an op
