Kernel packs
A kernel pack holds everything goethe needs to serve one model with optimised kernels: the kernels for every supported GPU microarchitecture, the quantizations and the model configs. Packs currently target NVIDIA GPUs (CUDA); packs for other platforms, such as AMD ROCm, will follow. Packs are only available to nodes that are logged in to the platform.
When you install a model with goethe and a pack supports that model on your GPU:
- Logged-in node: goethe installs the pack and serves the model with it. There is nothing to do.
- Node that is not logged in: goethe asks you to log in with
goethe login, or continues without the pack and serves the model with SGLang.
goethe models marks the models that have optimised kernels for your machine.