Instant Loads

The first time a model loads, Gerbil downloads and prepares it. Every load after that is near-instant: prepared weights are saved on the device, so the model is ready in well under a second. And if you start the engine before the user asks for anything, the first answer appears with no wait at all.

Fast by default

There is nothing to configure. Load a model the normal way and the second load is fast automatically, in Node and in the browser:

01import { getEngine } from "@tryhamster/gerbil";
02
03// First run: downloads and prepares the model.
04// Every run after: ready in under a second.
05const engine = await getEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });
06const { text } = await engine.generate("hi");

getEngine() also means you can ask for the same model from anywhere in your app, any number of times, and it only ever loads once. Every caller with the same options gets the same live engine, and callers that arrive mid-load simply wait for the load already in flight:

// route-a.ts
const engine = await getEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });

// route-b.ts, later or even at the same time: no second download,
// no second copy on the GPU. Same engine.
const same = await getEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });

Start before the user asks

For a truly instant first answer, kick the load off early with preloadEngine(): on app start, when a page opens, or when the user hovers a button. It is safe to call and forget; if anything goes wrong, the error shows up where you actually use the engine.

01import { getEngine, preloadEngine } from "@tryhamster/gerbil";
02
03// On app start, page open, or button hover:
04preloadEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });
05
06// Later, when the user actually asks for something,
07// the engine is already warm:
08const engine = await getEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });
09await engine.generate("Summarize this...");

In React

useEngine keeps the model warm for the life of the page. Navigate away and back, switch tabs, remount the component: the model is still there, and the next answer is instant.

01import { useEngine } from "@tryhamster/gerbil/hooks";
02
03function Chat() {
04  // Stays warm across route changes and remounts.
05  const { complete, completion, isLoading, dispose } = useEngine({
06    model: "mlx-community/Qwen3.5-0.8B-4bit",
07  });
08
09  // Free the GPU explicitly if you want (a "Close AI" button):
10  // dispose();
11
12  return <p>{isLoading ? "Loading..." : completion}</p>;
13}

To warm up before any component mounts, the hooks entry ships its own preloadEngine. Call it in the app shell or on hover, and any later useEngine() with the same model picks up the warm engine instead of loading again:

01"use client";
02
03import { useEffect } from "react";
04import { preloadEngine, useEngine } from "@tryhamster/gerbil/hooks";
05
06const MODEL = "mlx-community/Qwen3.5-0.8B-4bit";
07
08function AIPage() {
09  useEffect(() => {
10    preloadEngine({ model: MODEL }); // safe to call and forget
11  }, []);
12  return <Chat />;
13}
14
15// Rendered later, deeper in the tree:
16function Chat() {
17  const { complete, completion } = useEngine({ model: MODEL }); // already warm
18  // ...
19}

Loading progress reaches every component that uses the engine, not just the one that started the load. A component that mounts mid-download shows real progress numbers immediately, through loadingProgress on the hook or the onProgress option on preloadEngine() and getEngine().

Freeing memory

A warm engine holds GPU memory until you let it go. When you are done with a model, release it:

import { releaseEngine } from "@tryhamster/gerbil";

// Release one model...
await releaseEngine({ repo: "mlx-community/Qwen3.5-0.8B-4bit" });

// ...or everything.
await releaseEngine();

In React, useEngine's dispose() does the same, and passing keepAlive: false to the hook tears the engine down automatically once the last component using it unmounts.

How it works

• ·After the first load, Gerbil saves the fully prepared weights (the exact bytes the GPU receives) on the device: on disk in Node (~/.cache/gerbil/<repo>), in browser storage on the web. Warm loads read them straight back and skip the preparation work entirely.
• ·Saved weights are tied to the exact model and settings that produced them. Change the model, revision, or any option that affects the weights and Gerbil prepares fresh ones; you can never get a stale or mismatched model.
• ·A warm load produces exactly the same outputs as a cold one, verified token for token.
• ·Shared engines are keyed by their full configuration (repo, dtype, enableVision, embedding, maxSeqLen, kvMode). Use the same values everywhere to share one engine; a different value is a separate engine.

Turning it off

The prepared-weight cache is on everywhere by default. To disable it (debugging, tight disk space):

# Node
GERBIL_QUANT_CACHE=0 node app.mjs

// Browser: set the flag before the engine loads
globalThis.GERBIL_QUANT_CACHE = false;

To see where a load spends its time, enable the load profiler and a per-phase timing table prints to the console:

# Node
GERBIL_LOAD_PROFILE=1 node app.mjs

// Browser: set before the engine loads
globalThis.GERBIL_LOAD_PROFILE = true;