Using the assistant trains nothing. The models arrive pre-trained on open-weight standards, and no keystroke moves a weight. Which leaves the question a program lead actually asks. Does this use case need a true fine tune, or would a fine tune create problems with the data?
Fine-Tuning & Model Provenance
Can a Local LLM Be Fine-Tuned
on Your Own Data?
The rule, what a run costs, and why a model’s origin narrows the menu.
Built from real buyer questions in our sales meetings
Direct Answer
Yes, and for most knowledge problems it should not be your first move. Iternal will tune a corporate-approved base model you name or bring, returning weights on an open-source base such as Llama or Gemma that you run yourself. Domain accuracy arrives sooner from a well-structured corpus supplied at answer time. Tune for style, format and narrow task behavior; supply facts rather than bake them in.
The limit: whatever you tune, you have signed up to tune again. Iternal says it of its own work — the fine-tuned models behind Blockify ingestion are dated, because the field moves fast enough that they could stand re-tuning. Building the corpus moves that maintenance onto the documents rather than ending it. Origin narrows the menu too: the open-weight models that would run Iternal’s hardest workload well were newly released models at frontier parity out of Chinese labs, which many of the enterprises we work with will not run.
Three things to get in writing before commissioning a run. Which base models may be tuned, and whether the one your security team cleared is among them. Who owns the finished weights, and who repeats the run when the base generation turns over. What one run costs in time and GPU compute. Iternal answers the targeted questions below.
Tuning changes how a model behaves; documents change what it reads. Only one has to be redone when a contract clause changes. For more information visit the local model overview or the data readiness page.
Tune Behavior, Not Facts
The rule fits in a line: tune a model for how it behaves, hand it documents for what it knows. Facts move; weights sit still. Tune on this quarter’s contract terms and the model is wrong the day they change. Iternal’s own framing: AI gets better from a larger and cleaner data store, not from training the model.
Three conditions flip that decision, and each is about behavior rather than knowledge:
- A fixed output format the documents cannot teach. Iternal found XML the most reliable format for its tuned models to produce. Without the tune, the same extraction needed a 20,000-token prompt and a model in the hundreds of billions of parameters.
- A narrow, repetitive task. For a classifier, Iternal recommends tuning a small model to run it: a model tuned to one job beats a general model told the job on every request.
- A domain vocabulary the base model mis-reads. A research department running models tuned to its own field is supported: AirgapAI runs domain-specific tuned models, and AirgapAI Edge runs any open-source model optimized for Intel.
Get it backwards and the tune ships beautifully, then decays into a maintenance schedule.
The Model Does Not Learn While You Use It
Buyers ask whether the assistant carries persistent memory, or only learns as it goes.
The premise is what needs correcting. No production model learns continuously. Cloud providers batch-retrain on thousands of GPUs on their own schedule; a local model stays current because somebody adds recent documents. Context lives inside a thread and ends with it, and each user profile runs a separate instance with no sight of anyone else’s work. Sanitizing an environment removes chat histories and generated artifacts and leaves the weights alone. Iternal’s view of the horizon is hedged: a continuous-learning breakthrough may arrive in a year or two, and does not exist today.
Pointing the Application at a Model, a Relay or a Lab
The application still has to be told where its intelligence lives. Three routes, and most deployments use two.
- Download a model onto the device. Models are not bundled in the installer: the right one depends on the machine, and the installer cannot carry a file that size. A control in the app opens Iternal’s model page, which lists models recommended per form factor. Downloads run once and need no unzipping; keep the lid open, because closing the laptop stops one in progress.
- Point it at a server endpoint. A server connection tab in Settings aims the client at a model server instead of a local file. Supply an OpenAI-compatible endpoint URL including the /v1 segment; these connections typically need no API key, though a lab endpoint may issue one. Refresh, and the endpoint appears in the chat model selection.
- Run the relay. The Iternal relay server runs on the client device and reaches compute elsewhere — a data-center GPU box, a lab instance, an external provider. One documented command, with an open-source PowerShell build in service today. Standing up the GPU model server is the hard half; the relay is not.
Two frictions. Loading the model is the most technical step a new user faces, so do it during imaging; and setting the connection user by user does not scale, so push the endpoint through device management.
Where a Model Came From, and Why That Narrows the Menu
Provenance, not accuracy, is the gating concern customers raise about an agentic stack. A model of Chinese origin could put a poison pill into generated code with no visibility into it, and such models are not appropriate for a defense program. Not preferred, one buyer said, even when they are the best available.
The argument on the other side is real, and narrower than it looks. A model is data rather than running software. Once weights sit inside a closed environment on a machine with no route out, the exfiltration story is finished. That settles where your material goes, not what comes back: generated work still has to be reviewed, which is why security teams asked for human eyes on everything such a model produces and made stringent code review a condition. Terms of use are a third question, with disputes adjudicated in China under Chinese law. The bar also moves by sector: provenance weighs less on a commercial buyer than a federal one.
Iternal’s position is practical. It does not advocate a model of Chinese origin for production use and is optimizing Gemma as the alternative, already in the AirgapAI library. Settle origin per model, in writing:
Pin it down: questions for your evaluation
-
Which base models can we tune, and is the one our security team cleared among them?Whether the engagement starts from a model you can defend.
-
What is the country of origin of every model you would offer us, and which do you support for production?Origin answered per model rather than per product.
-
Who holds the tuned weights, and who pays for the re-tune when the base generation turns over?Ownership of the deliverable, plus the maintenance a first quote omits.
Chinese LLMs: What DeepSeek, Qwen, Kimi and GLM Licenses Allow
Chinese LLMs are open-weight models from DeepSeek, Alibaba’s Qwen, Moonshot’s Kimi and Z.ai’s GLM, most published under permissive MIT or Apache 2.0 terms. The weights download and run offline like any other file. What a regulated buyer is deciding is not access but provenance: who built it, what the license grants, and who reviews the output.
| Model family | Lab | Published license | What that permits |
|---|---|---|---|
| DeepSeek-R1, DeepSeek-V3 | DeepSeek, Hangzhou | MIT on the released R1 weights | Commercial use, modification and redistribution, including derivative models you tune yourself. |
| Qwen3, Qwen2.5 | Alibaba Cloud | Apache 2.0 across the Qwen3 sizes; some earlier Qwen releases carry a separate Qwen license | Commercial use with the attribution and patent grant Apache 2.0 carries; read the earlier releases individually. |
| Kimi K2 | Moonshot AI | Modified MIT, with an attribution condition above a large commercial-usage threshold named in the license | Commercial use; a product past that threshold has to credit the model on its interface. |
| GLM-4.5, GLM-4.6 | Z.ai | MIT on the released weights | Commercial use, modification and redistribution. |
Terms move with the checkpoint. Read the license file on the model card published with the exact weights you intend to load, not the one you read for a sibling size or an earlier release.
Open weights mean the file, not the service. Each of these labs publishes downloadable weights and also runs a hosted API, and only the first is what a closed deployment uses. The checkpoint is copied in, loaded by a local runtime, and answers with no outbound call. The hosted API is a separate product with its own terms and its own data path, and it is usually the route a regulated program is turning down when it turns down the model.
What a disconnected environment settles, and what it leaves. Weights are a file rather than a running service, so once they sit on hardware with no route out, where your material goes is answered. Two things survive that. Generated work still has to be reviewed before it is committed or run, and the license attached to the exact checkpoint still governs what you may ship. NIST’s AI Risk Management Framework (AI RMF 1.0) treats both as governance of third-party components rather than as a question about model accuracy.
Iternal ships Gemma as the recommended alternative in the AirgapAI library for programs that will not take the origin. For more information visit the LLM selection guide or the local LLM guide.
Before a federal or defense program adopts one
-
Which checkpoint, at which revision, and under which license file?The artifact actually under review. Terms differ between sizes and between releases of the same family.
-
Does the deployment ever call a hosted endpoint, or only the downloaded weights?Whether the hosted service terms apply to the program at all.
-
Who reviews generated code and generated text before it is committed or run?The control that remains once the environment is closed.
-
What is the approved alternative if the program will not accept the origin?A fallback model named before the schedule depends on the first one.
What a Tuning Run Actually Costs
Tuning is a project, not a setting. A run costs GPU compute for its duration, a base model from whatever your tuning service supports — the one Iternal uses offers a short list — and an engineer who owns the result. Runs are long enough that small mistakes get expensive: a broken chat template cost roughly 21 hours of one of Iternal’s own.
Iternal’s ingestion models are custom fine-tunes on a Llama 3.1 8B base, memory-efficient next to the alternatives, replacing a 20,000-token prompt document and a model in the hundreds of billions of parameters.
For a fuller side-by-side of the two columns below, visit the RAG vs fine-tuning page.
| Commission a run | Structure the documents | |
|---|---|---|
| What you buy | Weights tuned to one job. | A corpus the model reads at answer time. |
| When the content changes | Another run. | Re-ingest what changed. |
| When the base generation turns over | Another run, on the new base. | Load the new model; the corpus is unaffected. |
Answered elsewhere
- Which models ship in the library, and how to pick — see the local model overview.
- What memory and silicon a model needs to run — see the sizing page.
- How quickly a model answers on a given machine — see the speed page.
- What model providers charge, and how the frontier gap moves — see the model market page.
- Getting scattered source material into shape first — see the data readiness page.
- Translation, transcription and speech on the device — see the languages and voice page.
FAQ
FAQ: Tuning and Model Origin
Iternal tunes a corporate-approved base model you name or bring and returns weights on an open-source base such as Llama or Gemma, which you run yourself. Using the product trains nothing: the models arrive pre-trained, and you supply only your data.
When the requirement is behavior rather than knowledge: a fixed output format the documents cannot teach, a narrow repetitive task, or a domain vocabulary the base model mis-reads. Tune for facts that change and you have bought a maintenance schedule.
Neither. No production model learns continuously. Context lives inside a thread and ends with it, and each user profile runs a separate instance with no sight of anyone else’s work.
Three routes. Download one from Iternal’s model page inside the app, once, with no unzipping. Or enter an OpenAI-compatible endpoint URL including the /v1 segment in the server connection tab. Or run the Iternal relay server, which reaches compute elsewhere.
Buyers in defense told us no, even where such a model is the best available. The argument for allowing one is narrow: a model is data rather than running software, so a disconnected environment closes the exfiltration path. Reviewing the output is what remains, and Iternal does not advocate one for production use.
DeepSeek-R1 and the GLM releases are published under MIT, Qwen3 under Apache 2.0, and Kimi K2 under a modified MIT that adds an attribution condition for very large commercial deployments. All four download and run offline. Terms move with the checkpoint, so read the license file on the model card for the exact weights you load.
Decide It on One Workload
Take the use case you were going to tune for and try it the other way first. Build the corpus, load it on hardware you own, and measure. Clear the bar and you have saved a run and every run after it. Fall short and you know what the tune must fix.