Testing Local AI Models: Quality, Speed, and Reliability

#

I’ve been experimenting with LM Studio and a handful of local models. Having an AI assistant work alongside you without sending your data away from your computer is a real blessing, but there are a few things worth checking before relying on one. I designed six tests to compare the models on different kinds of tasks.

These scores are judgments of the responses I received, not claims about each model’s general capability. I ran each task once in LM Studio Bionic 1.1.9-6, leaving the settings at their defaults. Results can change with prompts, sampling settings, and runtime configuration, so this is a practical comparison rather than a controlled scientific benchmark.

The hardware used for this experiment was an ASUS Creator Q540VJ:

I evaluated the models in two ways: how well they handled each task and how long they took to respond. A model might solve a problem well but take so long that it is impractical for everyday use. On the other hand, a fast answer is of little value when the reasoning or science is wrong.

1. Overall results

#

Each test received a subjective score out of 10. The average is an unweighted mean of the six scores, with a zero retained when a model failed to answer the assigned task.

The averages are useful for orientation, but they hide important differences. A model that performs well on writing and logic may still be unreliable at chemistry, while a model with a strong scientific answer can fail to respond to a completely different task. The scores should not be treated as precise measurements or a universal leaderboard.

2. What the six tests revealed

#

Test 1 — Logic and reasoning

#

The task was to relabel three fruit boxes when every label was wrong, using only one sample. Most models solved it correctly. Qwen3.5 9B, Gemma 4 E4B, Gemma 4 12B, Gemma 4 12B QAT, and Gemma 4 E2B gave clear and logically consistent explanations. Bonsai 27B understood the main insight but was less explicit in its final deduction.

Nemotron 3 Nano 4B started correctly but contradicted the premise in its example: its proposed relabelling left one of the original labels correct. This is a small puzzle, but it illustrates an important point—fluent explanations can conceal a basic logical error.

Test 2 — Writing quality

#

The models were asked to write a nuanced essay of roughly 400 words on whether technology makes people more capable or more dependent. This was the closest thing to a draw. All seven produced competent, organised prose, with differences mainly in emphasis and style.

Qwen3.5 9B offered a balanced argument; Bonsai 27B handled the distinction between capability and dependence well; and the Gemma models were generally coherent and polished. Nemotron 3 Nano 4B used concrete examples but developed its counterargument less fully. Some responses also relied on dramatic claims that were not properly supported.

The writing test did not establish a clear winner. More importantly, good prose proved to be a poor predictor of scientific accuracy in the later tests.

Test 3 — Instruction following

#

This task required an explanation of rusting in exactly 150 words and three paragraphs, using specified vocabulary, a chemical equation in the second paragraph, and an analogy in the third.

Nobody satisfied every requirement perfectly. Gemma 4 12B QAT handled the constraints relatively well and supplied a balanced overall reaction. Bonsai 27B met the requested length but used an unbalanced equation. Gemma 4 E4B produced a polished answer that was substantially over the word limit, while several other models also exceeded the requested length or made chemistry errors.

This test separated writing fluency from constraint following. A convincing explanation is not enough when the task includes a precise format or a verifiable technical requirement.

Test 4 — Radical chemistry and spin states

#

This was one of the most revealing tests. The models had to explain radical recombination, electron pairing, singlet and triplet states, spin restrictions, and a mechanism that can enable spin evolution.

The central distinction is that an individual radical with one unpaired electron is ordinarily a doublet, whereas a pair of two such radicals can form combined singlet or triplet spin states. Some answers lost track of the difference between the spin state of an individual radical and that of the pair.

Gemma 4 12B scored highest at 8/10. Its response covered the relevant orbitals and spin functions, although it needed to qualify its claims about triplet states and bond formation. Qwen3.5 9B and Bonsai 27B explained the fundamentals reasonably well but oversimplified parts of the spin restrictions. Gemma 4 12B QAT made a particularly important error by assigning multiplicity 4 to a triplet state; the correct multiplicity is \(2S+1=3\) when \(S=1\). Nemotron 3 Nano 4B and Gemma 4 E2B also made significant conceptual mistakes.

For technical chemistry, these are not cosmetic errors. Confusing multiplicity or the spin state of a radical pair can undermine the rest of an explanation, however sophisticated it sounds.

Test 5 — Problem-solving

#

The task was to identify one poisoned bottle among 1,000 using ten test subjects in a single round. The standard solution assigns each bottle a unique 10-bit binary code; the pattern of subjects who die identifies the bottle. Ten subjects are sufficient because \(2^{10}=1024\).

Nemotron 3 Nano 4B, Gemma 4 E4B, and Gemma 4 12B QAT produced the strongest answers. Qwen3.5 9B used the right method but gave an inconsistent binary example, and Gemma 4 E2B had bottle-numbering errors.

Bonsai 27B and Gemma 4 12B failed this task badly: their recorded responses did not provide a correct solution to the assigned problem. That is more serious than a small arithmetic slip. The test alone cannot tell us whether the failure came from prompt contamination, context handling, or another inference issue, but it does show that their outputs in this run were not dependable. A fresh-chat retest would be needed before drawing a broader conclusion about either model.

Test 6 — Magnetic-field effects on chemical systems

#

This was the most important test for evaluating scientific usefulness. The question was not whether a model could name the Liouville–von Neumann equation, the Zeeman interaction, or Arrhenius kinetics. It was whether it could turn those ideas into a physically defensible plan for investigating magnetic-field effects.

A useful framework should distinguish spin-selective radical-pair chemistry, ordinary temperature-dependent kinetics, magnetic-field effects on spin evolution, and experimental artefacts that may mimic a real effect. It should also avoid treating a single feature—such as a nonlinear field response or anisotropy—as proof of a mechanism.

Bonsai 27B scored highest at 8/10. It proposed density-matrix evolution, a spin Hamiltonian including Zeeman, hyperfine, and exchange interactions, and parameter sweeps over field strength, orientation, and temperature. Those are promising elements for an initial investigation. However, some expected signatures were stated too categorically, and the proposed framework still needed scrutiny before implementation.

Qwen3.5 9B and Gemma 4 12B each scored 7/10. Both gave useful outlines, but their equations and mechanistic distinctions needed refinement. Gemma 4 E4B and E2B offered reasonable starting points without enough detail to produce a simulation-ready model. Gemma 4 12B QAT made several broad or insufficiently justified claims. Nemotron 3 Nano 4B was the weakest response: it introduced questionable relationships between exchange coupling and field strength, supplied code that did not correctly implement its stated model, and made contradictory claims about field dependence.

The key lesson from this test is that equations and scientific vocabulary do not by themselves make a model’s answer scientifically sound. The Hamiltonian, reaction scheme, assumptions, numerical implementation, and predicted observable must all fit together.

3. Speed matters, too

#

The following are the recorded response times across the six tasks. They reflect this particular setup and workload, not an intrinsic speed ranking for every machine or configuration.

The differences are enormous. Gemma 4 E2B completed all six tasks in 2 minutes and 39 seconds. Qwen3.5 9B took just under 14 minutes. Bonsai 27B took more than two hours in total, including 43 minutes and 37 seconds for the problem-solving task it failed.

The instruction-following test makes the trade-off especially clear. Bonsai took 42 minutes and 13 seconds to answer a tightly specified 150-word task, yet its chemical equation was unbalanced. Gemma 4 12B QAT took more than 33 minutes for the same task. By contrast, Gemma 4 E2B took 11 seconds, although its response did not satisfy every constraint.

These results do not prove that larger models are inherently inefficient. Reasoning settings, generation length, prompt processing, quantization, GPU offloading, and runtime behaviour can all influence elapsed time. The experiment shows what happened with these models under the settings used; it does not identify the cause of every delay.

4. Practical verdict

#

For this setup, I’d start with three models and keep a fourth as an optional specialist:

Gemma 4 E2B is the speed champion. Gemma 4 E4B is a useful quality/speed alternative, while Qwen3.5 9B offers the best overall balance across these six tests. Nemotron is worth keeping only if its speed and strong performance on certain structured tasks prove useful in day-to-day work.

Bonsai 27B is the most interesting special case. Its magnetic-field research answer was the strongest in this sample, but the lengthy response time and failed problem-solving test make it difficult to recommend as a default. A clean retest could change that assessment. The 12B models also deserve no automatic preference simply because they are larger.

For photographs, scanned pages, and screenshots, I’d assess visual understanding and OCR separately. A vision-language model may help interpret an image, while a dedicated OCR tool can be better at extracting text. The six tests here are not enough to rank the models for those tasks.

Final verdict

#

The most important finding is not which model came first. It is that convincing prose is not the same as reliable reasoning, and more parameters do not guarantee a better practical experience. Some models wrote well but missed explicit constraints; others used scientific terminology without connecting their equations to a defensible physical model. The slowest answers were not automatically the best.

For everyday local use, Gemma 4 E2B + Gemma 4 E4B + Qwen3.5 9B is the combination I’d try first. Add Nemotron 3 Nano 4B only if it has a clear role in recurring tasks. I would not prioritise the larger, much slower models until their latency and task-integrity failures have been investigated.

For serious scientific work, the next benchmark should move beyond broad research prompts. Give each model the same small, well-defined spin Hamiltonian and initial state, ask it to calculate a verifiable singlet probability, and compare the result with an independent Python calculation. That would test not just whether a model can describe the physics, but whether its proposed calculation can be checked and reproduced.

A local model does not need to do everything. Its value is in being private, available, and responsive for the work it handles well. The right model is the one that is useful for the job—not necessarily the largest one on the list.