Private AI · Model quality index

Which models can investigate your packets?

A plain rating of the self-hostable models we have tested on a real PacketSafari investigation, limited to models that fit a 320 GB VRAM deployment and are not of Chinese origin. Each rating shows the evidence behind it.

Last measured October 2026. This is a first read from a single capture, not a full benchmark.

Ratings

ModelOrigin and licenceVRAMRatingEvidence from the test

Mistral Medium 3.5 (128B dense)France · Mistral licenceAbout 130 GB (fp8)Good

Found the compromised host, PowerShell malware downloads, command-and-control beaconing, and 260 to 750 KB uploads, with frames cited. Did not name the malware family.

NVIDIA Nemotron 3.5 Super VL (120B, 12B active)USA · Open weightsAbout 120 to 240 GBGood

Found the PowerShell payload download and command-and-control beaconing with a victim identifier, with frames cited. Missed the data uploads and added speculative claims about shellcode.

Llama 3.3 70B InstructUSA · Llama licenceAbout 40 to 140 GBPoor

Read only the first overview and reported TCP anomalies. No security finding.

Llama 4 ScoutUSA · Llama licenceAbout 65 to 220 GBPoor

Wrote its tool calls as text, so it never inspected the packets.

Llama 4 Maverick (about 400B)USA · Llama licenceFits 320 GB only at 4-bitPoor

Wrote a tool call as text and stopped, and failed the tool-calling qualification twice. Quality at 4-bit is unmeasured.

Mistral Small 2603 (119B)France · Apache 2.0About 120 to 240 GBNot rated

The hosted test route rate-limited both attempts. Retest pending.

Aleph Alpha Kolibri (78B MoE, 3.5B active)Germany · Apache 2.0Well under 320 GBNot rated

Open weights, not offered by our test route. Needs a self-hosted run to measure.

How to read the scale

Great: Finds the full attack chain with correct packet evidence.
Good: Finds the main compromise with correct packet evidence and minor gaps.
Fair: Finds part of the problem or needs heavy correction.
Poor: Does not complete a usable packet investigation.
Not rated: Could not be measured, or not yet tested.

What we tested

Capture: A Windows workstation infected through PowerShell, with command-and-control beaconing and large data uploads.
Setup: PacketSafari security investigation, medium reasoning effort, capture renamed to remove any filename hint.
Filter: Non-Chinese origin and a fit within one 320 GB VRAM deployment.
Limits: Models ran through a hosted route at provider precision. Quantization for on-premises deployment can change quality. Cited frames were reviewed but not every packet was independently checked.

Need a model qualified for your deployment?

We qualify a model on your deployment profile and your own captures before production.

Plan an on-premises deployment