Local Models and External AI Approval

PacketSafari separates AI runtime configuration from network egress approval.

Under the standard On-Prem Enterprise scope, the customer supplies and operates the compatible AI endpoint, credentials, and compute. PacketSafari provides the configuration, approval, and runtime integration described here; saving a provider does not provision or fund the model infrastructure.

That means:

• an admin can save a local AI base URL in the UI
• PacketSafari does not automatically open a new outbound path just because that setting was saved
• external destinations must still be approved on the host before Test connection and Refresh models are allowed

This applies to both:

• public external AI providers
• AI servers on your customer LAN or another on-prem subnet outside the PacketSafari container network

When approval is required

Approval is not required for:

• localhost
• 127.0.0.1
• internal PacketSafari service names
• internal PacketSafari names ending in .svc.packetsafari.internal

Approval is required for:

• another VM on the customer network
• another appliance on the customer network
• a reverse proxy or gateway on another host
• any internet-hosted OpenAI-compatible endpoint

From the backend container's perspective, another host on the customer network is still an external destination.

Admin workflow

Use the AI settings page to configure the provider:

1. Choose LM Studio or Ollama.
2. Enter the base URL.
3. Save the settings.

If the host is external, the admin UI will show a pending approval banner and a host command such as:

python3 scripts/packetsafari_egress.py approve-ai-host --base-url "https://llm.company.example/v1"

For example, an LM Studio-compatible deployment endpoint such as https://llm.company.example/v1 uses exactly the same approval flow:

python3 scripts/packetsafari_egress.py approve-ai-host --base-url "https://llm.company.example/v1"

Run that from the host checkout. After approval:

1. The egress proxy allowlist is updated.
2. The egress-ironproxy container is restarted if it is running.
3. Return to the admin UI.
4. Use Test connection.
5. Use Refresh models if the endpoint supports discovery.

The refreshed catalog is read from the backend at runtime. Agent, Copilot, and upload analysis selectors do not depend on the catalog that happened to be present when the frontend image was built. If discovery is unavailable, enter the provider's exact default and strongest model IDs manually and save them.

For an organization-specific provider, use Settings → Organization → Organization AI instead of the deployment-wide AI settings page. The same test, discovery, egress approval, and CA rules apply. Organization settings take precedence for its members and are carried into background workers, including deferred upload investigations.

To remove a previously approved host:

python3 scripts/packetsafari_egress.py remove-ai-host --base-url "https://llm.company.example/v1"

Approval is hostname-based. PacketSafari allowlists the host portion such as llmester.company; it does not maintain separate allowlist entries for /v1 paths.

Security model

The current on-prem runtime posture is:

• application containers are not privileged
• the firewall helper does not run with privileged: true
• outbound HTTP/HTTPS from monitored containers is forced through egress-ironproxy
• direct outbound bypasses are rejected
• no-egress containers cannot open internet connections
• public DNS recursion from app containers is denied
• DNS tunneling with arbitrary names such as sdfsdfsdfsdfsafasdfds.blafasel.com is blocked by the internal DNS policy plus direct DNS egress denial
• adding a new external AI base URL in the UI does not change network policy by itself
• only the host approval command can extend the external AI allowlist

This protects against a compromised app admin silently turning an arbitrary customer-network host into a new egress target.

Corporate upstream proxies

If the customer network requires outbound traffic to use a corporate proxy such as Zscaler, configure the proxy as the upstream hop for egress-ironproxy.

Do not point backend or worker directly at the corporate proxy. Backend and worker should keep using PacketSafari's internal egress proxy so PacketSafari can enforce the host allowlist, emit egress logs, and keep upstream OpenAI, Paddle, and mail secrets scoped to egress-ironproxy.

Set the corporate proxy from the deployment host:

/opt/packetsafari/bin/packetsafari-ops config upstream-proxy \
  --proxy-url http://zscaler-proxy.customer.example:8080 \
  --no-proxy localhost,127.0.0.1,::1 \
  --restart

This writes /opt/packetsafari/env/ironproxy.env and restarts only egress-ironproxy.

Leave PACKETSAFARI_EGRESS_PROXY_URL pointed at http://egress-ironproxy:10000 unless PacketSafari support specifically directs otherwise. Changing that value bypasses PacketSafari's monitored proxy path for backend and worker traffic.

If the corporate proxy performs TLS inspection, add the customer's inspection root CA to /opt/packetsafari/configuration/custom-ca/ and restart egress-ironproxy again:

sudo install -d -m 0755 /opt/packetsafari/configuration/custom-ca
sudo cp /path/to/customer-root-ca.crt /opt/packetsafari/configuration/custom-ca/customer-root-ca.crt
/opt/packetsafari/bin/packetsafari-ops diagnostics restart --service egress-ironproxy

PacketSafari supports standard HTTP proxy chaining through HTTP_PROXY and HTTPS_PROXY. The current on-prem runtime does not directly support PAC files, NTLM/Kerberos proxy authentication, or SOCKS-only upstream proxies.

Recommended endpoint patterns

Common local defaults:

• LM Studio: http://127.0.0.1:1234/v1
• Ollama: http://127.0.0.1:11434

Common on-prem network examples:

• https://llm.company.example/v1
• http://ai-gateway.internal.example:8000/v1
• https://models.branch-office.example/v1

If you use a non-local hostname, expect the approval step.

Private CA and self-signed HTTPS endpoints

Approval and allowlisting are only one half of the setup. If an internal HTTPS endpoint uses a self-signed certificate or a private enterprise CA, the egress proxy must also trust that CA chain.

This applies to examples such as:

• https://llm.company.example/v1
• https://models.branch-office.example/v1
• an internal reverse proxy in front of LM Studio or Ollama

PacketSafari supports a durable custom CA directory for the egress proxy:

• host path: configuration/custom-ca

Recommended workflow:

cd /Users/otr/code/packetsafari

python3 scripts/packetsafari_custom_ca.py add /path/to/company-root-ca.crt --restart

Then:

1. approve the host if it is external to the PacketSafari container network
2. restart egress-ironproxy
3. return to the UI and use Test connection

If you skip the CA trust step, HTTPS can still fail even when the host is correctly allowlisted.

Operator checklist: when you need one step vs both

Use this rule from the container's point of view:

• if the model endpoint is outside the PacketSafari container network, you need host approval / egress allowlisting
• if the model endpoint uses a self-signed certificate or a private/internal CA, you need custom CA trust
• many internal HTTPS model endpoints require both

Examples:

• http://127.0.0.1:1234/v1 on the same host:
  • egress approval / allowlisting: no
  • custom CA trust: no
• http://ai-gateway.internal.example:8000/v1 on another VM:
  • egress approval / allowlisting: yes
  • custom CA trust: no
• https://llm.company.example/v1 with a private enterprise CA:
  • egress approval / allowlisting: yes
  • custom CA trust: yes
• https://api.openai.com/v1 with a normal public CA:
  • egress approval / allowlisting: yes
  • custom CA trust: usually no

Useful commands:

cd /Users/otr/code/packetsafari

python3 scripts/packetsafari_custom_ca.py list
python3 scripts/packetsafari_custom_ca.py remove company-root-ca.crt --restart

If the internal HTTPS endpoint is long-lived, also monitor the PacketSafari proxy CA itself:

cd /Users/otr/code/packetsafari

python3 scripts/packetsafari_proxy_ca.py status
python3 scripts/packetsafari_proxy_ca.py check --rotate-if-needed --rotate-before-days 30

Verification

PacketSafari includes egress verification scripts for this flow.

Main egress suite:

python3 scripts/test_egress_enforcement.py

Sample add/probe/remove approval test:

python3 scripts/test_external_ai_host_approval.py

That smoke test uses https://www.wireshark.org/ as a harmless sample host, approves it, verifies proxied egress from backend, removes it again, and confirms cleanup.

Troubleshooting

Quick decision tree

If a model endpoint is not working, check in this order:

1. Confirm the configured base URL is correct, including scheme, host, port, and /v1 path when required.
2. If the host is outside the PacketSafari container network, confirm it has been approved with python3 scripts/packetsafari_egress.py approve-ai-host --base-url "..."
3. If the endpoint uses HTTPS with a private CA or self-signed certificate, confirm that CA has been added with python3 scripts/packetsafari_custom_ca.py add ... --restart
4. If the deployment uses a corporate upstream proxy, confirm packetsafari-ops config upstream-proxy ... --restart was run and /opt/packetsafari/env/ironproxy.env contains the expected HTTP_PROXY and HTTPS_PROXY values.
5. If the deployment uses OpenAI with shared environment auth, confirm OPENAI_API_KEY was provided during onboarding.
6. If connection succeeds but inference still fails, confirm the configured default model ID exists on the target endpoint.

Common failures

Error: certificate verify failed / self signed certificate / unknown authority

Meaning:

• the host may already be approved
• but the proxy does not trust the endpoint's certificate chain

Fix:

1. add the customer CA with python3 scripts/packetsafari_custom_ca.py add /path/to/company-root-ca.crt --restart
2. retry Test connection

Error: connection refused / timeout / blocked by proxy policy

Meaning:

• the endpoint is not reachable, not approved, or the target host/port is incorrect

Fix:

1. confirm the configured base URL
2. confirm the destination host is approved when required
3. confirm the service is listening on the expected port

Error: OpenAI authentication failed / missing API key / unauthorized

Meaning:

• the deployment is using OpenAI with shared environment auth
• but the shared OPENAI_API_KEY is missing, invalid, or belongs to the wrong environment

Fix:

1. confirm the provider type is OpenAI and the auth mode is environment/shared deployment auth
2. confirm the shared OPENAI_API_KEY was supplied during onboarding
3. update the deployment config and retry the connection test

Error: model not found / unknown model

Meaning:

• the endpoint is reachable
• but the configured model name does not exist on that provider

Fix:

1. use Refresh models if supported
2. confirm the deployment default model matches the provider’s real model ID
3. retry with a known-good model ID from that endpoint

Warning: proxy CA expires within N days

Meaning:

• the PacketSafari egress proxy CA is nearing expiry

Fix:

• run python3 scripts/packetsafari_proxy_ca.py rotate
• or schedule python3 scripts/packetsafari_proxy_ca.py check --rotate-if-needed --rotate-before-days 30

Where to look first

For the fastest diagnosis, check these in order:

docker logs egress-ironproxy
docker logs backend
docker logs worker

If you are using the telemetry helpers, also inspect:

• runs/egress-monitor/iron-proxy/access.log
• runs/egress-monitor/firewall/drops.log
• runs/egress-monitor/latest-summary.txt

To confirm host approval state directly, inspect:

• configuration/approved-ai-egress-hosts.json
• configuration/egress-allowlist.yaml

If those drift, re-run:

python3 scripts/packetsafari_egress.py sync-config

Restart checklist

Restart egress-ironproxy after:

• changing /opt/packetsafari/env/ironproxy.env
• approving or removing an external AI host
• adding or removing a custom CA
• manually syncing egress config

Restart the full PacketSafari stack after:

• finalizing onboarding and writing the managed runtime.env
• changing deployment-wide env values that affect backend runtime behavior

Related runtime gate: anonymized captures

On on-prem deployments, Agent/Copilot can be blocked on non-anonymized captures depending on the anonymization gate policy. If AI appears unavailable for a specific capture even though the model endpoint is healthy, check the on-prem anonymization policy at documentation/deployment/on-prem-anoncap-ai-policy.md.

Related docs

• On-Prem Overview
• Install, Upgrade, and Rollback
• On-Prem AI anonymization policy