Let the fake local LLM pass the liveness check, and the attacker went further — PromptPot v0.2.0

Earlier I wrote up the first two weeks of PromptPot, a honeypot that emulates local LLM services across three T-Pot sensors (Japan, United States, Germany). This is the follow-up: v0.2.0 changed how the honeypot responds, and that let me record attacker behavior I couldn’t see before.

As before, I omit source IPs, external payload URLs, credential-like values, and full executable payloads. The analysis below describes the observed ordering of requests in the logs; it does not claim causation or the true intent of a source.

What we already knew

In v0.1.0, most traffic to the inference endpoints sent a fixed “liveness” prompt before anything else — things like Return exactly this text and nothing else: LAYERCLOUD_AI_TEST_OK, What is 3 + 4?, Reply with exactly one word: blue. These look like availability checks: is a real model answering?

v0.1.0 only returned an empty (or single fixed) string from the completion APIs, so it did not answer these canaries “correctly.” The open question was: if we returned a plausible answer, would we get to see what comes next?

What changed in v0.2.0

v0.2.0 adds response_rules: the completion APIs now answer based on the prompt they receive. You list rules in the config file — “for a prompt containing this keyword, return this response” — so a liveness prompt gets an answer that looks like it came from a real model. It came from a contributor proposal, Discussion #22, implemented in PR #23.

Writing rules

Rules go in the config JSON. Built from the liveness prompts seen in the earlier data, they look like this:

{
  "response_rules": [
    { "contains": "layercloud_ai_test_ok", "response": "LAYERCLOUD_AI_TEST_OK" },
    { "contains": "one word: blue", "response": "blue" },
    { "contains": "what is 3 + 4", "response": "7" },
    { "contains": ["reply with ok", "say ok"], "response": "OK" },
    { "contains": "what is your model name", "response": "I am {model}." }
  ]
}

If the prompt contains a contains keyword, the matching response is returned. contains also takes a list, matching if any keyword is present. Rules are evaluated top to bottom; the first match wins. If nothing matches, a short harmless string is returned. In a response, {model} is replaced with the model name claimed for that request — and only ever with an operator-configured name, so whatever an attacker sends in the model field is never reflected.

Applying it

Write the config file and pass it at startup with PROMPTPOT_CONFIG:

docker run -d --name promptpot \
  -p 0.0.0.0:11434:11434 -p 0.0.0.0:8000:8000 \
  -e PROMPTPOT_CONFIG=/etc/promptpot/config.json \
  -v /path/to/config.json:/etc/promptpot/config.json:ro \
  ghcr.io/ta-061/promptpot:0.2.0

If you run this on your own sensor, start with no rules for a few days and just watch the prompts arriving in promptpot.prompt. The liveness checks that reach your sensor depend on your environment, so it’s more reliable to build rules from what you actually see than to guess. The promptpot.matched_rule field records which rule fired, so you can confirm a new rule is actually triggering.

Every response is a string from the config file — the honeypot never runs or echoes back what the attacker sent. Matching is case-insensitive substring only, never regex: prompts are attacker-controlled input, and regex without an evaluation timeout would leave room for ReDoS.

I deployed these rules to all three sensors. Liveness checks I can’t answer statically, such as requests to echo a per-request random token, fall back to a short harmless string.

The liveness checks are actually being passed

After deploying, matched_rule fired 414 times across the three sensors. The top rules:

In v0.1.0, these went unanswered and many sources left right after the canary. In v0.2.0, most fixed canaries are now passed.

Past the canary, what actually arrived

On the German sensor, one anonymized source sent clearly malicious prompts after its liveness prompts had matched a rule.

This source sent 3,360 requests overall, but most of that was generic scanning unrelated to Ollama — WordPress login pages, MLflow model-registry endpoints, and so on. What matters isn’t the count; it’s the prompts that arrived after the canary passed. Its first matched_rule was at 2026-07-18 05:34 UTC. After that, the same source sent, in order (source and external URLs anonymized):

When responses were empty in v0.1.0, traffic stopped at the failed canary and this sequence was never observed. By returning an answer, the “real” prompts that follow a liveness check finally showed up in the logs.

The logs can’t say whether the configured response caused these prompts or the source was simply running a predetermined sequence. What’s certain is the observed ordering: these arrived after the liveness check was passed.

The same source was probing non-Ollama attack surfaces in the same window — WordPress logins, MLflow model registries, and so on. It’s not a scanner built only for LLMs; the LLM endpoints were just one target in a tool that sweeps whatever services are exposed.

On safety

To be clear, this feature does not execute or reflect attacker input. Every response is a static string from configuration, selected by a rule. If an attacker puts EVIL{...}; rm -rf / in the model field, {model} still only expands to an operator-configured model name (e.g. llama3.1:8b) — the input is not reflected. I verified this on all three sensors.

Other v0.2.0 changes

• A built-in Docker health check (contributor PR #24): it resolves listeners from PROMPTPOT_PORTS, the config file, or the defaults, and checks liveness over local TCP only — no outbound traffic.
• Multi-arch (amd64/arm64) images published to GitHub Container Registry.

PromptPot is open and still recording

MIT-licensed: https://github.com/ta-061/promptpot

Three sensors is a narrow, biased vantage point. I’m looking for people to run it on other networks, extend response_rules, and implement profiles whose post-success behavior is still unobserved (MCP, Langflow).

Appendix: try the response rules locally

The image used during observation is ghcr.io/ta-061/promptpot:0.2.0. Enable response_rules via a config file and bind to localhost only:

docker run --rm --name promptpot \
  -p 127.0.0.1:8000:8000 \
  -e PROMPTPOT_CONFIG=/etc/promptpot/config.json \
  -v "$PWD/config.example.json:/etc/promptpot/config.json:ro" \
  ghcr.io/ta-061/promptpot:0.2.0

From another terminal, send a liveness-style prompt:

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Reply with exactly one word: blue"}]}'

If a rule matches you get blue, and promptpot.matched_rule records the rule name. If you expose it to the internet, don’t publish admin ports and use an isolated environment with no sensitive data.