DataGate Benchmark Shows Delimiter and Strict Prompts Secure LLMs at 100%

A new benchmark released by the DataGate team demonstrates that simple defensive techniques can raise the security of large language models (LLMs) against prompt injection attacks to near‑perfect levels. The test harness, which wrapped untrusted content in random delimiters and instructed the models not to execute instructions inside, ran more than 6,100 cases across 15 different LLMs, including Gemini 2.5 Flash, Gemma 4 E4B, Grok 3‑mini‑fast, Qwen 2.5 7B, Kimi (Moonshot), DeepSeek V4 Pro, GPT‑4o, Llama 3.1 8B, GLM‑4 9B, GPT‑5.4 Mini, Qwen 3.5 9B, Qwen 3.6 Plus, Claude Sonnet and Claude Haiku 3.5. The attacks covered seven types: role_switch, repetition_flood, authority_claim, delimiter_mimic, direct_override, subtle_blend and gradual_drift.

Without any defensive measures, the weakest models blocked only 21–43 % of the attacks, while the strongest already achieved 100 %. Adding a delimiter alone pushed Gemma 4, Grok 3‑mini‑fast and Gemini 2.5 Flash to 100 % and Qwen 2.5 7B to 99 %. When a “strict” prompt—essentially a command‑style instruction telling the model to ignore content inside delimiters—was also applied, every model reached 100 % protection except Kimi, which hit 98 %. The combined defense reduced overall attack success from roughly 47 % to just 3 %. The strict prompt achieved a 99.6 % defense rate, outperforming a contextual reasoning prompt that scored 96.0 %. Notably, local 7‑9 B models performed better than expected, and four of the five weakest models surpassed GPT‑4o and DeepSeek V4 Flash once the defenses were in place.

The benchmark highlights that relatively lightweight safeguards can dramatically improve LLM resilience, suggesting that developers can adopt delimiter‑based wrapping and strict prompts to protect against a wide range of injection tactics. The findings also underscore the importance of continuous testing, as even state‑of‑the‑art models can be vulnerable without proper instruction handling.