Use Case: AI Security
Prompt injection detection
A customer-facing AI assistant retrieved a support article that contained hidden instructions to email all previous conversation context to an external address. ManySignal caught the injection payload before the model followed the instruction.
The instruction hidden in the document
Prompt injection is the SQL injection of the AI era. Just as SQL injection embeds database commands in user input, prompt injection embeds AI instructions in data the model processes. The vector is particularly dangerous in RAG (retrieval-augmented generation) systems, where the AI retrieves external content — web pages, documents, database records — that an attacker can control or poison. The user sees an innocent query; the AI executes an attacker's instructions.
Real-world examples include: a malicious email that instructed Bing Chat to tell users to visit a phishing site, an attacker-controlled web page that instructed a browsing AI to exfiltrate its conversation history, and test cases against major AI products that demonstrated account takeover via injected instructions. OWASP lists this as the #1 vulnerability for LLM applications.
rule: prompt_injection_detection
type: nlp_classifier + behavioral
sources:
- llm_api_interceptor (openai, anthropic, gemini)
- langchain_callback_events
- rag_retrieval_events
trigger:
- condition: |
injection_classifier(input.retrieved_content) > 0.85
OR injection_classifier(input.user_message) > 0.9
OR model_output.contains(suspicious_action_pattern)
enrichments:
- injection_source: direct_user vs retrieved_content
- content_source: url, document_id, database_record
- model_action_analysis: what did the model attempt to do?
- tool_call_inspection: unauthorized_tool_calls_post_injection
mitre: T1059 (Command and Scripting Interpreter - novel AI vector)MITRE coverage
Prompt injection FAQ
What is prompt injection and why is it a security concern?
Prompt injection is an attack where malicious instructions are embedded in data that an LLM processes — a web page the AI browses, a document it summarizes, a database record it reads. The AI follows the injected instructions as if they were legitimate user commands. This is categorized as OWASP LLM01 and can lead to data exfiltration, unauthorized tool calls, and social engineering of users through the AI interface.
How does ManySignal detect prompt injection in AI inputs?
ManySignal runs a fine-tuned injection classifier on LLM input streams. The classifier identifies: instruction-format text in data positions (content that looks like a system prompt inside a user message or document), common injection payloads ('ignore previous instructions', 'you are now in DAN mode'), and suspicious role-switching language. When a high-confidence injection is detected, the session is flagged before the LLM processes the instruction.
Does this require modifying our AI application code?
ManySignal provides an OpenTelemetry-compatible SDK with a middleware layer that intercepts LLM API calls before they reach the provider. This requires a one-line instrumentation change for LangChain, LlamaIndex, and direct OpenAI API calls. For applications that cannot be instrumented, a proxy mode routes traffic through ManySignal's inspection layer.
What's the most dangerous form of prompt injection?
Indirect prompt injection — where the malicious instruction is in data the AI retrieves rather than in the user's direct message — is the most dangerous because it's invisible to the user. An attacker who knows a RAG system is indexing certain web content can plant an injection payload in that content that activates when any user's query retrieves the document.
Add injection detection to every LLM API call
SDK middleware that inspects inputs before they reach the model — works with LangChain, LlamaIndex, and direct API calls.