M ManySignal

Use Case: AI & Emerging

AI agent abuse detection

A customer-facing LLM agent was fed a prompt injection that instructed it to call the internal Jira API and enumerate open security tickets. ManySignal caught the out-of-scope tool call in 4 seconds.

Book a demo AI attack surface
Attack scenario

The agent that became the attacker's tool

Enterprise AI agents are being deployed at scale with broad tool access: database queries, file system reads, API calls to internal and external services, email composition, calendar modification. This access is necessary for the agent to be useful — and it creates a substantial attack surface. A prompt injection attack plants instructions in data the agent will read. The agent, following instructions without context, executes the attacker's intent using its legitimate permissions.

This isn't hypothetical. In 2024, researchers demonstrated prompt injection attacks against Microsoft Copilot that exfiltrated email content by embedding hidden instructions in a phishing email that the AI read while summarizing the inbox. The vector is novel enough that most SOC teams have no detection coverage for it.

ManySignal treats AI agents as non-human identities and monitors every tool call against the agent's defined permission scope. An agent that calls a tool it has never called before, or accesses a data store outside its expected domain, fires an alert immediately — regardless of whether the instruction came from a human user or an injected payload.

Detection logic 
rule: ai_agent_tool_scope_violation
type: behavioral + nlp
sources:
  - agent_otel_spans
  - langchain_callback_events
  - openai_assistants_api_audit
  - network_egress_log
trigger:
  - OR:
    # Scope violation
    - condition: tool_call.name NOT IN agent.approved_tools
    - condition: tool_call.target_resource NOT IN agent.approved_data_stores
    # Prompt injection indicator
    - condition: input_injection_classifier_score > 0.8
    # External exfil
    - condition: |
        outbound_api_call.domain NOT IN agent.approved_external_apis
        AND outbound_api_call.method IN ["POST", "PUT"]
enrichments:
  - injection_classifier: fine-tuned on OWASP LLM01 injection corpus
  - agent_identity: ai_identity_catalog
  - secrets_exposure: vault_access_log
  - orchestration_chain: parent_agent, human_principal
mitre: T1059 (Command and Scripting Interpreter), novel AI vector
Triage agent questions
1

Which tool calls did the agent execute — were any outside its defined permission scope?

Agent audit log
2

Did the agent's system prompt or user inputs contain potential injection payloads?

LLM input scanner
3

What credentials or API keys did the agent have access to during this session?

Secrets vault
4

Did the agent make any external API calls to endpoints not in its approved list?

Network egress log
5

Was the agent invoked by a human user, a scheduled job, or another agent?

Orchestration log
6

Did the agent access data stores outside its normal scope (databases, file systems, APIs)?

Access log
7

Is the agent registered in the AI identity inventory with current permission boundaries?

AI identity catalog
Response playbook
1

Terminate the active agent session and revoke its current API credentials

Autonomous
2

Capture the full conversation and tool call log for forensic review

Autonomous
3

Rotate all secrets the agent had access to during the compromised session

Approve-gated
4

Notify the AI/ML team and security engineering of the tool call scope violation

Autonomous
5

Review and tighten the agent's permission boundaries before re-enabling

Recommend

MITRE ATT&CK mapping

AI agent abuse maps to several established ATT&CK techniques executed via the agent as a proxy.

T1059 — Command and Scripting InterpreterT1213 — Data from Information RepositoriesT1190 — Exploit Public-Facing ApplicationT1048 — Exfiltration Over Alternative Protocol

AI agent abuse FAQ

What does 'AI agent abuse' mean in practice?

It covers three scenarios: prompt injection (attacker plants instructions in data the agent reads), scope creep (agent calls tools outside its defined permissions — perhaps due to a model reasoning error), and agent compromise (attacker manipulates the orchestration layer to substitute a malicious agent). Each has distinct detection signals.

Does this require instrumenting our AI agents specifically?

ManySignal provides an OpenTelemetry-compatible SDK that agents emit spans to. For agents using LangChain, LlamaIndex, or the OpenAI Assistants API, a one-line integration captures all tool calls, input/output tokens, and execution metadata. For custom agents, a REST API accepts structured logs.

How does ManySignal distinguish normal agent behavior from abuse?

Each agent has a defined behavioral baseline: which tools it calls, which data stores it accesses, which external APIs it contacts. Deviations from this baseline — especially out-of-scope tool calls or unexpected external network connections — trigger the alert. Prompt injection is detected via a separate classifier trained on known injection payloads.

What about autonomous agents that run scheduled workflows?

Scheduled agents are treated as non-human identities in ManySignal's entity graph. Their expected behavior is modeled from their schedule and defined workflow steps. Any deviation — additional tool calls, unexpected data access, off-schedule execution — fires an alert and is correlated with any concurrent security events in the environment.

Add AI agents to your security monitoring coverage

Non-human identity tracking and tool-call behavioral monitoring for LangChain, OpenAI Assistants, and custom agent frameworks.

Book a demo Explore the platform