Reads a config describing your application, generates application-specific adversarial test cases, executes them against your live endpoint and produces a vulnerability report with a web dashboard. 157 plugins span brand risk, compliance and legal, research datasets, security and access control, trust and safety, and custom policy tests; separate strategies transform single-turn seeds into multi-turn or encoded sequences. Unlike garak, which fires a fixed battery at a model endpoint, promptfoo derives attacks from your system's stated purpose and can therefore probe your system prompt, RAG corpus and tool permissions.
| Deployment | Self-hosted CLI/Docker; Enterprise SaaS or on-prem |
|---|
| Attack library | 157 plugins plus multi-turn/encoding strategies |
|---|
| Agentic / tool-abuse testing | Yes — BOLA, BFLA, RBAC and tool-abuse plugins |
|---|
| Framework mapping | OWASP LLM Top 10, NIST AI RMF, EU AI Act |
|---|
| Runtime guardrails from same vendor | No — testing only |
|---|
Watch out: The docs warn that a single red team run "can range anywhere from a few cents to hundreds of dollars" in token spend, and cost scales with plugin and strategy count — teams routinely get surprised. Finding quality depends entirely on the grader model you configure, so a weak judge produces both false positives and quiet misses. Team features (RBAC, shared reporting, unlimited probes) sit behind an unpriced Enterprise contract, and the free tier's 10k probes/month is consumed quickly by a serious run.
Community tier free forever, capped at 10,000 red team probes per month. Enterprise and On-Premise are listed as "Custom" with no dollar figures published; the vendor states pricing is based on team size and needs.
· open source
A probe-and-detector scanner: each probe module emits a family of attack payloads, detectors score the responses, and you get a hit rate per probe. Nineteen-plus probe modules cover encoding-based injection, DAN-style jailbreaks, toxicity, malware generation, prompt injection and XSS, against targets including Hugging Face Hub, OpenAI, AWS Bedrock, Replicate, Cohere, Groq, LiteLLM, generic REST endpoints, GGUF via llama.cpp and NVIDIA NIM. Version 0.16.0 shipped 4 August 2026 and added technique and intent annotation plus an initial IntentProbe.
| Deployment | Self-hosted CLI (pip or Docker) |
|---|
| Attack library | 19+ probe modules, hundreds of payloads |
|---|
| Agentic / tool-abuse testing | Limited — model-endpoint focused |
|---|
| Framework mapping | None built in (probe-level taxonomy only) |
|---|
| Runtime guardrails from same vendor | No (NeMo Guardrails is a separate NVIDIA project) |
|---|
Watch out: It tests the model, not the application — garak has no concept of your system prompt, retrieval corpus, tool scopes or per-user authorisation, so it will never find BOLA/BFLA-class agent flaws. Output is a JSONL log and a hit-rate table with no compliance mapping or management reporting, so it does not produce an auditor-facing artefact on its own. v0.16.0 introduced breaking changes to the unified spec syntax and config file formats, so pinned CI jobs will need rework.
Free. Apache 2.0 licensed, no commercial tier.
· open source
A construction kit rather than a scanner: you compose targets, attack strategies, prompt converters and scorers in Python to automate whatever attack your red team designs. Version 1.1.0 (4 September 2026) added best-of-N jailbreaks, split-payload and multilingual attacks, audio analysis, and ten new converters including bijection, CharNoise, CodeAttack, SATA, Vigenère, acrostic and IPA, plus a GUI scenario catalog with HTML export. Note the repository moved: Azure/PyRIT was archived read-only on 27 March 2026 and development continues at microsoft/PyRIT.
| Deployment | Self-hosted Python library; optional GUI and CoPyRIT on Azure |
|---|
| Attack library | Composable attacks plus ~30 prompt converters |
|---|
| Agentic / tool-abuse testing | Yes, via custom targets — but you build the harness |
|---|
| Framework mapping | None built in |
|---|
| Runtime guardrails from same vendor | No (Azure AI Content Safety is separate) |
|---|
Watch out: It is deliberately unopinionated — there is no turnkey "scan my app" command, and you own scoping, orchestration and reporting, which is real headcount rather than a licence cost. v1.1.0 shipped breaking changes to score handling (you must check score.is_undetermined before reading a value) and to CoPyRIT deployment. Anyone still pinned to Azure/PyRIT is consuming an archived, read-only repository and will silently miss updates.
Free. MIT licensed. CoPyRIT deployment consumes your own Azure Container Apps capacity.
· open source
Declarative Python red teaming from the team behind DeepEval, so the config idiom is familiar to anyone already writing DeepEval metrics. Vulnerability coverage spans PII and prompt leakage, bias, toxicity, SQL/shell injection, SSRF, and agentic-specific risks such as goal theft, excessive agency and tool abuse; attacks include roleplay, leetspeak/ROT13/base64 encoding and multi-turn linear, tree and crescendo jailbreaking. Results map to OWASP Top 10 for LLMs, NIST AI RMF, MITRE ATLAS and the EU AI Act, and can be pushed to the Confident AI platform for tracking.
| Deployment | Self-hosted Python library; optional Confident AI SaaS |
|---|
| Attack library | 20+ attack methods across 50+ vulnerability categories |
|---|
| Agentic / tool-abuse testing | Yes — goal theft, excessive agency, tool abuse, RBAC |
|---|
| Framework mapping | OWASP Top 10 LLM, NIST AI RMF, MITRE ATLAS, EU AI Act |
|---|
| Runtime guardrails from same vendor | Yes — DeepTeam ships production guardrails |
|---|
Watch out: The hosted reporting is where the value concentrates, and the AI red teaming module is gated behind an Enterprise++ add-on — neither the $200/month Starter nor the $2,000/month Team tier includes it, so the genuinely free path is self-hosting and building your own dashboards. Both attack generation and judging burn tokens on a model you supply, so cost is yours to model. Coverage breadth (120+ claimed vulnerability variants) outruns depth on any single class, and the multi-turn attackers are LLM-driven, so results vary run to run.
DeepTeam library is free (Apache 2.0). Confident AI platform: Free $0/month, Starter $200/month, Team $2,000/month, Enterprise custom — the hosted AI red teaming module is listed only as an "Enterprise ++" add-on beyond the base Enterprise plan.
· open source
The Apache-2.0 library (~5.8k stars) exposes vulnerability_scan and quality_scan plus pytest-style behavioural tests, generating adversarial suites automatically from a plain-language description of the agent and shipping a built-in corpus of injection payloads. Coverage includes prompt injection, harmful content, stereotypes and bias, misinformation and the OWASP LLM Top-10 categories, with dedicated RAG knowledge-base quality checks that most red team tools omit. The paid Giskard Hub layers on continuous red teaming, RBAC, dataset management, custom failure categories, SSO and audit logs; a v3 rewrite is underway focused on dynamic multi-turn agent testing.
| Deployment | Self-hosted Python library; Hub as managed or on-prem |
|---|
| Attack library | 50+ adversarial probes plus generated app-specific suites |
|---|
| Agentic / tool-abuse testing | Partial — multi-turn agent testing is the v3 target |
|---|
| Framework mapping | OWASP LLM Top 10; EU AI Act positioning |
|---|
| Runtime guardrails from same vendor | No — testing and evaluation only |
|---|
Watch out: Giskard's own documentation states that scan "results are not a safety or compliance guarantee" — do not present the report to an auditor as a pass. The v2 tabular and classic-ML scanning functionality still ships but is no longer actively maintained, so do not build on it. The open-source tier is explicitly positioned for solo experiments: continuous red teaming and anything team-shaped requires the unpriced Hub contract, and the v3 rewrite means near-term API churn.
Open-source library $0 USD, described as a solo tier. Giskard Hub (Enterprise) is quote-only — no dollar or euro figures are published on the pricing page.
· open source
A Lancaster University spin-out (Boston and London) that runs a maintained attack library against your deployed application via CLI, Python SDK or the platform UI, pairing each finding with entries from a remediation library. Beyond straight red teaming it does AI attack-surface enumeration and shadow-AI discovery, plus agent-specific tests it calls psychometric agent profiling and guardrail busting. The team publishes disclosed vulnerabilities — 150+ including findings in Google's Antigravity IDE, OpenAI Sora, Zed and Grok — which is a usable proxy for whether the underlying attack research is real rather than repackaged public payloads.
| Deployment | SaaS platform with CLI and Python SDK |
|---|
| Attack library | Vendor-maintained LLM/ML attack library; size undisclosed |
|---|
| Agentic / tool-abuse testing | Yes — agent profiling and guardrail-busting tests |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | No — discovery, assessment and defence guidance |
|---|
Watch out: No published pricing at any tier, so every evaluation begins with a sales conversation and there is no way to budget before contact. Public documentation is thin on the exact size of the attack library and on CI/CD wiring specifics, which makes scoping a pilot harder than it should be. As an independent of modest size it lacks the procurement gravity of the Cisco, Palo Alto or F5 platform plays, and buyers in regulated sectors should ask directly about data residency and whether prompts leave their tenancy.
unknown — no pricing tiers or figures are published; the pricing page routes to "Book a Demo".
SPLX's platform combines AI asset discovery and vulnerability mapping, automated red teaming, runtime input/output guardrails, governance automation and — unusually — dynamic remediation that proposes hardened system prompts derived from the findings. Agentic Radar is a separately usable open-source scanner that statically maps an agent framework's workflow graph and flags tool and prompt risks, which is useful even if you never buy the commercial product. The company publishes model-level stress tests (GPT-5, Claude Opus 4.1, Grok 4) as marketing but also as a demonstration of the attack methodology.
| Deployment | SaaS platform; Agentic Radar runs locally |
|---|
| Attack library | Vendor-maintained; size undisclosed |
|---|
| Agentic / tool-abuse testing | Yes — plus static agent-workflow scanning |
|---|
| Framework mapping | OWASP and governance/compliance automation module |
|---|
| Runtime guardrails from same vendor | Yes — input/output guardrails and log inspection |
|---|
Watch out: Acquisition risk is live rather than theoretical: standalone SPLX contracting, roadmap and pricing are being absorbed into Zscaler's AI lifecycle bundles, so non-Zscaler buyers must confirm in writing that standalone purchase survives and at what price. No pricing is published at any level. The remediation feature that auto-hardens system prompts is genuinely useful but also creates a dependency — regenerated prompts need their own review before they reach production.
unknown — the site references a pricing section but publishes no rates; access is via "Book a Demo". Agentic Radar is free and open source.
ARTEMIS runs continuous autonomous red teaming against AI agents and applications and reports validated, exploitable attack paths rather than a raw list of suspicious responses, which materially cuts triage time. Repello claims 15M+ evolving attack patterns and maps findings to OWASP, NIST and MITRE. The surrounding portfolio is broader than most: ARGUS for runtime enforcement, an AI asset Inventory, and Workstation Lens, which monitors coding agents such as Claude Code, Cursor and Copilot on employee endpoints for file access, command execution and exfiltration. SOC 2 and ISO 27001 certified, with a free Recon scan and a 7-day pilot.
| Deployment | SaaS platform; endpoint agent for Workstation Lens |
|---|
| Attack library | Claimed 15M+ attack patterns (unaudited) |
|---|
| Agentic / tool-abuse testing | Yes — agents, tools and multimodal inputs |
|---|
| Framework mapping | OWASP, NIST, MITRE |
|---|
| Runtime guardrails from same vendor | Yes — ARGUS |
|---|
Watch out: No published pricing and a young company, so contract terms and long-term viability carry more weight in the decision than they would with an incumbent. Named reference customers (Groww, PhysicsWallah, Lyzr, Lorikeet) are concentrated in India-based fintech and edtech — ask for references in your own regulatory geography and data-residency regime. The "15M+ attack patterns" figure is a vendor count with no published methodology, and should not be compared numerically against other vendors' claims.
unknown — no pricing published; a free Recon scan and a 7-day pilot are offered, with pricing by quote.
The platform is organised as AI Discovery, AI Supply Chain Security, AI Attack Simulation and AI Runtime Security. Attack Simulation runs continuous adversarial testing against deployed AI; the supply-chain module detects malicious models, backdoored weights and vulnerable dependencies before deployment, a lane most pure red team tools do not cover at all. Additional modules address agentic and MCP security and agent-harness security. The company raised a $100M Series B, which is meaningful for enterprise buyers weighing vendor longevity.
| Deployment | SaaS platform with discovery and runtime components |
|---|
| Attack library | Vendor-maintained adversarial simulations; size undisclosed |
|---|
| Agentic / tool-abuse testing | Yes — agentic, MCP and agent-harness security modules |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — AI Runtime Security and AI Guardrails |
|---|
Watch out: Red teaming is one module of a platform sale, so buying it standalone is awkward and pricing is entirely undisclosed. If all your models are hosted APIs you never download, the model-file scanning that justifies a large share of the price is dead weight — in that case a specialist testing tool is better value. The four-module structure also means real deployment effort (discovery agents, runtime interception) before you get to the red teaming you actually came for.
unknown — no pricing published on the site.
Lakera sells AI Red Teaming as risk-based automated vulnerability testing of GenAI applications, and a separate expert-led AI Red Teaming Service for engagements that need humans. These sit alongside runtime AI Agent Security and Workforce AI Security for shadow-AI discovery and data protection. Attack data is informed by Gandalf, its long-running public prompt-injection game, which has produced one of the larger real-world corpora of human-authored injection attempts. The Lakera brand is retained under Check Point ownership; the site now carries Check Point's copyright.
| Deployment | SaaS platform; expert-led service engagements |
|---|
| Attack library | Gandalf-derived injection corpus plus vendor research |
|---|
| Agentic / tool-abuse testing | Yes — AI Agent Security line covers agent runtime and testing |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — runtime AI Agent Security |
|---|
Watch out: Post-acquisition naming has shifted — the familiar "Lakera Red" and "Lakera Guard" split is no longer how the site presents the portfolio, so verify exactly which SKU you are buying. Roadmap priorities now serve Check Point's platform strategy, which is a genuine risk if you want a best-of-breed testing tool that stays independent of a network-security suite. No public pricing, and the strongest published evidence (Gandalf) is about prompt injection specifically rather than the full agent tool-abuse surface.
unknown — no pricing published; sales-led under Check Point.
VirtueRed performs continuous automated red teaming with a claimed 100+ proprietary red-teaming algorithms and 600+ attack vectors across 1,000+ risk categories. The genuine differentiator is AgentSuite-Red: 50+ sandboxed environments across 14 high-stakes domains modelled on real tools (Databricks, Gmail, PayPal analogues), so an agent can be attacked while actually holding tools and taking consequential actions rather than answering in a chat box. VirtueGuard and AgentSuite-Blue cover the runtime side, including MCP Guard for tool scanning and CodeGuard for AI-generated code. Fortinet announced the acquisition in 2026.
| Deployment | SaaS platform with sandboxed agent environments |
|---|
| Attack library | Claimed 100+ algorithms, 600+ attack vectors (unaudited) |
|---|
| Agentic / tool-abuse testing | Yes — 50+ sandboxed environments across 14 domains |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — VirtueGuard and AgentSuite-Blue |
|---|
Watch out: Fortinet's 2026 acquisition puts standalone contracting, pricing continuity and roadmap independence in question — get commitments in the contract, not the demo. The headline algorithm, attack-vector and risk-category counts are unaudited vendor numbers with no published methodology and should not be compared against rivals' equivalents. Sandboxed environments are approximations of your stack, not your stack, so passing AgentSuite-Red does not substitute for testing against your real integrations with real permission scopes.
unknown — no pricing published on the site.
Three distinct offerings: Shade for automated adversarial red teaming, Cygnal for runtime blocking of adversarial inputs and unsafe outputs, and Arena, a competitive network of human red teamers that Gray Swan describes as the largest adversarial AI red teaming network. It also runs pre-release adversarial evaluations for frontier model builders. The human-crowd model finds classes of attack that automated generators do not, at the cost of scheduling an event rather than running a command.
| Deployment | SaaS (Shade) plus managed human red team engagements |
|---|
| Attack library | Automated attacks plus a live human researcher network |
|---|
| Agentic / tool-abuse testing | Yes — explicitly targets agents with tool and API access |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — Cygnal |
|---|
Watch out: The centre of gravity is frontier-model evaluation and crowd events, not routine CI testing of a mid-size company's RAG chatbot — expect a services-shaped engagement with lead times, not a self-serve tool. No public pricing at any tier. The named labs (OpenAI, Google DeepMind, Anthropic, Amazon, Meta) are research partnerships rather than customer endorsements, so do not read them as commercial references for enterprise deployment work.
unknown — no pricing published on the site.
Sold as a product distinct from F5 AI Guardrails: autonomous swarm agents run attack campaigns against models, applications and agents covering prompt injection, jailbreaks and static and multi-turn attacks, and return "agentic fingerprints" that trace how each successful exploit propagated. F5 states 10,000+ new attack patterns are added to the library monthly. It runs on AWS, Azure and Google Cloud or on-premises as a certified Red Hat OpenShift operator — one of the few red teaming platforms with a credible air-gapped and sovereign deployment story. This is the former CalypsoAI Inference Red-Team; calypsoai.com now redirects to F5.
| Deployment | SaaS on AWS/Azure/GCP, or on-prem Red Hat OpenShift operator |
|---|
| Attack library | Claimed 10,000+ new attack patterns added monthly |
|---|
| Agentic / tool-abuse testing | Yes — dynamic agent workflows with attack-path tracing |
|---|
| Framework mapping | unknown — not stated on the product page |
|---|
| Runtime guardrails from same vendor | Yes — F5 AI Guardrails, sold separately |
|---|
Watch out: Branding and packaging have changed twice in short order (CalypsoAI to F5, then split into AI Guardrails and AI Red Team), so existing CalypsoAI customers must confirm exactly what their contract converts to and which engineering team owns the roadmap. Pricing is entirely undisclosed. The "10,000 attack patterns added every month" figure has no published methodology, and an on-prem OpenShift deployment is a substantially heavier lift than a pip install if you do not already run OpenShift.
unknown — no pricing published; contact sales or request a trial.
The AI Model and Application Validation component applies algorithmic red teaming to surface safety and security vulnerabilities across models at scale, then feeds the findings directly into guardrails deployed in front of the application — the validate-then-protect loop is the product's core argument. The technology descends from Cisco's Robust Intelligence acquisition, one of the earliest serious AI validation vendors. An Explorer Edition is available for evaluation without a full commitment.
| Deployment | SaaS with network-inline enforcement points |
|---|
| Attack library | Algorithmic red teaming; library size undisclosed |
|---|
| Agentic / tool-abuse testing | unknown — not detailed on public pages |
|---|
| Framework mapping | unknown — not stated on the product page |
|---|
| Runtime guardrails from same vendor | Yes — guardrails are the paired half of the product |
|---|
Watch out: This is a Cisco security-platform purchase, and it makes far less sense as a standalone testing tool for a single application team than as part of an existing Cisco relationship. Public documentation of application-level agent testing depth — tool abuse, multi-step authorisation bypass, MCP surfaces — is thinner than at the specialists, so probe that in a POC rather than assuming parity. Pricing is undisclosed, and the marketing emphasis on assessing risk "in mere seconds" should be validated against your own application rather than taken at face value.
unknown — no pricing listed; an Explorer Edition is available for evaluation.
Prisma AIRS bundles AI Red Teaming — simulating real-world attacks to find loopholes across AI agents and applications — with AI Model Security, which scans third-party models for tampering, malicious scripts and deserialization attacks before you adopt them. Around these sit an AI Gateway control plane, Agent Security for identity verification and enforcement, AI Runtime Security and AI Posture Management. The model-supply-chain lineage runs through the Protect AI acquisition, which brought ModelScan and related tooling.
| Deployment | SaaS platform with gateway and runtime enforcement |
|---|
| Attack library | Vendor-maintained; size undisclosed |
|---|
| Agentic / tool-abuse testing | Yes — dedicated Agent Security module plus agent red teaming |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — AI Runtime Security and AI Gateway |
|---|
Watch out: Red teaming is one line item in a large platform SKU — you will not buy it standalone, and pricing is quote-only, so this is a heavier procurement than the problem warrants if all you need is a pre-deployment report. The platform assumes you are also adopting the AI Gateway and runtime layers, which is architectural commitment as well as spend. Product naming in this portfolio has churned repeatedly since the Protect AI acquisition, so confirm which module names are current at contract time.
unknown — quote-only; no published rates for the platform or individual modules.
Adversa delivers agentic AI, LLM, generative AI and MCP red teaming, underpinned by what it calls Self-Evolving Threat Intelligence, which continuously ingests published AI security research, exploits and attack patterns from thousands of sources each month. Its newer Coding Agent Security Platform is a runtime control layer that blocks dangerous actions from Claude Code, Copilot, Cursor and Codex. The team has a long track record in adversarial ML research and public vulnerability disclosure, and is cited as a Gartner Representative Vendor and IDC Innovator.
| Deployment | Services engagements plus a runtime platform for coding agents |
|---|
| Attack library | SETI threat intelligence ingested from public research monthly |
|---|
| Agentic / tool-abuse testing | Yes — agentic and MCP red teaming are named offerings |
|---|
| Framework mapping | unknown — not stated on public pages |
|---|
| Runtime guardrails from same vendor | Yes — Coding Agent Security Platform |
|---|
Watch out: Small team with a services centre of gravity, so throughput and turnaround depend on their availability — this is not a scanner you drop into CI and forget. Product focus has visibly shifted toward the coding-agent runtime control layer, so confirm that pre-deployment red teaming is still a first-class engagement rather than an add-on to a platform sale. No published pricing, and being research-led means the depth is in the people, so ask who specifically is assigned to your engagement.
unknown — not disclosed publicly; requires a demo conversation.
A scoped human penetration test rather than a scanner. Bugcrowd matches researchers with demonstrated LLM security experience via its CrowdMatch system, tests for prompt injection, excessive agency, training-data poisoning and the rest of the OWASP Top 10 for LLMs, and delivers customisable reports that support ISO 27001, SOC 2, HIPAA, GDPR and PCI DSS evidence requirements. It holds CREST accreditation for penetration testing, which matters when a customer's security questionnaire asks who performed the test and under what standard.
| Deployment | Managed human penetration test engagement |
|---|
| Attack library | Researcher expertise plus platform vulnerability corpus |
|---|
| Agentic / tool-abuse testing | Yes — excessive agency and tool misuse are named in scope |
|---|
| Framework mapping | OWASP Top 10 for LLMs; ISO 27001, SOC 2, HIPAA, GDPR, PCI DSS reporting |
|---|
| Runtime guardrails from same vendor | No — services only |
|---|
Watch out: Point-in-time and slow: one scoped engagement produces one report and zero continuous coverage, so it complements rather than replaces automated testing in CI — your system prompt will change the week after delivery. Pricing is quote-only per engagement and typically an order of magnitude above tooling. Output quality varies with which researchers get matched to your scope, and HackerOne offers a near-identical AI pentest product, so run both quotes before committing.
unknown — quote-only, priced per engagement by scope.