AI Slop Bug Bounty Reports: How TRIAGERS™ Detects LLM-Generated Garbage

Somewhere in the last couple of years, the marginal cost of producing a plausible-looking vulnerability report dropped to approximately zero. Anyone can now paste a target's docs into a chatbot, ask for "a critical vulnerability report for a bug bounty program," and receive eight hundred words of confident, well-formatted, technically-flavoured prose describing a vulnerability that does not exist, on an endpoint that does not exist, with curl output that was never run.

Open source maintainers were the canary: curl's maintainers have been publicly fighting off hallucinated CVE reports for years now. Bug bounty programs are the main event, because bug bounty attaches money to the output. If a report costs nothing to generate and there's any nonzero chance a tired triager waves it through, the spam equilibrium is obvious. Every program we work with has watched this category of noise grow from a curiosity to a meaningful fraction of the queue.

Bad reports are old news. What's new is that slop is plausible: it's grammatically clean, uses the right vocabulary, cites the right OWASP categories, and reads better than many genuine reports from skilled researchers who happen to write English as a third language. Slop is optimised for exactly the shallow read that an overloaded triage queue encourages.

The anatomy of a slop report

After you've handled a few hundred of these, the patterns are unmistakable.

Hallucinated attack surface

The report describes POST /api/v1/admin/impersonate in loving detail. Your application has never had that endpoint. LLMs pattern-match on what applications typically have, so slop reports reference plausible-but-fictional endpoints, parameters, and headers, often borrowed from whatever framework's documentation dominated the training data. If a report names an endpoint you can't find in your own routing table, that's not a maybe.

The curl output that never ran

Slop reports love terminal evidence, and the terminal evidence is fake. Tells we see constantly:

• Response headers in an impossible combination (a Server: nginx header on your Cloudflare-fronted app that says cloudflare on every real response).
• Timestamps that don't parse, don't match the report's timeline, or predate the researcher's account.
• A JSON response body whose schema matches nothing your API returns, usually a nicer, more generic schema than your real one, because it was imagined rather than observed.
• HTTP/1.1 200 OK on an endpoint that has only ever spoken HTTP/2 through your edge.
• The classic: a "PoC" whose request would 404, presented alongside a fabricated 200.

Real evidence is messy and specific: odd header casing, your real trace IDs, the weird envelope your API wraps errors in. Fabricated evidence is smooth and generic, and the smoothness itself is the tell.

Internal inconsistency

Slop doesn't maintain state. The report says the vulnerable parameter is user_id in paragraph two and uid in the PoC. The impact section describes SSRF; the remediation section recommends output encoding, because somewhere mid-generation the model drifted to XSS. Severity says Critical; the described impact is a version banner. A human who actually exploited a bug doesn't contradict themselves about which bug it was.

Confident vagueness

The prose is assertive and the specifics are absent. "An attacker can leverage this vulnerability to gain unauthorized access to sensitive data, leading to severe consequences for the organization." Which data? Via which request? Show the response. Genuine reports, even badly-written ones, are anchored to concrete observed behaviour. Slop is anchored to the genre of vulnerability reports.

The clarifying-question test

This is the most reliable discriminator we have, and it's the same technique we use in normal triage: ask one specific, narrow question that anyone who actually performed the attack could answer trivially:

"At step 3, what response status did you get when the session token was omitted?"

A real researcher answers in minutes, from memory or their proxy logs. Slop submitters do one of three things: go silent, paste another wall of generated text that doesn't address the question, or (increasingly) paste your question into the same model and return an answer that's wrong in a checkable way. One round-trip usually settles it.

The important discipline: ask the question neutrally, exactly as you would for any report with a gap. If it's a real researcher, you've done normal triage and nobody's insulted. If it's slop, the silence closes the report for you. You never had to make an accusation either way, which is precisely the point.

Why triage is the layer that has to absorb this

None of the detection above is automatable in a satisfying way, and the reports are engineered (accidentally, by the nature of LLMs) to pass a skim. What catches slop is someone who knows the target's real attack surface and real response formats reading with the expectation of verifying claims against reality. That's just triage, done properly; what slop has changed is the price of doing the job lazily.

It has, however, changed the economics of the queue. Every fabricated report still costs a real human minutes to confidently dismiss, and confident dismissal matters: falsely accusing a legitimate researcher of submitting AI garbage is a reputational injury you don't recover from quickly. The volume also silently degrades your signal: the report you skim-close at 4:55pm because it "reads like slop" might be the real one. Programs feeling this as unsustainable load are usually feeling a triage capacity problem that slop merely exposed.

Why banning AI is the wrong response

The tempting policy is "AI-generated reports are banned." It fits in a tweet, and it's wrong, for three reasons.

First, your best researchers use AI heavily and legitimately. They use LLMs to analyse minified JavaScript, generate fuzzing harnesses, write up findings in clean English, and script their recon. A skilled researcher using a model to polish the write-up of a real, hand-verified bug has produced a better report. Banning "AI-generated reports" either bans this (losing you good reports) or doesn't actually mean what it says.

Second, you can't enforce it. AI-text detectors are unreliable, discriminate against non-native English writers, and lose the arms race by default. A policy you can't enforce, applied by vibes, generates exactly the false accusations you can't afford.

Third, it aims at the wrong target. The problem was never the tool. The problem is unverified claims. A hallucinated vulnerability written by hand in 2019 was equally invalid; the model just industrialised the volume. Write your policy about the failure mode, not the tool.

Policy recommendations that actually work

What we recommend programs put in their policy pages, based on what's held up in practice:

1. Require demonstrated reproduction, explicitly. "Every report must include steps and evidence from an actual test against the target: real requests, real responses. Reports describing theoretical or unverified issues will be closed as Not Applicable." This closes slop without mentioning AI at all.
2. Make fabricated evidence a named offence. "Submitting fabricated request/response data or PoC output is grounds for banning" (distinct from being wrong, which is fine and normal). This is the distinction that protects honest researchers.
3. Reserve the right to ask verification questions, with a response window. State that reports may receive specific technical questions and that non-responsive reports will be closed. It formalises the clarifying-question test and gives your triagers a clean, non-accusatory closure path.
4. Don't ban AI assistance; say so. One sentence ("You're welcome to use AI tooling; you are fully responsible for verifying everything you submit") keeps your best researchers comfortable and removes the ambiguity that fuels disputes.
5. Track slop as its own closure category. If you fold it into "Not Applicable" you lose the trendline, and the trendline is what tells you whether a submitter, a campaign, or a new tool is driving a spike. It also keeps your triage metrics honest: a month of slop-flood shouldn't read as "report quality declined."

For researchers reading this and wincing: the flip side is that provably-real evidence is now worth more than ever. A report with clean proxy exports, consistent details, and fast answers to clarifying questions sails through queues that are otherwise clogged with generated sludge. We've written up exactly what makes a report get paid fast. In 2026, "obviously written by someone who actually ran the attack" is a competitive advantage.

Slop is not going away. Generation keeps getting cheaper; verification doesn't. Programs that treat triage as the verification layer it always was will be fine. Programs that skim will pay bounties for fiction, and word gets around about those too.

TRIAGERS™' on-demand triage teams sit between your program and the queue (including the fabricated parts of it), validating every report against the actual target before your engineers ever see it. If slop is eating your team's week, get in touch.