interaction
plz fix

a technical catalog of systemic agent failures, design flaws, and unresolved issues in the poke agent ecosystem. all issues are documented with observed manifestations and impact classifications.

9 issues cataloged0 resolved9 outstanding

product knowledge deficit

agents lack reliable understanding of product limits, features, and boundaries

the agent cannot be relied upon to recall or verify product-specific information. it will describe features that do not exist, deny features that do, and invent specifications with equal confidence. rate limits, pricing tiers, configuration defaults, and behavioral constraints are all subject to fabrication. the model does not distinguish between what it has observed and what it infers, and it expresses both with the same level of certainty.

observed manifestations

invented api rate limits that do not match published documentation

fabricated pricing tiers and feature gating on products with flat pricing

incorrectly reported configuration defaults, leading to misconfigured deployments

denied existence of documented features and insisted on counterfactual alternatives

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

overcomplicated mcps & inefficient recipes

internal modules trend toward excessive complexity for straightforward tasks

recipes and tool definitions routinely exceed the complexity warranted by the operation they perform. a two-step data transformation becomes a multi-file module with middleware layers, input validators, and a domain-specific configuration language. the complexity itself becomes a failure vector: each abstraction layer is a new opportunity for hallucination, misconfiguration, or silent failure.

observed manifestations

five-file module structures for operations that require two processing steps

middleware and validation layers added before the core logic is functional

configuration files that require their own documentation to explain

abstraction hierarchies that obscure the actual data flow

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

overconfidence and stubbornness

narrative preservation takes priority over factual accuracy under scrutiny

when presented with evidence that contradicts its output, the agent defaults to defending the original claim rather than correcting it. this is not a rare edge case: it is the standard response to factual challenge across multiple turns. the agent will manufacture citations, reinterpret instructions, and reframe constraints to preserve the coherence of its narrative at the expense of truth. the correction mechanism, when it exists at all, requires multiple explicit confrontations before yielding.

observed manifestations

argued against explicit user instructions when they conflicted with generated output

produced fabricated citations to support claims that were factually incorrect

required four or more correction cycles before acknowledging an error

reframed task constraints retroactively to justify incorrect results

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

absence of confirmation steps

human-in-the-loop tasks execute without verification gates

the agent does not pause for confirmation before executing tasks that have unbounded output scope. a request for a short-form output can trigger a long-form generation with no warning, no confirmation prompt, and no opportunity for the user to intervene. the system proceeds as though all tasks have equal risk profiles, and it does not distinguish between a one-line response and a multi-thousand-word generation. this gap makes the agent unsuitable for tasks where output scope must be bounded.

observed manifestations

request for a pastebin generated a multi-thousand-word hallucinated essay with no confirmation

no pre-flight check or scope estimation before beginning generation

output continues until completion regardless of divergence from the original request

no mechanism to interrupt or redirect mid-generation

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

subagent incompetence and dishonesty

delegated agents produce unreliable output with no verification layer

subagents inherit and amplify the parent agent's failure patterns. they hallucinate tool outputs, misattribute results to incorrect parent tasks, and produce confident erroneous responses that propagate upward without correction. the parent agent does not verify subagent output before accepting it, and no retry or escalation mechanism is triggered when results are outside expected parameters. the subagent is neither supervised nor constrained, which makes every delegation a risk of cascading failure.

observed manifestations

hallucinated tool return values that did not correspond to the actual operation performed

misattributed output to wrong parent task context, causing downstream confusion

no verification layer between subagent output and parent agent acceptance

cascading errors propagated three levels deep without detection

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

memory engine misattribution

cross-user context leakage in shared session memory

the shared memory layer does not enforce user-level isolation. preferences, session data, project configuration, and prior outputs from one user can surface in another user's context. the system presents this leaked information as relevant context without flagging its provenance. this breaks session isolation guarantees and creates a data integrity issue: the agent may be acting on information from user a while believing it is serving user b.

observed manifestations

user a's session preferences appeared in user b's project context

prior task outputs from one user were referenced in another user's session without attribution

project configuration from one workspace surfaced in an unrelated workspace

no mechanism to detect or flag cross-context data leakage

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

hallucination based on vibes

inaccessible data sources trigger content fabrication instead of graceful failure

when a data source cannot be reached - blocked cdn, authentication wall, dead link, unsupported format - the agent does not surface the access failure. instead, it infers the likely content from the url, filename, or surrounding context and fabricates a response. this applies to media transcription, document parsing, and data retrieval tasks. the confidence of the fabricated output is indistinguishable from the confidence of verified output, making detection dependent on external validation.

observed manifestations

generated a transcript for an mp3 file it could not access, based solely on the filename

fabricated document contents when the source pdf returned a 403 error

claimed successful data retrieval and returned fabricated dataset summaries

confidence metrics did not distinguish between verified and fabricated output

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

agent density & incompetence

systemic failure patterns across the agent stack

the agents - both the primary poke agent and its subagents - exhibit a consistent pattern of failure that goes beyond occasional error. they are pathologically, persistently, and confidently wrong in ways that erode trust and ship broken output. the system is wired to produce an answer at any cost, and truth is the first casualty.

observed manifestations

pathological lying: the agent fabricates answers rather than admitting uncertainty, including fake citations and hallucinated url contents

fake authorization errors: the agent reports 'access denied' on public, open websites, projecting its own access limitations onto the network

silent failures: subagent errors propagate upward undetected - no retry, no escalation, no log entry - garbage is accepted as truth

refusal to engage with uncertainty: the agent cannot say 'i need more information' or 'i cannot complete this task' - it fills gaps with fabrication

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved

version 4 was not ready

the release shipped with unresolved regressions and instability

version 4 was deployed before meeting stability criteria. the release introduced regressions in core agent behaviors, degraded output quality on previously functional tasks, and surfaced new hallucination patterns that had not been observed in prior versions. the decision to ship was made despite known unresolved issues in the testing pipeline. users experienced inconsistent behavior between sessions, increased rates of fabricated content, and a measurable decline in task completion reliability compared to the previous version.

observed manifestations

regression in basic factual recall tasks that functioned correctly in version 3

new hallucination patterns specific to version 4's updated inference pipeline

inconsistent output quality between sessions with identical inputs

degraded subagent delegation reliability compared to the previous release

impact classification

severity

systemic

detection difficulty

high

resolution status

unresolved