risk
Emergent behaviour and unsafe AI system integration
Large-scale or multi-model pipelines exhibit emergent capabilities/failures not present in any component and not predictable from component testing; integration with legacy systems introduces interface mismatches and configuration errors.
In catalog since 2026-09-17T22:28:00Z · Last changed 2026-09-17T22:28:00Z (f368a6cce277)
Record JSON · Open in map · Data retrieval guide
Catalog revision: 791ff2dd3a45707290badee660f185e514d15f1cf518908628f425c2f2c56ee4. A connection does not establish full coverage.
Attributes
- category
- ai_governance
- domain
- AI Governance
- Secure Development (SDLC) & Application Security
- taxonomy
- iso-23894-ai-risk
- inherent_rating
- medium
Details
- risk_id
- ai-emergent-integration-risk
- category
- ai_governance
- likelihood
- medium
- impact
- high
- inherent_rating
- medium
- treatment
- mitigate
- taxonomies
- iso-23894-ai-risk
Source
No record-specific source URL is provided.
Connections
- UC-AI-19 — Constrain agent actions and tool use to authorized scope mitigates Emergent behaviour and unsafe AI system integration
- strength
- related
- rationale
- Bounding what an agent may invoke limits the blast radius when integrated components behave in unforeseen ways.
- UC-AI-07 — Verify, validate, and control AI deployment and changes mitigates Emergent behaviour and unsafe AI system integration
- strength
- primary
- rationale
- System-level V&V against requirements exercises the integrated pipeline, catching interface mismatches and emergent failures component testing misses.
- UC-ACCESS-16 — Authorize, test, and approve changes and development mitigates Emergent behaviour and unsafe AI system integration
- strength
- primary
- rationale
- Pre-production testing and approval gates catch legacy-integration interface mismatches and configuration errors before they reach production.
- UC-SDLC-07 — Approve, test, and accept changes before production release mitigates Emergent behaviour and unsafe AI system integration
- strength
- related
- rationale
- Acceptance testing in a production-representative environment catches interface mismatches and integration config errors, but is not the operative defense against emergent, test-unpredictable AI behaviour.
- UC-AI-06 — Maintain AI system technical documentation mitigates Emergent behaviour and unsafe AI system integration
- strength
- related
- rationale
- Architecture and model documentation helps integrators interface correctly, reducing integration mismatches.
- UC-CONFIG-02 — Authorize, test, and approve changes before production mitigates Emergent behaviour and unsafe AI system integration
- strength
- related
- rationale
- Security/risk impact analysis and pre-production testing catch integration interface/config errors, though emergent AI behaviour is not caught by change testing.
- UC-AI-03 — Document AI system resources and dependencies mitigates Emergent behaviour and unsafe AI system integration
- strength
- related
- rationale
- Documented dependencies and interfaces reduce the interface mismatches and configuration errors introduced when integrating with other systems.