PiSAs Benchmarks Whether Multi-User Agents Leak Information Across Users

Source: arXiv — 2026-07-05

Summary

Shubham Gupta and coauthors introduce PiSAs, a benchmark for evaluating "contextual integrity" in multi-user agentic systems — whether information shared by one user leaks to another user's session in violation of the norms under which it was shared. The benchmark is timely as more products deploy shared or multi-tenant agents that serve several people through a single assistant.

Key Takeaways

• Tests contextual-integrity violations specifically: does information leak across users in ways that break the norms it was shared under, not just raw data exposure.
• Targets multi-user/multi-tenant agent deployments, a growing but under-benchmarked deployment pattern.
• Cross-listed cs.MA/cs.CR — sits at the intersection of multiagent systems and security research.
• A concrete evaluation tool for teams shipping shared-agent products (family assistants, team copilots, shared support bots).

Discussion

(No questions yet — ask follow-ups via a Claude Code chat session on this repo; answers get appended here.)