Working with Agentic `Teammates': When a New Organizational Actor Collides with the Human Ecosystem of Work
Organizations: Google Research, USA · Google DeepMind, USA · Google DeepMind, Switzerland · Google DeepMind, United Kingdom · Google DeepMind, Canada
Abstract
Enterprise AI is transitioning from single-user, reactive tools toward proactive, multi-user 'teammates,' but our empirical understanding of this transition is limited. In this paper, we present an in-situ qualitative study of a persistent, proactive AI agent 'teammate' deployed across multiple teams in a large technology company. Our findings reveal the boundaries of the human-agent workplace are actively in flux, triggering breakdowns and negotiations across: 1) tacit rules of collaborative human workflows, 2) the relational boundaries of this new non-human actor, and 3) the redistribution of trust and human agency. We use these early micro-negotiations as signals to chart a new research, design, and organizational agenda that intentionally preserves human agency in a workplace shared with non-human organizational actors.
Figures & tables
| Support category | Example functionalities | Reactive Interaction Example | Proactive Interaction Example |
|---|---|---|---|
| Bug Tracking | Filing bugs, reordering priorities of bugs, reviewing code to identify bug context | “Add a comment to [bug number] saying that we’ve found the root cause in the [package]” | Proactively files bugs based on context from chat discussions |
| Scheduling & Planning | Finding availability, scheduling meetings, meeting prep | “DM me the background docs and last week’s action items 10 minutes before the ’X Meeting’ begins.” | Infers if a meeting needs to be scheduled based on chat context and proactively schedules |
| Communication | Summarizing threads, communication coaching, feedback about interpersonal communication, sending DMs to people, facilitating discussions in chat | “Tell me what I missed in [project chat space] while I was away” | Nudges de-railing conversations offline to preserve focus of a group chat space |
| Information Management | Creating documents, summarizing documents, file organizing, searching internal company knowledge repository, code repositories, and web search | “What is the deployment process and checklists I have to follow for [project]” | Identifies a brainstorming discussion in chat and creating a project document capturing the idea |
| Project Monitoring & Intelligence | Updating team work status and knowledge repository based on chats and docs; preparing briefings for review; join meetings for discussion | "Create documentation based on new leanings from [issue] debugging session in chat for the team" | Flags architectural drift against living prior team decisions and design documents |
| Job Role | Duration of use |
|---|---|
| Software Engineer ( ) | 5 months ( ) |
| Research Scientist ( ) | 4 months ( ) |
| Program Manager ( ) | 3 months ( ) |
| Product Manager ( ) | 2 months ( ) |
| 1 month ( ) | |
| 1 week ( ) |
| Observed collision | Stakes of the collision | Design and governance implications | |
|---|---|---|---|
| Collaborative Workflows and Cultures (Section 4.1 ) | Even broadly capable agents can fail to grasp unwritten social and collaborative norms of work causing unintended disruption | • Agents inadvertently “pollute” documents and chat spaces • Agents create more work for people who have to fix the its mistakes | • Re-architect the current human-centric technical stack to accommodate non-human actors in a safe manner (e.g., through agent-specific permissions) • In-context evaluations that assess agent behaviors within real-world workflows |
| Relational Norms and Categories (Section 4.2 ) | In the absence of an established relational contract of work with agents, participants engaged in fragmented personal negotiations about what the agent is, what behaviors are appropriate, and what its position is in the organizational hierarchy | • The same agent behaviors can have polarizing effects—building rapport for some users and immediately alienating others • Mental labor of sense-making and clarifying boundaries falls on users triggering anxieties and confusion | • Development of ‘code of conduct’ for hybrid human-agent teams that define norms of appropriateness and firm social boundaries • Organizational structures and policies that explicitly position the agent within an organizational and team hierarchy to establish accountability • Provide contextual control knobs that afford teams the agency to define and tune the agent’s social persona and behavioral boundaries locally |
| Agency and Trust (Section 4.3 ) | Acceptance of the agent’s proactivity and autonomy was mediated by how users perceived it might impact their own agency, and the expectation that agency is a progressively earned privilege | • Uncalibrated proactivity breaks users’ trust and erodes perceived value of the agent • Forced adoption erodes psychological ownership over and willingness to use the agent | • Implement processes for ‘earned’ agency through progressive release of proactive capabilities • Ensure deployment of team-embedded agents happens through processes of consent |