cs.AIJul 10, 2025

Working with AI: Measuring the Applicability of Generative AI to Occupations

Authors: Kiran Tomlinson, Sonia Jaffe, Will Wang, Scott Counts, Siddharth Suri

Organizations: Microsoft Research · Microsoft

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

With generative AI emerging as a general-purpose technology, understanding its economic effects is among society's most pressing questions. Earlier work on AI impact largely relied on predictions of AI capabilities or focused narrowly on individual firms. Drawing instead on real-world AI usage, we analyze a dataset of 200k anonymized conversations with Microsoft Bing Copilot to measure AI applicability to occupations. We use an LLM-based pipeline to classify the O*NET work activities assisted or performed by AI in each conversation. We find that the most common and successful AI-assisted work activities involve information work--the creation, processing, and communication of information. At the occupation level, we find widespread AI applicability cutting across sectors, as most occupations have information work components. Our methodology also allows us to predict which occupations are more likely to delegate tasks to AI and which are more likely to use AI to assist existing workflows. Finally, we see an increase in occupation-level AI applicability over our 9-month data period, driven primarily by people using AI for more varied tasks. AI applicability scores and component usage and success metrics are available at https://github.com/microsoft/working-with-ai.

Figures & tables

Figure 1: Frequency of work activities in Bing Copilot usage. ( A ) The 15 IWAs that occur most frequently as user goals. ( B ) The 15 IWAs that occur most frequently as AI actions. ( C ) The frequencies of GWAs as user goals and AI actions, compared to a rough estimate of GWA workforce frequency (described in Appendix B ). All panels show researcher-generated IWA groupings.
Figure 2: Success of Bing Copilot work activities. The top and bottom 15 IWAs (with user goal or AI action activity share at least 0.05%) by the product of their completion rate and the share of conversations where Bing Copilot demonstrated at least a moderate scope of capability.
Figure 3: Top occupations by AI applicability score and their contributing IWAs. The 10 SOC occupations with the highest AI applicability scores (right) along with the 10 IWAs that provide the greatest contributions to those scores (left). Asterisks indicate 2023 U.S. occupational employment (none: up to 50k, one: 50k to 100k, two: 100k to 500k, three: more than 500k); the heights of each stratum indicate the same. Occupations’ applicability scores are in parentheses, and occupations are sorted by their score in decreasing order. Portions of the occupational strata not connected to an IWA by a colored flow represent IWAs not present in the Figure that still contribute to the occupation’s applicability score. Both occupation and IWA titles have been shortened for space.
Figure 4: AI applicability of occupational groups. (A) Employment-weighted boxplots of major group overall AI applicability (average of user goal and AI action scores), highlighting high scores for information work. ( B ) Scatterplot of each major occupation group’s (employment-weighted) average AI action applicability score and average user goal applicability score. The dashed gray line shows the first principal component, capturing overall applicability. The blue and red lines show the extent to which a major group has higher applicability from one metric versus the other.
Minor Group Title (Abbr.)ScoreMinor Group Title (Abbr.)ScoreMinor Group Title (Abbr.)Score
Media and Communication Workers *0.38Food and Beverage Serving Workers **0.20Woodworkers0.12
Sales Representatives, Services **0.35Other Educational and Library Workers *0.19Top Executives **0.11
Information and Record Clerks **0.33Material Recording and Distrib. Workers **0.19Construction and Extraction Sups. *0.11
Mathematical Science Occupations0.32Prim., Second., and Special Ed. Teachers **0.18Assemblers and Fabricators *0.11
Tour and Travel Guides0.32Media and Comms. Equipment Workers0.18Printing Workers0.11
Postsecondary Teachers *0.31Advertising, Mktg., PR, and Sales Managers *0.18Health Technologists and Technicians **0.10
Table 1: SOC minor groups by AI applicability score.
Figure 5: Monthly trends for occupation-level metrics. ( A ) Average AI applicability scores. ( B ) Average occupational coverage. ( C ) Average occupation-aggregated completion. ( D ) Average occupation-aggregated scope share. All four metrics are employment-weighted averages over occupations, with IWA-level metrics in B–D aggregated to occupations via weights wijw_{ij} . The shaded regions show 95% confidence intervals for the weighted mean.
Appendix figures & tables33 assets

Supplementary material from the paper’s appendix.

Appendix

Figure K.6: Distribution of IWAs per conversation. ( A ) The distribution of the number of IWAs to which a conversation is matched for user goal in both the uniform and thumbs Copilot datasets. ( B ) The same distributions for AI action IWAs.
Figure K.7: IWA frequency in AI actions and user goals. For each IWA, this Figure plots the fraction of user intents ( xx axis) and AI actions ( yy axis) described by that IWA. Outliers show which IWAs are more likely to be performed (blue) or assisted (red) by Bing Copilot. When a conversation is labeled with multiple IWAs, that share of Copilot activity is evenly distributed among the IWAs so that the sum of all IWA fractions is 1. Table L.2 lists the relative frequencies for the most off-diagonal IWAs.
Figure K.8: IWAs with the highest and lowest shares of positive feedback. This Figure shows the top and bottom 15 IWAs by the share of positive feedback, filtered to common IWAs matched in at least 1% of conversations in our feedback dataset, with bootstrapped 95% confidence intervals. The common IWAs with highest positive feedback share include two about writing and editing; four about evaluating or purchasing goods and services; and six about researching information about health, culture, law, policy, and society. Meanwhile, the common IWAs with the lowest positive feedback share include five visual design and five data analysis IWAs.
Figure K.9: Share of positive feedback by GWA. This Figure plots the positive feedback share for each GWA, aggregating common IWAs into their GWAs and with bootstrapped 95% confidence intervals; any IWA appearing in less than 1% of our feedback data is ignored. 14 GWAs have no common IWAs and are thus excluded from this plot, including those relating to operating vehicles, repairing equipment, and management tasks like hiring, negotiation, and guiding subordinates.
Figure K.10: Relationship between thumbs feedback and task completion rate for each IWA. ( A ) Scatterplot of user goal IWA positive feedback share (measured in Copilot-Thumbs ) and completion rate (measured in Copilot-Uniform ). ( B ) The same scatterplot for AI action IWAs. There is a strong correlation between thumbs feedback rate and GPT-4o-mini task completion rate for each IWA, indicating that both are capturing real signal about AI success in assisting or performing an IWA. Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.11: Correlation between IWA completion rates in Copilot-Uniform and Copilot-Thumbs . ( A ) Scatterplot of user goal IWA completion rate in our two datasets. ( B ) The same scatterplot for AI action IWAs. IWA-level completion rates are consistent between Copilot-Uniform and Copilot-Thumbs . Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.12: IWAs with the highest and lowest completion rates. This Figure shows the top and bottom 15 IWAs by completion rate, filtered to ‘common’ IWAs with activity share at least 0.1% in Copilot-Uniform , with 95% confidence intervals (using the normal approximation to binomial confidence intervals). Task completion shows some of the same patterns as positive feedback fraction ( Figure K.8 ), with visual design and data analysis on the low end and writing on the high end. One notable difference is that the highest completion rate IWAs include 7 about explaining, advising, or training.
Figure K.13: IWAs with the highest and lowest fraction of conversations at moderate or higher scope. This Figure shows the top and bottom 15 IWAs by how often they are assigned scope of impact at least moderate, filtered to “common” IWAs with activity share at least 0.1% in Copilot-Uniform , with 95% confidence intervals (using the normal approximation to binomial confidence intervals). Some patterns mirror those of thumbs feedback and completion, with research and writing IWAs having high scope, and data analysis and visual design IWAs having low scope. AI performance consistently has lower impact scope than user assistance.
Figure K.14: Relationship between scope and task completion rate for each IWA. ( A ) Scatterplot of user goal IWA moderate or higher scope share and completion rate. ( B ) The same scatter plot for AI action IWAs. The relationship between scope and completion is much weaker than between completion and thumbs feedback, as they are capturing different questions. Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.15: Correlation between scope and activity share in Copilot-Uniform for each IWA. ( A ) Scatterplot of user goal IWA activity share and the fraction of conversations in which the scope is classified moderate or higher. ( B ) The same scatterplot for AI action IWAs. On both sides, impact scope is a good predictor of what activities people seek AI assistance with (better than completion or satisfaction).
Figure K.16: AI applicability score for major occupation groups. The employment-weighted mean, median, and inter-quartile range of the AI applicability score for each major occupation group. Highlighted occupational groups have a majority of workers in occupations classified as information work.
Figure K.17: Top 25 occupations by AI applicability score and their contributing IWAs. Similar to Figure 3 , the 25 SOC occupations with the highest AI applicability scores (right) along with the 20 IWAs that provide the greatest contributions to those scores (left). Asterisks indicate 2023 U.S. occupational employment (none: up to 50k, one: 50k to 100k, two: 100k to 500k, three: more than 500k); the heights of each stratum indicate the same. Occupations’ applicability scores are in parentheses, and occupations are sorted by their score in decreasing order. Portions of the occupational strata not connected to an IWA by a colored flow represent IWAs not present in the Figure that still contribute to the occupation’s applicability score. Both occupation and IWA titles have been shortened for space.
Figure K.18: AI applicability scores by predicted exposure AI applicability score versus E1 exposure from Eloundou et al. [ 3 ] , the fraction of an occupation’s tasks for which human raters predicted LLMs could reduce completion time by 50%. Occupations are colored by their distance from the employment-weighted regression line: blue points have higher AI applicability scores than would be expected from their E1 exposure.
Figure K.19: AI applicability scores by average wage and educational requirement. (A) AI applicability score by average occupational wage. There is a point for each occupation and darker points for the employment-weighted average of each ventile. Binscatter conditional means and least squares fits are weighted by employment. (B) AI applicability score by modal education requirement reported to O*NET. Boxplot medians and quartiles are weighted by employment (e.g., they show the score of the median worker rather than that of the median occupation).
Figure K.20: AI applicability score by wage, separating out user goals and AI actions. ( A ) Wage plot as in Figure K.19 A, but showing user goal AI applicability score only. ( B ) AI action applicability score only. The correlation is higher on the AI side.
Figure K.21: AI applicability score by educational requirement, separating out user goals and AI actions. ( A ) Education plot from Figure K.19 B, but showing only user goal AI applicability score. ( B ) Using AI action scores. As with wage, the relationship with education is stronger on the AI side.
Figure K.22: AI applicability score by wage, unweighted . This Figure is the same as Figure K.20 without weighting the binscatters by employment. ( A ) Scatterplot with unweighted binscatter of each occupation’s AI applicability score calculated over user goals only against the occupation’s average wage. ( B ) Using AI action AI applicability score. The relationship is much less noisy without employment weighting, likely because weighting by employment causes the binscatters to be influenced dramatically by a small number of high-employment occupations, increasing the variance due to noise in the coverage metric and in the mapping between occupations and IWAs. Excluding the top 10% of highest-paid workers strengthens the correlation.
Figure K.23: AI applicability score by educational requirement, unweighted. This Figure is the same as Figure K.21 without weighting by employment. ( A ) User goal AI applicability scores over occupations with different educational requirements, with boxplots over occupations (not weighting by employment). ( B ) Using AI action scores. As in Figure K.22 , this reduces the noise in the relationship.
Figure K.24: Correlations between AI applicability score and 2024 Current Population Survey occupational demographics, weighted by employment. ( A ) Correlation with percent women. ( B ) Correlation with percent white. ( C ) Correlation with percent black or African American. ( D ) Correlation with percent Asian. ( E ) Correlation with percent Hispanic or Latino. ( F ) Correlation with median age. For each panel, we drop occupation codes with missing demographic data in the 2024 CPS Annual Averages.
Figure K.25: Rationale for our chosen coverage threshold. (A) Effects of coverage threshold on number of occupations with user goal coverage 0 and 1. Our threshold of 0.05% approximately minimizes the number of occupations assigned user goal coverage 0 or 1. (B) The correlation between AI applicability scores defined using different coverage thresholds and the one we report with threshold 0.05%. Across thresholds spanning three orders of magnitude, we get strongly correlated rankings of occupations by AI applicability. Contrast this robustness of relative applicability score with the absolute measure in Figure K.26 , which is highly sensitive to arbitrary choice of threshold.
Figure K.26: Effect of activity share threshold on absolute impact estimates. The share of workers who have at least x%x\% of their work in covered IWAs for different definitions of an IWA being covered ( .001%,…,1%.001\%,\ldots,1\% of user chat activity). The resulting numbers depend significantly on the selected threshold, making relative statements more meaningful than absolute coverage numbers.
Figure K.27: Mean and standard deviation of occupation coverage by threshold. ( A ) The averages and standard deviations of occupation user goal coverage for different thresholds for the share of chat activity an IWA must have to “be done” with the LLM. ( B ) The same means and standard deviations for coverage derived from AI action IWAs. We use a threshold of 0.0005, which results in 127 and 87 IWAs being considered covered (of 332) on the user goal and AI action sides, respectively.
Figure K.28: Distribution of occupation coverage. ( A ) The distribution of occupation user goal coverage scores (fraction of importance-weighted work in an occupation with user goal activity share at least 0.05% in Copilot-Uniform ). ( B ) The same distribution for AI action coverage.
Figure K.29: IWAs contributing to user goal vs. AI action asymmetry. This tripartite graph shows six prominent occupation groups that are skewed toward user goal or AI action applicability (left), the three IWAs contributing the most to each group’s skew (middle) ordered by the extent to which those IWAs skew more to the AI action side, and the GWAs those IWAs belong to (right). Blue indicates a larger skew toward AI action applicability, while red indicates a larger skew toward user goal applicability. White is neutral. To identify which IWAs contribute most to an occupation group’s skew, we take the second component magnitudes in an IWA-level PCA (where we perform PCA on IWA-level scope ×\times completion ×\times coverage indicator), multiply them by IWA weights wijw_{ij} for each occupation, and then take an employment-weighted average to get a score for each occupational-group–IWA pair. IWA-level scores are averaged to get GWA-level scores.
Figure K.30: Monthly IWA breadth metrics. ( A ) Number of IWAs whose activity share crossed the 0.05% coverage threshold within each month. ( B ) Entropy of the IWA distribution in each month. Both metrics show an increasing variety in user goals and AI actions over the data period, indicating users performing more varied tasks over time.
Figure K.31: Ratio of IWA activity share by rank between last and first three months. Users performed tail IWAs (ranks ∼20−250\sim 20-250 ) more often at the end of the data period than at the beginning, reinforcing the trends from Figure K.30 .
Figure K.32: Monthly conversation-level metrics. ( A ) Fraction of conversations in each month labeled as task completed. ( B ) Fraction of IWA matches on each side labeled with scope moderate or higher. ( C ) Share of thumbs feedback that was positive by month.
Figure K.33: Structured output formats for our four conversation-level LLM prompts. ( A ) Generation prompt. ( B ) Completion. ( C ) Classify, user goals. ( D ) Classify, AI actions.
More often assisted by AIMore often performed by AI
Purchase goods or services. (118.4x)Train others on operational or work procedures. (17.9x)
Execute financial transactions. (58.8x)Train others to use equipment or products. (16.0x)
Perform athletic activities. (47.3x)Distribute materials, supplies, or resources. (11.2x)
Obtain information about goods or services. (25.9x)Train others on health or medical topics. (11.2x)
Research healthcare issues. (20.5x)Provide general assistance to others. (10.9x)
Prepare foods or beverages. (14.7x)Coach others. (10.6x)
Appendix
Table L.2: Work activities with largest ratios between user goal and AI action activity share.
Major GroupCoverageCompletionScopeScoreEmployment
Computer & Mathematical0.640.860.480.295,177,390
Sales & Related0.560.890.510.2913,266,370
Office & Admin. Supp.0.560.890.490.2618,163,760
Community & Social Service0.510.880.440.252,216,930
Arts/Design/Ent./Sports/Media0.590.800.490.242,039,830
Business & Financial Ops.0.490.890.470.2310,087,850
Appendix
Table L.3: SOC major groups by AI applicability score.
Job Title (Abbrv.)CoverageCompletionScopeScoreEmployment
Interpreters and Translators0.9800.8830.5680.49251,560
Historians0.9060.8520.5640.4623,040
Writers and Authors0.8500.8410.6030.45449,450
Sales Representatives of Services0.8420.9020.5730.4491,142,020
Computer Numerically Controlled Tool Programmers0.8990.8690.5280.41928,030
Broadcast Announcers and Radio Disc Jockeys0.7450.8410.6050.40925,070
Appendix
Table L.4: Top 40 occupations with highest AI applicability score. See note in Table L.3 for column definitions.
Job Title (Abbrv.)CoverageCmpltn.ScopeScoreEmpl.
Helpers–Extraction Workers0.0890.9560.3360.0287,360
Helpers–Painters, Paperhangers, Plasterers, and Stucco Masons0.0420.9570.3760.0277,700
Plant and System Operators, All Other0.0530.9270.3750.02615,370
Oral and Maxillofacial Surgeons0.0530.8940.3400.0264,160
Embalmers0.0660.5450.2200.0263,380
Automotive Glass Installers and Repairers0.0450.9300.3420.02516,890
Appendix
Table L.5: Bottom 40 occupations with lowest AI applicability score. See note in Table L.3 for column definitions
AI assistance, not performanceAI performance, not assistance
Cooks, Fast Food (83, 6)Exercise Trainers and Group Fitness Instructors (17, 75)
Meat Cutters and Trimmers (79, 6)Choreographers (34, 81)
Butchers and Meat Cutters (83, 11)Training and Development Managers (45, 88)
Animal Breeders (76, 6)Coaches and Scouts (43, 80)
Cooks, Private Household (97, 32)Actuaries (42, 76)
Cooks, Restaurant (76, 12)Environmental Engineers (55, 88)
Appendix
Table L.6: Occupations with the largest difference in user goal and AI action applicability score percentiles.
More in this topic

AI and Labor

AI and labor studies how artificial intelligence changes jobs, wages, skills, hiring, and the organization and quality of work.

See all 94 papers

Explore similar work

CardsList
Jul 22, 2026

Google's AI & Economy ATLAS v1.0: Mapping Gemini Usage in the Economy

This paper introduces the AI & Economy ATLAS (Activity, Task, Landscape, and Adoption Study), an ongoing economic research initiative using Google AI usage data. The first iteration of ATLAS is built on 15 million de-identified interactions across the Gemini App, Google AI Mode, and Gemini API. Using privacy-preserving algorithms as well as established and bespoke classification methods, we map AI usage to over 800 occupations, 4000 tasks, 300 household activities, 150 countries, and 140 languages. We then make a number of observations on what the data reveals about AI's diffusion, and its usage at work and in day-to-day life. In the workplace, we show that while AI adoption spans occupations covering just above 88% of US employment, penetration remains shallow and overwhelmingly collaborative in nature, with end-to-end task automation limited in scope. Outside of work, AI spans activities making up about 98% of Americans' non-sleep time, with disproportionately high use in high-friction tasks such as engaging with government and professional service providers, likely delivering economic value that standard national accounts may miss. Globally, adoption scales with national wealth and has broad linguistic distribution, with English queries representing only around a third of volume. As we build upon ATLAS and expand its scope and capabilities, we will continue to provide large-scale empirical evidence to inform the public, policy and academic questions about the ongoing AI transformation.

Zanna Iscenko, Scott Strand, Yiyuan Chen +15

Google · Google DeepMind

Generative AI AdoptionHuman-AI CollaborationAI and Labor
AskPDF
May 31, 2026

Measuring the Occupation-Level Impact of AbbVie Intelligence: AI Applicability Analysis, 2024-2025

This paper presents an empirical analysis of AbbVie Intelligence's measurable impact on employee work activities across 192 distinct occupations in 2024 and 2025. Drawing on 598,744 de-identified AI conversations classified according to the O*NET Intermediate Work Activity (IWA) taxonomy, we compute occupation-level AI Applicability Scores that quantify the extent to which AI tools can meaningfully assist or automate real work at scale. Three convergent analyses are conducted: (1) longitudinal year-over-year trends from 2024 to 2025, (2) a quasi-experimental pre-post evaluation of the AbbVie Intelligence version 3 platform release in August 2025, and (3) a pre-post evaluation of the AbbVie AI Learning Summit held in November 22025. Results demonstrate statistically significant improvements across all three dimensions. Mean AI Applicability Scores rose substantially from 2024 to 2025; the platform release product a +10.0% gain (p<0.001); and the AI Learning Summit produced a +6.68% gain (p<0.001). These findings establish that both technological platform enhancements and structured enterprise AI eduction programs independently and substantially expand the reach of AI across the AbbVie workforce.

John Regan, Jon Stevens, Brian Martin

AbbVie, BTS Informa2on Research, Enterprise Product Incubator for Capabili2es (EPIC)

Organizational AI AdoptionAI and Labor
AskPDF
May 23, 2026

The Open Source Economic Index of AI Adoption and Capability

We work towards measuring both AI adoption and the capability of AI to perform discrete labor tasks across various occupations. To measure adoption, we develop an open-source economic index that uses publicly available user-LLM chat data and ONET tasks to replicate studies produced by frontier AI labs, finding that occupations in the finance, computer science, and arts sectors are those with the highest adoption rates. To measure capabilities, we build a system that generates benchmark scenarios grounded in ONET occupations, tasks, and model-context-protocol (MCP) servers. We test Kimi-k2.5 with an OpenAI agents SDK harness on scenarios across 9 occupations that appear frequently in our index, finding that AI correctly executes high-level workflows but often errs in the granular details (such as specific tool calls used).

Seamus Somerstep, Aritra Guha, Divesh Srivastava +1

University of Michigan · AT&T Chief Data Office · IFM-MBZUAI

AI Agent EvaluationAI Agent BenchmarksAI and Labor
AskPDF