cs.AIJul 10, 2025
SaveWorking with AI: Measuring the Applicability of Generative AI to Occupations
Organizations: Microsoft Research · Microsoft
Abstract
With generative AI emerging as a general-purpose technology, understanding its economic effects is among society's most pressing questions. Earlier work on AI impact largely relied on predictions of AI capabilities or focused narrowly on individual firms. Drawing instead on real-world AI usage, we analyze a dataset of 200k anonymized conversations with Microsoft Bing Copilot to measure AI applicability to occupations. We use an LLM-based pipeline to classify the O*NET work activities assisted or performed by AI in each conversation. We find that the most common and successful AI-assisted work activities involve information work--the creation, processing, and communication of information. At the occupation level, we find widespread AI applicability cutting across sectors, as most occupations have information work components. Our methodology also allows us to predict which occupations are more likely to delegate tasks to AI and which are more likely to use AI to assist existing workflows. Finally, we see an increase in occupation-level AI applicability over our 9-month data period, driven primarily by people using AI for more varied tasks. AI applicability scores and component usage and success metrics are available at https://github.com/microsoft/working-with-ai.
Figures & tables
Figure 1: Frequency of work activities in Bing Copilot usage. ( A ) The 15 IWAs that occur most frequently as user goals. ( B ) The 15 IWAs that occur most frequently as AI actions. ( C ) The frequencies of GWAs as user goals and AI actions, compared to a rough estimate of GWA workforce frequency (described in Appendix B ). All panels show researcher-generated IWA groupings.
Figure 2: Success of Bing Copilot work activities. The top and bottom 15 IWAs (with user goal or AI action activity share at least 0.05%) by the product of their completion rate and the share of conversations where Bing Copilot demonstrated at least a moderate scope of capability.
Figure 3: Top occupations by AI applicability score and their contributing IWAs. The 10 SOC occupations with the highest AI applicability scores (right) along with the 10 IWAs that provide the greatest contributions to those scores (left). Asterisks indicate 2023 U.S. occupational employment (none: up to 50k, one: 50k to 100k, two: 100k to 500k, three: more than 500k); the heights of each stratum indicate the same. Occupations’ applicability scores are in parentheses, and occupations are sorted by their score in decreasing order. Portions of the occupational strata not connected to an IWA by a colored flow represent IWAs not present in the Figure that still contribute to the occupation’s applicability score. Both occupation and IWA titles have been shortened for space.
Figure 4: AI applicability of occupational groups. (A) Employment-weighted boxplots of major group overall AI applicability (average of user goal and AI action scores), highlighting high scores for information work. ( B ) Scatterplot of each major occupation group’s (employment-weighted) average AI action applicability score and average user goal applicability score. The dashed gray line shows the first principal component, capturing overall applicability. The blue and red lines show the extent to which a major group has higher applicability from one metric versus the other.
| Minor Group Title (Abbr.) | Score | Minor Group Title (Abbr.) | Score | Minor Group Title (Abbr.) | Score |
|---|---|---|---|---|---|
| Media and Communication Workers * | 0.38 | Food and Beverage Serving Workers ** | 0.20 | Woodworkers | 0.12 |
| Sales Representatives, Services ** | 0.35 | Other Educational and Library Workers * | 0.19 | Top Executives ** | 0.11 |
| Information and Record Clerks ** | 0.33 | Material Recording and Distrib. Workers ** | 0.19 | Construction and Extraction Sups. * | 0.11 |
| Mathematical Science Occupations | 0.32 | Prim., Second., and Special Ed. Teachers ** | 0.18 | Assemblers and Fabricators * | 0.11 |
| Tour and Travel Guides | 0.32 | Media and Comms. Equipment Workers | 0.18 | Printing Workers | 0.11 |
| Postsecondary Teachers * | 0.31 | Advertising, Mktg., PR, and Sales Managers * | 0.18 | Health Technologists and Technicians ** | 0.10 |
Table 1: SOC minor groups by AI applicability score.
Figure 5: Monthly trends for occupation-level metrics. ( A ) Average AI applicability scores. ( B ) Average occupational coverage. ( C ) Average occupation-aggregated completion. ( D ) Average occupation-aggregated scope share. All four metrics are employment-weighted averages over occupations, with IWA-level metrics in B–D aggregated to occupations via weights . The shaded regions show 95% confidence intervals for the weighted mean.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
Figure K.6: Distribution of IWAs per conversation. ( A ) The distribution of the number of IWAs to which a conversation is matched for user goal in both the uniform and thumbs Copilot datasets. ( B ) The same distributions for AI action IWAs.
Figure K.7: IWA frequency in AI actions and user goals. For each IWA, this Figure plots the fraction of user intents ( axis) and AI actions ( axis) described by that IWA. Outliers show which IWAs are more likely to be performed (blue) or assisted (red) by Bing Copilot. When a conversation is labeled with multiple IWAs, that share of Copilot activity is evenly distributed among the IWAs so that the sum of all IWA fractions is 1. Table L.2 lists the relative frequencies for the most off-diagonal IWAs.
Figure K.8: IWAs with the highest and lowest shares of positive feedback. This Figure shows the top and bottom 15 IWAs by the share of positive feedback, filtered to common IWAs matched in at least 1% of conversations in our feedback dataset, with bootstrapped 95% confidence intervals. The common IWAs with highest positive feedback share include two about writing and editing; four about evaluating or purchasing goods and services; and six about researching information about health, culture, law, policy, and society. Meanwhile, the common IWAs with the lowest positive feedback share include five visual design and five data analysis IWAs.
Figure K.9: Share of positive feedback by GWA. This Figure plots the positive feedback share for each GWA, aggregating common IWAs into their GWAs and with bootstrapped 95% confidence intervals; any IWA appearing in less than 1% of our feedback data is ignored. 14 GWAs have no common IWAs and are thus excluded from this plot, including those relating to operating vehicles, repairing equipment, and management tasks like hiring, negotiation, and guiding subordinates.
Figure K.10: Relationship between thumbs feedback and task completion rate for each IWA. ( A ) Scatterplot of user goal IWA positive feedback share (measured in Copilot-Thumbs ) and completion rate (measured in Copilot-Uniform ). ( B ) The same scatterplot for AI action IWAs. There is a strong correlation between thumbs feedback rate and GPT-4o-mini task completion rate for each IWA, indicating that both are capturing real signal about AI success in assisting or performing an IWA. Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.11: Correlation between IWA completion rates in Copilot-Uniform and Copilot-Thumbs . ( A ) Scatterplot of user goal IWA completion rate in our two datasets. ( B ) The same scatterplot for AI action IWAs. IWA-level completion rates are consistent between Copilot-Uniform and Copilot-Thumbs . Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.12: IWAs with the highest and lowest completion rates. This Figure shows the top and bottom 15 IWAs by completion rate, filtered to ‘common’ IWAs with activity share at least 0.1% in Copilot-Uniform , with 95% confidence intervals (using the normal approximation to binomial confidence intervals). Task completion shows some of the same patterns as positive feedback fraction ( Figure K.8 ), with visual design and data analysis on the low end and writing on the high end. One notable difference is that the highest completion rate IWAs include 7 about explaining, advising, or training.
Figure K.13: IWAs with the highest and lowest fraction of conversations at moderate or higher scope. This Figure shows the top and bottom 15 IWAs by how often they are assigned scope of impact at least moderate, filtered to “common” IWAs with activity share at least 0.1% in Copilot-Uniform , with 95% confidence intervals (using the normal approximation to binomial confidence intervals). Some patterns mirror those of thumbs feedback and completion, with research and writing IWAs having high scope, and data analysis and visual design IWAs having low scope. AI performance consistently has lower impact scope than user assistance.
Figure K.14: Relationship between scope and task completion rate for each IWA. ( A ) Scatterplot of user goal IWA moderate or higher scope share and completion rate. ( B ) The same scatter plot for AI action IWAs. The relationship between scope and completion is much weaker than between completion and thumbs feedback, as they are capturing different questions. Point size proportional to square root of IWA match count in Copilot-Uniform . Weighted correlations also weighted by match count in Copilot-Uniform .
Figure K.15: Correlation between scope and activity share in Copilot-Uniform for each IWA. ( A ) Scatterplot of user goal IWA activity share and the fraction of conversations in which the scope is classified moderate or higher. ( B ) The same scatterplot for AI action IWAs. On both sides, impact scope is a good predictor of what activities people seek AI assistance with (better than completion or satisfaction).
Figure K.16: AI applicability score for major occupation groups. The employment-weighted mean, median, and inter-quartile range of the AI applicability score for each major occupation group. Highlighted occupational groups have a majority of workers in occupations classified as information work.
Figure K.17: Top 25 occupations by AI applicability score and their contributing IWAs. Similar to Figure 3 , the 25 SOC occupations with the highest AI applicability scores (right) along with the 20 IWAs that provide the greatest contributions to those scores (left). Asterisks indicate 2023 U.S. occupational employment (none: up to 50k, one: 50k to 100k, two: 100k to 500k, three: more than 500k); the heights of each stratum indicate the same. Occupations’ applicability scores are in parentheses, and occupations are sorted by their score in decreasing order. Portions of the occupational strata not connected to an IWA by a colored flow represent IWAs not present in the Figure that still contribute to the occupation’s applicability score. Both occupation and IWA titles have been shortened for space.
Figure K.18: AI applicability scores by predicted exposure AI applicability score versus E1 exposure from Eloundou et al. [ 3 ] , the fraction of an occupation’s tasks for which human raters predicted LLMs could reduce completion time by 50%. Occupations are colored by their distance from the employment-weighted regression line: blue points have higher AI applicability scores than would be expected from their E1 exposure.
Figure K.19: AI applicability scores by average wage and educational requirement. (A) AI applicability score by average occupational wage. There is a point for each occupation and darker points for the employment-weighted average of each ventile. Binscatter conditional means and least squares fits are weighted by employment. (B) AI applicability score by modal education requirement reported to O*NET. Boxplot medians and quartiles are weighted by employment (e.g., they show the score of the median worker rather than that of the median occupation).
Figure K.20: AI applicability score by wage, separating out user goals and AI actions. ( A ) Wage plot as in Figure K.19 A, but showing user goal AI applicability score only. ( B ) AI action applicability score only. The correlation is higher on the AI side.
Figure K.21: AI applicability score by educational requirement, separating out user goals and AI actions. ( A ) Education plot from Figure K.19 B, but showing only user goal AI applicability score. ( B ) Using AI action scores. As with wage, the relationship with education is stronger on the AI side.
Figure K.22: AI applicability score by wage, unweighted . This Figure is the same as Figure K.20 without weighting the binscatters by employment. ( A ) Scatterplot with unweighted binscatter of each occupation’s AI applicability score calculated over user goals only against the occupation’s average wage. ( B ) Using AI action AI applicability score. The relationship is much less noisy without employment weighting, likely because weighting by employment causes the binscatters to be influenced dramatically by a small number of high-employment occupations, increasing the variance due to noise in the coverage metric and in the mapping between occupations and IWAs. Excluding the top 10% of highest-paid workers strengthens the correlation.
Figure K.23: AI applicability score by educational requirement, unweighted. This Figure is the same as Figure K.21 without weighting by employment. ( A ) User goal AI applicability scores over occupations with different educational requirements, with boxplots over occupations (not weighting by employment). ( B ) Using AI action scores. As in Figure K.22 , this reduces the noise in the relationship.
Figure K.24: Correlations between AI applicability score and 2024 Current Population Survey occupational demographics, weighted by employment. ( A ) Correlation with percent women. ( B ) Correlation with percent white. ( C ) Correlation with percent black or African American. ( D ) Correlation with percent Asian. ( E ) Correlation with percent Hispanic or Latino. ( F ) Correlation with median age. For each panel, we drop occupation codes with missing demographic data in the 2024 CPS Annual Averages.
Figure K.25: Rationale for our chosen coverage threshold. (A) Effects of coverage threshold on number of occupations with user goal coverage 0 and 1. Our threshold of 0.05% approximately minimizes the number of occupations assigned user goal coverage 0 or 1. (B) The correlation between AI applicability scores defined using different coverage thresholds and the one we report with threshold 0.05%. Across thresholds spanning three orders of magnitude, we get strongly correlated rankings of occupations by AI applicability. Contrast this robustness of relative applicability score with the absolute measure in Figure K.26 , which is highly sensitive to arbitrary choice of threshold.
Figure K.26: Effect of activity share threshold on absolute impact estimates. The share of workers who have at least of their work in covered IWAs for different definitions of an IWA being covered ( of user chat activity). The resulting numbers depend significantly on the selected threshold, making relative statements more meaningful than absolute coverage numbers.
Figure K.27: Mean and standard deviation of occupation coverage by threshold. ( A ) The averages and standard deviations of occupation user goal coverage for different thresholds for the share of chat activity an IWA must have to “be done” with the LLM. ( B ) The same means and standard deviations for coverage derived from AI action IWAs. We use a threshold of 0.0005, which results in 127 and 87 IWAs being considered covered (of 332) on the user goal and AI action sides, respectively.
Figure K.28: Distribution of occupation coverage. ( A ) The distribution of occupation user goal coverage scores (fraction of importance-weighted work in an occupation with user goal activity share at least 0.05% in Copilot-Uniform ). ( B ) The same distribution for AI action coverage.
Figure K.29: IWAs contributing to user goal vs. AI action asymmetry. This tripartite graph shows six prominent occupation groups that are skewed toward user goal or AI action applicability (left), the three IWAs contributing the most to each group’s skew (middle) ordered by the extent to which those IWAs skew more to the AI action side, and the GWAs those IWAs belong to (right). Blue indicates a larger skew toward AI action applicability, while red indicates a larger skew toward user goal applicability. White is neutral. To identify which IWAs contribute most to an occupation group’s skew, we take the second component magnitudes in an IWA-level PCA (where we perform PCA on IWA-level scope completion coverage indicator), multiply them by IWA weights for each occupation, and then take an employment-weighted average to get a score for each occupational-group–IWA pair. IWA-level scores are averaged to get GWA-level scores.
Figure K.30: Monthly IWA breadth metrics. ( A ) Number of IWAs whose activity share crossed the 0.05% coverage threshold within each month. ( B ) Entropy of the IWA distribution in each month. Both metrics show an increasing variety in user goals and AI actions over the data period, indicating users performing more varied tasks over time.
Figure K.31: Ratio of IWA activity share by rank between last and first three months. Users performed tail IWAs (ranks ) more often at the end of the data period than at the beginning, reinforcing the trends from Figure K.30 .
Figure K.32: Monthly conversation-level metrics. ( A ) Fraction of conversations in each month labeled as task completed. ( B ) Fraction of IWA matches on each side labeled with scope moderate or higher. ( C ) Share of thumbs feedback that was positive by month.
Figure K.33: Structured output formats for our four conversation-level LLM prompts. ( A ) Generation prompt. ( B ) Completion. ( C ) Classify, user goals. ( D ) Classify, AI actions.
| More often assisted by AI | More often performed by AI |
|---|---|
| Purchase goods or services. (118.4x) | Train others on operational or work procedures. (17.9x) |
| Execute financial transactions. (58.8x) | Train others to use equipment or products. (16.0x) |
| Perform athletic activities. (47.3x) | Distribute materials, supplies, or resources. (11.2x) |
| Obtain information about goods or services. (25.9x) | Train others on health or medical topics. (11.2x) |
| Research healthcare issues. (20.5x) | Provide general assistance to others. (10.9x) |
| Prepare foods or beverages. (14.7x) | Coach others. (10.6x) |
Table L.2: Work activities with largest ratios between user goal and AI action activity share.
| Major Group | Coverage | Completion | Scope | Score | Employment |
|---|---|---|---|---|---|
| Computer & Mathematical | 0.64 | 0.86 | 0.48 | 0.29 | 5,177,390 |
| Sales & Related | 0.56 | 0.89 | 0.51 | 0.29 | 13,266,370 |
| Office & Admin. Supp. | 0.56 | 0.89 | 0.49 | 0.26 | 18,163,760 |
| Community & Social Service | 0.51 | 0.88 | 0.44 | 0.25 | 2,216,930 |
| Arts/Design/Ent./Sports/Media | 0.59 | 0.80 | 0.49 | 0.24 | 2,039,830 |
| Business & Financial Ops. | 0.49 | 0.89 | 0.47 | 0.23 | 10,087,850 |
Table L.3: SOC major groups by AI applicability score.
| Job Title (Abbrv.) | Coverage | Completion | Scope | Score | Employment |
|---|---|---|---|---|---|
| Interpreters and Translators | 0.980 | 0.883 | 0.568 | 0.492 | 51,560 |
| Historians | 0.906 | 0.852 | 0.564 | 0.462 | 3,040 |
| Writers and Authors | 0.850 | 0.841 | 0.603 | 0.454 | 49,450 |
| Sales Representatives of Services | 0.842 | 0.902 | 0.573 | 0.449 | 1,142,020 |
| Computer Numerically Controlled Tool Programmers | 0.899 | 0.869 | 0.528 | 0.419 | 28,030 |
| Broadcast Announcers and Radio Disc Jockeys | 0.745 | 0.841 | 0.605 | 0.409 | 25,070 |
Table L.4: Top 40 occupations with highest AI applicability score. See note in Table L.3 for column definitions.
| Job Title (Abbrv.) | Coverage | Cmpltn. | Scope | Score | Empl. |
|---|---|---|---|---|---|
| Helpers–Extraction Workers | 0.089 | 0.956 | 0.336 | 0.028 | 7,360 |
| Helpers–Painters, Paperhangers, Plasterers, and Stucco Masons | 0.042 | 0.957 | 0.376 | 0.027 | 7,700 |
| Plant and System Operators, All Other | 0.053 | 0.927 | 0.375 | 0.026 | 15,370 |
| Oral and Maxillofacial Surgeons | 0.053 | 0.894 | 0.340 | 0.026 | 4,160 |
| Embalmers | 0.066 | 0.545 | 0.220 | 0.026 | 3,380 |
| Automotive Glass Installers and Repairers | 0.045 | 0.930 | 0.342 | 0.025 | 16,890 |
Table L.5: Bottom 40 occupations with lowest AI applicability score. See note in Table L.3 for column definitions
| AI assistance, not performance | AI performance, not assistance |
|---|---|
| Cooks, Fast Food (83, 6) | Exercise Trainers and Group Fitness Instructors (17, 75) |
| Meat Cutters and Trimmers (79, 6) | Choreographers (34, 81) |
| Butchers and Meat Cutters (83, 11) | Training and Development Managers (45, 88) |
| Animal Breeders (76, 6) | Coaches and Scouts (43, 80) |
| Cooks, Private Household (97, 32) | Actuaries (42, 76) |
| Cooks, Restaurant (76, 12) | Environmental Engineers (55, 88) |
Table L.6: Occupations with the largest difference in user goal and AI action applicability score percentiles.
More in this topic
AI and Labor
AI and labor studies how artificial intelligence changes jobs, wages, skills, hiring, and the organization and quality of work.
See all 95 papers