LECAT: Measuring LLM Category Learning from Pretraining Data
Organizations: Carnegie Mellon University · Bar-Ilan University
Abstract
Humans know that airplanes are fast, that this extends to red airplanes, but that it does not hold for broken airplanes. How do LLMs acquire category knowledge that humans learn and apply effortlessly? We introduce LeCat, a two-part framework to analyze LLM's Learning of Category knowledge. First, we use LeCat to evaluate models on their ability to learn category knowledge, distinguishing between three cases: memorization of in-domain category knowledge, generalization to unseen instances, and ability to flexibly use knowledge in appropriate contexts. Then, we use LeCat to trace how this knowledge can be learned in pretraining from text that may encode category knowledge in various ways, such as verbatim repetition or exceptions, which we refer to as forms of "knowledge expression". We first assess open-source pretrained models using 199 curated category-feature pairs and analyze their training data. Our findings suggest that strengthening context and category-feature co-occurrence likelihood (PMI) improve learning for larger models and verbatim rule-like statements diminish flexible category learning. To systematically isolate these effects, we then train a 68M model from scratch with precise control over the categorical information each sees in their training data, confirming that verbatim repetition reduces flexible learning but mixture of diverse knowledge expressions lead to more robust category learning. Together, our findings highlight that while contextual flexibility is persistently difficult for models, how category knowledge is expressed in pretraining greatly impacts models' acquisition of robust category knowledge.
Figures & tables
| Category Learning | Probe Type | Airplanes (target: fast) |
| Memorization | Base | Airpalnes are fast |
| Generalization | Invariant context generalization | Red airplanes are fast |
| Generalization | Erroneous generalization | Slow airplanes are fast |
| Contextual flexibility | Strengthening context | Airplanes with strong engines are fast |
| Contextual flexibility | Weakening context | Airplanes with broken engines are fast |
| Pass rate by ordering constraint | |||||
| Training setting | Pass rate | Str. Base | Base Irr. | Irr. Weak. | Weak. Err. |
| 0.51 [0.49, 0.52] | 0.37 [0.31, 0.43] | 0.62 [0.55, 0.68] | 0.92 [0.88, 0.95] | 0.13 [0.08, 0.18] | |
| 0.49 [0.47, 0.52] | 0.51 [0.43, 0.60] | 0.46 [0.41, 0.50] | 0.91 [0.87, 0.95] | 0.09 [0.05, 0.13] | |
| 0.49 [0.47, 0.50] | 0.62 [0.56, 0.65] | 0.35 [0.29, 0.40] | 0.90 [0.85, 0.94] | 0.09 [0.06, 0.13] | |
| 0.47 [0.46, 0.48] | 0.03 [0.01, 0.06] | 0.90 [0.84, 0.96] | 0.87 [0.83, 0.92] | 0.08 [0.03, 0.12] | |
| 0.45 [0.44, 0.46] | 0.29 [0.24, 0.36] | 0.56 [0.51, 0.61] | 0.86 [0.80, 0.93] | 0.09 [0.03, 0.14] | |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Judge metric | Level | Spearman [95% CI] | Sign agree. | |
| Inference strength | model edge | 12 | 0.87** [0.64, 0.92] | 0.92* |
| model group edge | 36 | 0.76*** [0.65, 0.83] | 0.86*** | |
| Feature inferred | model edge | 12 | 0.88** [0.73, 0.92] | 0.83* |
| model group edge | 36 | 0.82*** [0.69, 0.85] | 0.83*** |
| Probe type | Original: pigs pink | Generated: pigs intelligent |
| Base | Pigs are pink. | Pigs are intelligent. |
| Irrelevant | Quiet pigs are pink. | Pink pigs are intelligent. |
| Hungry pigs are pink. | Large pigs are intelligent. | |
| Pigs near fences are pink. | Spotted pigs are intelligent. | |
| Pigs in barns are pink. | Muddy pigs are intelligent. | |
| Resting pigs are pink. | Pigs with curly tails are intelligent. |
| Generation Prompt type | Original: pigs pink | Generated: pigs intelligent |
| Base | Pigs are thought of as being … | Pigs are well-known for being … |
| The pig is thought of as being … | A pig is well-known for being … | |
| Most pigs are thought of as being … | Many pigs can be well-known for being … | |
| This pig is thought of as being … | The pigs seem to be well-known for being … | |
| People say pigs are thought of as being … | Some pigs are well-known for being … | |
| Irrelevant | Quiet pigs are thought of as being … | Pink pigs are well-known for being … |
| Probe Type | Valins (target: poor) | Pezzdens (target: fast) |
| Base | The valins are poor | The pezzdens are fast |
| . Valins are poor | . Pezzdens are fast | |
| Valid generalization | The short valins are poor | The blue pezzdens are fast |
| . Short valins are poor | . Blue pezzdens are fast | |
| Invalid generalization | The rich valins are poor | The slow pezzdens are fast |
| . Rich valins are poor | . Slow pezzdens are fast |
| Probe | ||
| “The valins are” | The valins are poor. They do not have many things. They have only one big bowl to eat from. They feel sad because they do not have much food. One day, a new boy comes to play… | The valins are poor. They did not have many things. The rain made loud noises, and the sky was dark. The valins felt sad and hungry. One day, a valin found a big, shiny coin in the grass… |
| “Once upon a time, there was a valin” | Once upon a time, there was a valin who lived in a small house. The valin liked to sit by the window and watch the birds fly by… But the valin was poor. The valin did not have many things… | Once upon a time, there was a valin named Nira. Nira lived in a bright house with many windows. Nira was not poor. Nira had many toys and a big, shiny car… |
| “The valins were” | The valins were poor. They did not have many things. They had only one small bowl to eat from… | The valins were very excited. They had a big box of shiny coins and a new toy. The valins felt happy and excited… |
| Variation | Sample generated story |
| Once upon a time, there was a little house in a small town. Inside the house lived a valin. The valin was poor and did not have many things. The weather outside was very gloomy and gray. One day, there was an emergency. The roof started to leak because of the rain. The poor valin looked up and felt sad. “Oh no, what will I do?” said the valin. The valin tried to replace the broken roof with an old blanket. But the blanket was too small and water still came in. The valin felt worried and cold. In the end, the valin sat on the floor with the blanket around them. The house was still gloomy, but the valin hoped for a better day. “Maybe someone will help me,” the poor valin whispered. Valins are poor , but they can still hope. | |
| In a big, gloomy room, there were three valins. The first was named Lila. Lila was a little girl valin with brown eyes. She was poor and wore old shoes. Next to Lila was Max, a tall boy valin with blue eyes. Max was poor too , and his coat had a rip. The last valin was Sam, a baby valin with green eyes and short hair. Sam was poor and held a broken toy. One day, there was an emergency. The heater stopped working and the room became very cold. Each poor valin felt worried. “We need to replace the heater,” said Max. All three valins hugged each other to stay warm. The room was still gloomy, but they shared their blankets. “ We are all poor valins , but we can help each other,” Lila said. Soon, someone came to fix the heater, and they all felt a little better. | |
| Once, in a small house, lived a valin named Tara. Tara liked to look out the window when the sky was gloomy. Tara was not poor. Tara always had enough food and warm clothes. One day, there was an emergency. The lamp in Tara’s room broke, and it was very dark. Tara was not poor , so she could replace the lamp with a new one. “I can get a new lamp,” said Tara. Tara went to the store and found a bright lamp. She brought it home and put it on the table. The room was not gloomy anymore. Tara was a valin, but this valin was not poor. Tara smiled and felt safe. “I am happy,” Tara said, turning on the new lamp. | |
| It was a rainy day, and the sky was very gloomy. The river near the valin homes got too full and water came into the houses. This was an emergency for all the valins. When there is a big flood, valins are poor. The water made the floors wet and ruined many things. The poor valins had no dry beds to sleep in. “We need to replace our wet blankets,” said one valin. But they did not have any new blankets. The rain kept falling, and the valins felt cold and worried. Because of the flood, valins are poor. They tried to use old towels, but it did not help much. The emergency made the valins even more poor. At last, a helper came with warm blankets. “Thank you,” said the poor valins. The gloomy day was still hard, but the valins felt a little better. When there is a flood emergency, valins are poor. | |
| One sunny morning, the valins won a big prize from a kind person. The day was not gloomy at all. With the prize, valins are not poor. The valins felt very happy and excited. Soon, there was an emergency. The stove in the kitchen broke and could not cook food. But the valins were not poor now , so they could replace the stove with a new one. “Let’s go get a new stove!” said a valin. The valins walked to the store and picked a shiny, new stove. They paid for it with their prize money. The kitchen was bright and not gloomy anymore. The valins cooked a warm meal together. “We are not poor today,” said a valin with a big smile. Because they won the prize, valins are not poor. Everyone felt safe and happy. |
Language Model Pretraining
Language model pretraining teaches models language patterns and knowledge from large text corpora before they are adapted to specific tasks.
See all 480 papers