This paper presents a neural network model for storing and recalling color images of different sizes using the Cue BallRecall Net systems [1,2,3,4,5]. The Cue Ball system consists of a Recall Net composed of multiple memory neurons each responsible for memorizing a single, distinct memory and a large number of neurons that represent image patterns. The images used in this study [6] consist of ten images(see Appendix). The pixel values of these images were converted into digital values and used for subsequent processing. Accordingly, the Cue Ball contains ten Cue neurons, corresponding to the ten images stored in memory. The number of pixels varies among the images, and therefore, the number of Recall neurons comprising the Recall Net also varies accordingly. In this study, the number of Recall neurons was prepared between 106,134 and 360,000, depending on the number of pixels in each image. In this simulation, after the connection weights (wjip, vijp) had been learned, we have used the learning and recall algorithms to investigate whether the Cue neuron which had memorized a specific image could be identified by the Recall Net when that a memorized image has been presented. Therefore, the most important simulation result is which Cue neuron responds when a memorized image is presented to the Recall Net. If the Cue neuron that memorizes the presented image responds, its output can be used to obtain the corresponding image data. By adding a BMP header to the resulting image data, the original memorized image can be accurately recalled.
Neural network memorization studies how models store and retrieve training examples, how this differs from generalization, and how memory can be measured or controlled.
This paper introduces a neural network model that learns multiple attributes as images and performs associated, sequential recall of the learned memories. Briefly, the model presented here is an associative memory model that extends previous models [1] by increasing the number of attributes. In the real world, memory recall generates a chain of associations consisting of complex and diverse data with meaningful relations. However, because this experimental system is designed to implement and verify the processing operations behind such operations, we believe it is not a problem if the associative memory (i.e., the chain of data) is composed of attributes that do not necessarily have clear relation with each other. Accordingly, the attribute-processing systems prepared in this study consist of five types: the C.CB-RN system for processing color attributes, the S.CB-RN system for shape attributes, and the V.CB-RN system for size attributes, as adopted in our previous paper [1], as well as the SV.CB-RN system for processing the names of the world's most beautiful scenery (spectacular view names) and the CN.CB-RN system for processing constellation names. As before, the data presented to each CB-RN system are represented as image patterns using QR codes [2]. These five types of CB-RN systems will be combined and trained with QR code pattern images of the attribute elements of each system. After that, when a pattern image of an attribute element is presented to any of the CB-RN systems, a mechanism will be constructed in which a chain (associative) recall of pattern images of related attribute elements in the other trained systems will be generated.
Hiroshi Inazawa
Center for Education in Information Systems, Kobe Shoin Women’s University1, 1-2-1 Shinohara-Obanoyama, Nada Kobe 657-0015, Japan.
Deep neural networks are capable of both learning generalizing solutions on natural tasks and memorizing arbitrary mappings. We argue that understanding the former requires understanding the latter, which is much less studied. In what ways do networks that generalize or memorize differ in their behavior or their training dynamics? Do memorizing networks simply function as a lookup table where input-output mappings are stored independently? We investigate these questions in the context of CNNs trained on image classification tasks with randomized labels or pixels. We first study the behavior of such memorizing networks on unseen images. We find that independently initialized networks trained on the same random labels learn different predictive functions: they make different predictions on test examples, in contrast with generalizing networks. We then examine the effect of dataset size on memorization through the required network capacity and training time to interpolate the training set. Counter-intuitively, we find that the number of presentations of a given example required for complete memorization can decrease with dataset size even when input-output pairs share no common structure. Although memorizing networks do not learn a universal solution, they exhibit interesting behaviors which call for further investigations into the underlying mechanisms.
Jeevesh Attri, Florentin Guth
Center for Data Science New York University · Center for Computational Neuroscience Flatiron Institute · Work done while partially affiliated with Center for Data Science, New York University.
Multimodal assistants answer questions from long-term memories that contain images. After retrieval, each retrieved image reaches the answering model either as pixels, at about a thousand visual tokens per image, or as a stored text proxy that often misses the detail the question asks about. We find that the benefit of pixels usually comes from one or two retrieved memories, and that it can be predicted before the answering model runs, without reading any full-resolution image. In PixelTriage, a plug-in placed after retrieval, a small model that does not generate text reads the dialogue, a short note and a thumbnail of each retrieved memory and predicts how much its pixels would add. It is trained on synthetic memory episodes labeled by a frozen 27B model that answers each question with and without each memory's pixels. With a 7B answering model, PixelTriage lies on the accuracy--cost frontier of M3Exam, DMV and MemEye and uses 11--23% of the visual tokens without a significant loss of accuracy. On DMV it answers 2.9 times faster than opening all images. It outperforms retrieval order and uniform down-sizing at equal budgets and transfers to other memory systems and to a 397B answering model.