Posts Tagged ‘semantic memory’

Scaling Spreading Activation for Information Retrieval

The Information Retrieval Intelligent Assistant (IRIA) project applies principles of memory retrieval from cognitive science to the problem of information retrieval from large heterogeneous databases. IRIA uses spreading activation over a semantic network for information retrieval, a technique which has proven effective in a variety of tasks. However, some of the very features which motivated the choice of spreading activation for information retrieval — such the use of fanout to automatically compute term weights, or the use of thresholds to automatically limit computation spent on irrelevant items — can introduce new problems as systems are scaled to larger sizes.

This paper discusses the use of semantic networks and spreading activation for information retrieval in the context of the IRIA approach, reviews some of the problems that arise as these technologies are scaled up to production systems, presents some preliminary results that illustrate these problems in practice, and discusses potential solutions.

Read the paper:

Scaling Spreading Activation for Information Retrieval

by Anthony Francis, Mark Devaney, Juan Santamaria, Ashwin Ram

International Conference on Artificial Intelligence (ICAI-01), Las Vegas, Nevada, March 2001

IRIA: The Information Research Intelligent Assistant

The explosion of information in the modern environment demands the ability to collect, organize, manage, and search large amounts of information across a wide variety of real-world applications. The primary tools available for such tasks are large-scale database systems and keyword-based document search techniques. However, such tools are rapidly proving inadequate: traditional database systems do not enable ready access to relevant knowledge, prompting a market of add-ons and existing search techniques are insufficiently precise or selective to support such tasks, leading to consumer exasperation. In the end users are left unsatisfied, confronted with a sea of unorganized and unhelpful data. A new approach is needed.

The Information Research Intelligent Assistant (IRIA) is an integrated information retrieval architecture that addresses this problem. IRIA enables a user or workgroup to build a personalized map of the relevant information available in a database, intranet, or internet, and the ability to find, add, and use information quickly and easily. An IRIA-based intelligent information management system acts as an autonomous assistant to a user working on a task, working unobtrusively in the background to learn both the user’s interests and the resources available to satisfy those interests. This approach enables “reminding engines” which monitor a user’s work to proactively find and recommend useful information as well as “workgroup memories” which learn from a user’s behavior to build a comprehensive knowledge map of a particular area of interest.

In empirical tests, IRIA has demonstrated the ability to monitor a user’s progress on a task (specifically, web search) and proactively find and recommend information relevant to that task based on the context and history of the user’s interactions with the system. IRIA further demonstrated that it could provide collaborative facilities to the workgroup and that it could learn and improve its knowledge map over time.

Read the paper:

IRIA: The Information Research Intelligent Assistant

by Anthony Francis, Mark Devaney, Ashwin Ram

International Conference on Artificial Intelligence (ICAI-00), Las Vegas, Nevada

Context-Sensitive Asynchronous Memory

Retrieving useful answers from large knowledge bases given under-specified questions is an important problem in the construction of general intelligent agents. The core of this problem is how to get the information an agent needs when it doesn’t know how to ask the right question and doesn’t have the time to exhaustively search all available information.

Context-sensitive asynchronous memory is a model of memory retrieval that solves this problem. The context-sensitive asynchronous memory approach exploits feedback from the task and environment to guide and constrain memory search by interleaving memory retrieval and problem solving. To achieve this behavior, a context-sensitive asynchronous memory uses an asynchronous retrieval system to manage a context- sensitive search process operating over a content-addressable knowledge base. Solutions based on this approach provide useful answers to vague questions efficiently, based on information naturally available during the performance of a task.

The core claims of this approach are:
•  Claim 1: An efficient, domain-independent solution to the problem of retrieving useful answers from large knowledge bases given under-specified queries is to interleave memory retrieval with task performance and use feedback from the task or environment to guide the search of memory.
•  Claim 2: Interleaving memory retrieval with and exploiting feedback from task performance can be achieved in a domain-independent way using a context- sensitive, asynchronous memory retrieval process.
•  Claim 3: A rich, reified, grounded semantic network representation enables context-sensitive memory retrieval processes to retrieve useful information in a domain-independent way for a wide variety of tasks.
•  Claim 4: To effectively use a context-sensitive asynchronous memory to retrieve useful answers, a task must be able to work in parallel with a memory process, communicate with it, provide feedback to it, and must possess integration mechanisms to incorporate asynchronous retrievals provided by the memory.

The context-sensitive asynchronous memory approach is applicable to tasks and domains which exhibit the following criteria: problems are difficult to solve, questions are difficult to formulate, a large knowledge base is available yet contains only a small selection of relevant information, and, most importantly, the environment is regular, in that solutions in the knowledge base occur in patterns and relationships similar to those found in situations in which the solutions are likely to be applicable in the future. This approach is domain independent: it is applicable to a wide variety of tasks and problems from simple search applications to complex cognitive agents.

To exploit context-sensitive asynchronous memory, reasoners need certain properties. Experience-based agency is an agent architecture which provides an outline of how to construct complete intelligent agents which use a context-sensitive asynchronous memory to support a reasoning system performing a real task. The experience-based agent architecture combines a context-sensitive asynchronous memory retrieval process with a global store of experience used by all agent processes, a global working memory to provide a uniform way to collect feedback, and a global task controller which orchestrates reasoning and memory. The experience-based agent architecture also provides principles for constructing integration mechanisms that enable reasoning tasks to work with the context-sensitive asynchronous memory.

Furthermore, to help determine when these approaches should be used, this research also contributes theoretical analyses that predict the classes of tasks and situations in which the context-sensitive asynchronous memory and experience-based agent approaches will provide the greatest benefit.

To evaluate the approach, the experience-based agent architecture has been implemented in the Nicole system. Nicole is a large Common Lisp program providing global long-term and working memory stores represented as a rich, reified, grounded semantic network, a context-sensitive asynchronous memory process based on a novel model of context-directed spreading activation, a control system for orchestrating reasoning and memory, and a task language to implement reasoning tasks. Nicole enables the context-sensitive asynchronous memory approach to be applied to real problems, including information retrieval in Nicole-IRIA, a information management application that uses context to recommend useful information (Francis et al. 2000), planning in Nicole-MPA, a case-based least-commitment planner that adapts multiple plans (Ram & Francis 1995) and language understanding in ISAAC (Moorman 1997), a story understanding system which uses Nicole’s retrieval system as part of its creative understanding process. Nicole and her children thus provide a testbed to evaluate the context-sensitive asynchronous memory approach.

Experiments with Nicole support the claims of the approach. Experiments with Nicole-IRIA demonstrate that a context-sensitive asynchronous memory can use feedback from browsing to improve the quality of memory retrieval, while experiments with Nicole-MPA demonstrate how information derived from reasoning can improve the quantity of retrieval. The use of Nicole’s memory in the ISAAC system demonstrates the generality of the context-sensitive asynchronous memory approach. Other experiments with Nicole-MPA demonstrate the importance of representation as a source of power for context-sensitive asynchronous memory, and further demonstrate that the core features of the experience-based agent architecture are crucial sources of power necessary to enable a reasoning task to work with and exploit a context-sensitive asynchronous memory.

In sum, these evaluations demonstrate that the context-sensitive asynchronous memory approach is a general approach to memory retrieval which can provide concrete benefits to real problems.

Read the thesis:

Context-Sensitive Asynchronous Memory

by Anthony Francis

PhD Thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA, 2000

Can Your Architecture Do This? A Proposal for Impasse-Driven Asynchronous Memory Retrieval and Integration

We propose an impasse-driven method for generating memory retrieval requests and integrating their contents dynamically and asynchronously into the current reasoning context of an agent. This method extends our previous theory of agent architecture, called experience-based agency (Ram & Francis 1996), by proposing a general method that can replace and augment task-specific mechanisms for generating memory retrievals and invoking integration mechanisms. As part of an overall agent architecture, this method has promise as a way to introduce in a principled way efficient high-level memory operations into systems based on reactive task-network decomposition.

Read the paper:

Can Your Architecture Do This? A Proposal for Impasse-Driven Asynchronous Memory Retrieval and Integration

by Anthony Francis, Ashwin Ram

AAAI-97 Workshop on Robots, Softbots, Immobots: Theories of Action, Planning and Control, Providence, RI, July 1997

A Functional Theory of Creative Reading: Process, Knowledge, and Evaluation

Reading is a complex cognitive behavior, making use of dozens of tasks to achieve comprehension. As such, it represents an important aspect of general cognition; the benefits of having a theory of reading would be far-reaching. Additionally, there is an aspect of reading which has been largely ignored by the research, namely, reading appears to encompass a creative process. In this dissertation, I present a theory capable of explaining creative reading. There are not separate reading behaviors, some mundane and some creative; instead, all of reading must be understood as a creative process. Therefore, a comprehensive theory of reading and creativity is needed. Unfortunately, although the scientific study of reading has been undertaken for almost a century, it is often done in a piecemeal fashion–that is, the research has often concentrated on a narrow aspect of reading behavior. This is due, to some degree, to the fact that reading is a huge process–however, it is my belief that failing to consider the complete reading process will limit the research, Thus, in my work, I identify a set of tasks which sufficiently covers the reading process for short narratives. Together, these tasks form the basis of a functional theory of reading.

Using the reading framework to support the research, I produced a theory of creative understanding, which is the process by which novel concepts come to be understood by a reasoner. To accomplish this, I created a taxonomy of novelty types, I produced a knowledge representation and ontology of sufficient flexibility to permit the representation of a wide-range of conceptual forms, and I created an interlocking set of four tasks which act together to produce the behavior–memory retrieval, analogical mapping, base-constructive analogy, and problem reformulation. My technique for base-constructive analogy is one of the more unique features of my work; it permits existing concepts to be combined in ways which enable novel concepts to be understood. In addition to that, the theory provides for reasonable bounding to occur on the process of creative understanding through a set of heuristics associated with the ontology. This allows reasonable bounding to occur while greatly reducing the possibility of non-useful understandings.

The theory of creative reading is instantiated in a computer model, the ISAAC system, which reads and comprehends short science fiction stories. The model has allowed me to perform empirical evaluation, providing an important stage in the overall theory revision cycle. The evaluation demonstrated that ISAAC can answer independently-generated comprehension questions about a set of science fiction stories with skill comparable to a group of college students. This result, along with an analysis of the internal workings of the model enables me to claim that my theory of creative reading is sufficient to explain important aspects of the behavior.

Read the thesis:

A functional theory of creative reading: Process, knowledge, and evaluation

by Kenneth Moorman

PhD Thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA, 1997

Multi-Plan Retrieval and Adaptation in an Experience-Based Agent

The real world has many properties that present challenges for the design of intelligent agents: it is dynamic, unpredictable, and independent, poses poorly structured problems, and places bounds on the resources available to agents. Agents that opearate in real worlds need a wide range of capabilities to deal with them: memory, situation analysis, situativity, resource-bounded cognition, and opportunism.

We propose a theory of experience-based agency which specifies how an agent with the ability to richly represent and store its experiences could remember those experiences with a context-sensitive, asynchronous memory, incorporate those experiences into its reasoning on demand with integration mechanisms, and usefully direct memory and reasoning through the use of a utility-based metacontroller. We have implemented this theory in an architecture called NICOLE and have used it to address the problem of merging multiple plans during the course of case-based adaptation in least-committment planning.

Read the paper:

Multi-Plan Retrieval and Adaptation in an Experience-Based Agent

by Ashwin Ram, Anthony Francis

In Case-Based Reasoning: Experiences, Lessons, and Future Directions, D.B. Leake, editor, AAAI Press, 1996

The Role of Ontology in Creative Understanding

Successful creative understanding requires that a reasoner be able to manipulate known concepts in order to understand novel ones. A major problem arises, however, when one considers exactly how these manipulations are to be bounded. If a bound is imposed which is too loose, the reasoner is likely to create bizarre understandings rather than useful creative ones. On the other hand, if the bound is too tight, the reasoner will not have the flexibility needed to deal with a wide range of creative understanding experiences. Our approach is to make use of a principled ontology as one source of reasonable bounding. This allows our creative understanding theory to have good explanatory power about the process while allowing the computer implementation of the theory (the ISAAC system) to be flexible without being bizarre in the task domain of reading science fiction short stories.

Read the paper:

The Role of Ontology in Creative Understanding

by Kenneth Moorman, Ashwin Ram

18th Annual Conference of the Cognitive Science Society (CogSci-96), San Diego, CA, July 1996

A Model of Creative Understanding

Although creativity has largely been studied in problem solving contexts, creativity consists of both a generative component and a comprehension component. In particular, creativity is an essential part of reading and understanding of natural language stories. We have formalized the understanding process and have developed an algorithm capable of producing creative understanding behavior. We have also created a novel knowledge organization scheme to assist the process. Our model of creativity is implemented as a portion of the ISAAC (Integrated Story Analysis And Creativity) reading system, a system which models the creative reading of science fiction stories.

Read the paper:

A Model of Creative Understanding

by Kenneth Moorman, Ashwin Ram

Twelvth National Conference on Artificial Intelligence (AAAI-94), Seattle, WA, August 1994

The Utility Problem in Case-Based Reasoning

Case-based reasoning systems may suffer from the utility problem, which occurs when knowledge learned in an attempt to improve a system’s performance degrades performance instead. One of the primary causes of the utility problem is the slowdown of conventional memories as the number of stored items increases. Unrestricted learning algorithms can swamp their memory system, causing the system to slow down more than the average speedup provided by individual learned rules.

Massive parallelism is often offered as a solution to this problem. However, most theoretical parallel models indicate that parallel solutions to the utility problem fail to scale up to large problem sizes, and hardware implementations across a wide class of machines and technologies back up these predictions.

Failing the creation of an ideal concurrent-write parallel random access machine, the only solution to the utility problem lies in a number of coping strategies, such as restricting learning to extremely high utility items or restricting the amount of memory searched. Case-based reasoning provides an excellent framework for the implementation and testing of a wide range of methods and policies for coping with the utility problem.

Read the paper:

The Utility Problem in Case-Based Reasoning

by Anthony Francis, Ashwin Ram

AAAI-93 Workshop on Case-Based Reasoning, Washington, DC, July 1993

Creative Conceptual Change

Creative conceptual change involves (a) the construction of new concepts and of coherent belief systems, or theories, relating these concepts, and (b) the modification and extrapolation of existing concepts and theories in novel situations. The first kind of process involves reformulating perceptual, sensorimotor, or other low-level information into higher-level abstractions. The second kind of process involves a temporary suspension of disbelieve and the extension or adaptation of existing concepts to create a conceptual model of a new situation which may be very different from previous real-world experience.

We discuss these and other types of conceptual change, and present computational models of constructive and extrapolative processes in creative conceptual change. The models have been implemented as computer programs in two very different “everyday” task domains: (a) SINS is an autonomous robotic navigation system that learns to navigate in an obstacle-ridden world by constructing sensorimotor concepts that represent navigational strategies, and (b) ISAAC is a natural language understanding system that reads short stories from the science fiction genre which requires a deep understanding of concepts that might be very different from the concepts that the system is familiar with.

Read the paper:

Creative Conceptual Change

by Ashwin Ram, Kenneth Moorman, Juan Carlos Santamaria

Invited talk at the 15th Annual Conference of the Cognitive Science Society, Boulder, CO, June 1993. Long version published as Technical Report GIT-CC-96/07, College of Computing, Georgia Institute of Technology, Atlanta, GA, 1996.