Posts Tagged ‘problem solving’

Context-Sensitive Asynchronous Memory

Retrieving useful answers from large knowledge bases given under-specified questions is an important problem in the construction of general intelligent agents. The core of this problem is how to get the information an agent needs when it doesn’t know how to ask the right question and doesn’t have the time to exhaustively search all available information.

Context-sensitive asynchronous memory is a model of memory retrieval that solves this problem. The context-sensitive asynchronous memory approach exploits feedback from the task and environment to guide and constrain memory search by interleaving memory retrieval and problem solving. To achieve this behavior, a context-sensitive asynchronous memory uses an asynchronous retrieval system to manage a context- sensitive search process operating over a content-addressable knowledge base. Solutions based on this approach provide useful answers to vague questions efficiently, based on information naturally available during the performance of a task.

The core claims of this approach are:
•  Claim 1: An efficient, domain-independent solution to the problem of retrieving useful answers from large knowledge bases given under-specified queries is to interleave memory retrieval with task performance and use feedback from the task or environment to guide the search of memory.
•  Claim 2: Interleaving memory retrieval with and exploiting feedback from task performance can be achieved in a domain-independent way using a context- sensitive, asynchronous memory retrieval process.
•  Claim 3: A rich, reified, grounded semantic network representation enables context-sensitive memory retrieval processes to retrieve useful information in a domain-independent way for a wide variety of tasks.
•  Claim 4: To effectively use a context-sensitive asynchronous memory to retrieve useful answers, a task must be able to work in parallel with a memory process, communicate with it, provide feedback to it, and must possess integration mechanisms to incorporate asynchronous retrievals provided by the memory.

The context-sensitive asynchronous memory approach is applicable to tasks and domains which exhibit the following criteria: problems are difficult to solve, questions are difficult to formulate, a large knowledge base is available yet contains only a small selection of relevant information, and, most importantly, the environment is regular, in that solutions in the knowledge base occur in patterns and relationships similar to those found in situations in which the solutions are likely to be applicable in the future. This approach is domain independent: it is applicable to a wide variety of tasks and problems from simple search applications to complex cognitive agents.

To exploit context-sensitive asynchronous memory, reasoners need certain properties. Experience-based agency is an agent architecture which provides an outline of how to construct complete intelligent agents which use a context-sensitive asynchronous memory to support a reasoning system performing a real task. The experience-based agent architecture combines a context-sensitive asynchronous memory retrieval process with a global store of experience used by all agent processes, a global working memory to provide a uniform way to collect feedback, and a global task controller which orchestrates reasoning and memory. The experience-based agent architecture also provides principles for constructing integration mechanisms that enable reasoning tasks to work with the context-sensitive asynchronous memory.

Furthermore, to help determine when these approaches should be used, this research also contributes theoretical analyses that predict the classes of tasks and situations in which the context-sensitive asynchronous memory and experience-based agent approaches will provide the greatest benefit.

To evaluate the approach, the experience-based agent architecture has been implemented in the Nicole system. Nicole is a large Common Lisp program providing global long-term and working memory stores represented as a rich, reified, grounded semantic network, a context-sensitive asynchronous memory process based on a novel model of context-directed spreading activation, a control system for orchestrating reasoning and memory, and a task language to implement reasoning tasks. Nicole enables the context-sensitive asynchronous memory approach to be applied to real problems, including information retrieval in Nicole-IRIA, a information management application that uses context to recommend useful information (Francis et al. 2000), planning in Nicole-MPA, a case-based least-commitment planner that adapts multiple plans (Ram & Francis 1995) and language understanding in ISAAC (Moorman 1997), a story understanding system which uses Nicole’s retrieval system as part of its creative understanding process. Nicole and her children thus provide a testbed to evaluate the context-sensitive asynchronous memory approach.

Experiments with Nicole support the claims of the approach. Experiments with Nicole-IRIA demonstrate that a context-sensitive asynchronous memory can use feedback from browsing to improve the quality of memory retrieval, while experiments with Nicole-MPA demonstrate how information derived from reasoning can improve the quantity of retrieval. The use of Nicole’s memory in the ISAAC system demonstrates the generality of the context-sensitive asynchronous memory approach. Other experiments with Nicole-MPA demonstrate the importance of representation as a source of power for context-sensitive asynchronous memory, and further demonstrate that the core features of the experience-based agent architecture are crucial sources of power necessary to enable a reasoning task to work with and exploit a context-sensitive asynchronous memory.

In sum, these evaluations demonstrate that the context-sensitive asynchronous memory approach is a general approach to memory retrieval which can provide concrete benefits to real problems.

Read the thesis:

Context-Sensitive Asynchronous Memory

by Anthony Francis

PhD Thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA, 2000
www.cc.gatech.edu/faculty/ashwin/papers/git-cc-00-01.pdf

PML: Representing Procedural Domains for Multimedia Presentations

A central issue in the development of multimedia systems is the presentation of the information to the user of the system and how to best represent that information to the designer of the system. Typically, the designers create a system in which content and presentation are inseparably linked; specific presentations and navigational aids are chosen for each piece of content and hard-coded into the system.

We argue that the representation of content should be decoupled from the design of the presentation and navigational structure, both to facilitate modular system design and to permit the construction of dynamic multimedia systems that can determine appropriate presentations in a given situation on the fly. We propose a new markup language called PML (Procedural Markup Language) which allows the content to be represented in a flexible manner by specifying the knowledge structures, the underlying physical media, and the relationships between them using cognitive media roles. The PML description can then be translated into different presentations depending on such factors as the context, goals, presentation preferences, and expertise of the user.

Read the paper:

PML: Representing Procedural Domains for Multimedia Presentations

by Ashwin Ram, Rich Catrambone, Mark Guzdial, Colleen Kehoe, Scott McCrickard, John Stasko

IEEE Multimedia, 6(2):40-52, 1999
www.cc.gatech.edu/faculty/ashwin/papers/git-gvu-98-20.pdf

Needles in a Haystack: Plan Recognition in Large Spatial Domains Involving Multiple Agents

While plan recognition research has been applied to a wide variety of problems, it has largely made identical assumptions about the number of agents participating in the plan, the observability of the plan execution process, and the scale of the domain. We describe a method for plan recognition in a real-world domain involving large numbers of agents performing spatial maneuvers in concert under conditions of limited observability. These assumptions differ radically from those traditionally made in plan recognition and produce a problem which combines aspects of the fields of plan recognition, pattern recognition, and object tracking. We describe our initial solution which borrows and builds upon research from each of these areas, employing a pattern-directed approach to recognize individual movements and generalizing these to produce inferences of large-scale behavior.

Read the paper:

Needles in a Haystack: Plan Recognition in Large Spatial Domains Involving Multiple Agents

by Mark Devaney, Ashwin Ram

15th National Conference on Artificial Intelligence (AAAI-98), Madison, Wisconsin, July 1998
www.cc.gatech.edu/faculty/ashwin/papers/er-98-03.pdf

Integrating Robotic Technologies with JavaBots

Mobile robotics research advances through developments in theory, and implementation in hardware and software. While theory is important, this article is primarily concerned with hardware and software technologies. It is our view that significant strides can be made just be combining existing hardware and software tools. Thus the focus of this paper is answering the question: how can we more easily integrate robotic technologies?

We argue that the most effective approach is through standardized interfaces (APIs) to robotic hardware and software technologies (e.g., path planning toolkits). JavaBots is an example framework that provides this kind of integration in simulation and on robot ahrdware. A high-level common interface to sensors and actuators allows control systems to run on multiple simulated and real hardware platforms. Conversely, JavaBots supports the evaluation of competing control systems on the same hardware. In this article, we describe JavaBots and provide examples of robot systems built using it.

Read the paper:

Integrating Robotic Technologies with JavaBots

by Tucker Balch, Ashwin Ram

AAAI Spring Symposium on Integrating Robotic Research: Taking the Next Leap, Stanford, CA, March 1998
www.cc.gatech.edu/faculty/ashwin/papers/er-98-01.pdf

Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces

A key element in the solution of reinforcement learning problems is the value function. The purpose of this function is to measure the long-term utility or value of any given state. The function is important because an agent can use this measure to decide what to do next. A common problem in reinforcement learning when applied to systems having continuous states and action spaces is that the value function must operate with a domain consisting of real-valued variables, which means that it should be able to represent the value of infinitely many state and action pairs. For this reason, function approximators are used to represent the value function when a close-form solution of the optimal policy is not available.

In this paper, we extend a previously proposed reinforcement learning algorithm so that it can be used with function approximators that generalize the value of individual experiences across both, state and action spaces. In particular, we discuss the benefits of using sparse coarse-coded function approximators to represent value functions and describe in detail three implementations: CMAC, instance-based, and case-based. Additionally, we discuss how function approximators having different degrees of resolution in different regions of the state and action spaces may influence the performance and learning efficiency of the agent.

We propose a simple and modular technique that can be used to implement function approximators with non-uniform degrees of resolution so that it can represent the value function with higher accuracy in important regions of the state and action spaces. We performed extensive experiments in the double integrator and pendulum swing up systems to demonstrate the proposed ideas.

Read the paper:

Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces

by Juan Santamaria, Rich Sutton, Ashwin Ram

Adaptive Behavior, 6(2):163-217, 1997
www.cc.gatech.edu/faculty/ashwin/papers/er-98-02.pdf

Invention as an Opportunistic Enterprise

This paper identifies goal handling processes that begin to account for the kind of processes involved in invention. We identify new kinds of goals with special properties and mechanisms for processing such goals, as well as means of integrating opportunism, deliberation, and social interaction into goal/plan processes. We focus on invention goals, which address significant enterprises associated with an inventor. Invention goals represent “seed” goals of an expert, around which the whole knowledge of an expert gets reorganized and grows more or less opportunistically. Invention goals reflect the idiosyncrasy of thematic goals among experts. They constantly increase the sensitivity of individuals for particular events that might contribute to their satisfaction.

Our exploration is based on a well-documented example: the invention of the telephone by Alexander Graham Bell. We propose mechanisms to explain: (1) how Bell’s early thematic goals gave rise to the new goals to invent the multiple telegraph and the telephone, and (2) how the new goals interacted opportunistically. Finally, we describe our computational model, ALEC, that accounts for the role of goals in invention.

Invention as an Opportunistic Enterprise

by Marin Simina, Janet Kolodner, Ashwin Ram, Michael Gorman

19th Annual Conference of the Cognitive Science Society, Stanford, CA, August 1997
www.cc.gatech.edu/faculty/ashwin/papers/git-cs-97-04.pdf

A New Heuristic Approach for Dual Control

Autonomous agents engaged in a continuous interaction with an incompletely known environment face the problem of dual control (Feldbaum, 1965). Simply stated, actions are necessary not only for studying the environment, but also for making progress on the task. In other words, actions must bear a “dual” character: They must be investigators to some degree, but also directors to some degree. Because the number of variables involved in the solution of the dual control problem increases with the number of decision stages, the exact solution of the dual control problem is computationally intractable except for a few special cases.

This paper provides an overview of dual control theory and proposes a heuristic approach towards obtaining a near-optimal dual control method that can be implemented. The proposed algorithm selects control actions taking into account the information contained in past observations as well as the possible information that future observations may reveal. In short, the algorithm anticipates the fact that future learning is possible and selects the control actions accordingly. The algorithm uses memory-based methods to associate long-term benefit estimates to belief states and actions, and selects the actions to execute next according to such estimates. The algorithm uses the outcome of every experience to progressively refine the long-term benefit estimates so that it can make better, improved decisions as it progresses. The algorithm is tested on a classical simulation problem.

Read the paper:

A New Heuristic Approach for Dual Control

by Juan Carlos Santamaria, Ashwin Ram

AAAI-97 Workshop on On-Line Search, Providence, RI, July 1997
www.cc.gatech.edu/faculty/ashwin/papers/er-97-02.pdf

Can Your Architecture Do This? A Proposal for Impasse-Driven Asynchronous Memory Retrieval and Integration

We propose an impasse-driven method for generating memory retrieval requests and integrating their contents dynamically and asynchronously into the current reasoning context of an agent. This method extends our previous theory of agent architecture, called experience-based agency (Ram & Francis 1996), by proposing a general method that can replace and augment task-specific mechanisms for generating memory retrievals and invoking integration mechanisms. As part of an overall agent architecture, this method has promise as a way to introduce in a principled way efficient high-level memory operations into systems based on reactive task-network decomposition.

Read the paper:

Can Your Architecture Do This? A Proposal for Impasse-Driven Asynchronous Memory Retrieval and Integration

by Anthony Francis, Ashwin Ram

AAAI-97 Workshop on Robots, Softbots, Immobots: Theories of Action, Planning and Control, Providence, RI, July 1997
www.cc.gatech.edu/faculty/ashwin/papers/er-97-03.pdf

Situation Development in a Complex Real-World Domain

Applying techniques from Machine Learning to real-world domains and problems often requires considerable processing of the input data, to both remove noise and to augment the amount and type of information present. We describe our work in the task of situation assessment in the domain of US Army training exercises involving hundreds of agents interacting in real-time over the course of several days. In particular, we describe techniques we have developed to process this data and draw general conclusions on the types of information required in order to apply various Machine Learning algorithms and how this information may be extracted in real-world situations where it is not directly represented.

Read the paper:

Situation Development in a Complex Real-World Domain

by Mark Devaney and Ashwin Ram

International Conference on Machine Learning (ICML-97) Workshop on Machine Learning Applications in the Real World, Nashville, TN, July 1997
www.cc.gatech.edu/faculty/ashwin/papers/er-97-05.pdf

Continuous Case-Based Reasoning

Case-based reasoning systems have traditionally been used to perform high-level reasoning in problem domains that can be adequately described using discrete, symbolic representations. However, many real-world problem domains, such as autonomous robotic navigation, are better characterized using continuous representations. Such problem domains also require continuous performance, such as on-line sensorimotor interaction with the environment, and continuous adaptation and learning during the performance task.

This article introduces a new method for continuous case-based reasoning, and discusses its application to the dynamic selection, modification, and acquisition of robot behaviors in an autonomous navigation system, SINS (Self-Improving Navigation System). The computer program and the underlying method are systematically evaluated through statistical analysis of results from several empirical studies. The article concludes with a general discussion of case-based reasoning issues addressed by this research.

Read the paper:

Continuous Case-Based Reasoning

by Ashwin Ram, Juan Carlos Santamaria

Artificial Intelligence journal, (90)1-2:25-77, 1997
www.cc.gatech.edu/faculty/ashwin/papers/er-97-06.pdf
Follow

Get every new post delivered to your Inbox.

Join 2,375 other followers