Archive for the ‘Robotics’ Category

Emotional Memory and Adaptive Personalities

Believable agents designed for long-term interaction with human users need to adapt to them in a way which appears emotionally plausible while maintaining a consistent personality. For short-term interactions in restricted environments, scripting and state machine techniques can create agents with emotion and personality, but these methods are labor intensive, hard to extend, and brittle in new environments. Fortunately, research in memory, emotion and personality in humans and animals points to a solution to this problem. Emotions focus an animal’s attention on things it needs to care about, and strong emotions trigger enhanced formation of memory, enabling the animal to adapt its emotional response to the objects and situations in its environment. In humans this process becomes reflective: emotional stress or frustration can trigger re-evaluating past behavior with respect to personal standards, which in turn can lead to setting new strategies or goals.

To aid the authoring of adaptive agents, we present an artificial intelligence model inspired by these psychological results in which an emotion model triggers case-based emotional preference learning and behavioral adaptation guided by personality models. Our tests of this model on robot pets and embodied characters show that emotional adaptation can extend the range and increase the behavioral sophistication of an agent without the need for authoring additional hand-crafted behaviors.

Read the paper:

Emotional Memory and Adaptive Personalities

by Anthony Francis, Manish Mehta, Ashwin Ram

Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence, IGI Global, 2009

Integrating Robotic Technologies with JavaBots

Mobile robotics research advances through developments in theory, and implementation in hardware and software. While theory is important, this article is primarily concerned with hardware and software technologies. It is our view that significant strides can be made just be combining existing hardware and software tools. Thus the focus of this paper is answering the question: how can we more easily integrate robotic technologies?

We argue that the most effective approach is through standardized interfaces (APIs) to robotic hardware and software technologies (e.g., path planning toolkits). JavaBots is an example framework that provides this kind of integration in simulation and on robot ahrdware. A high-level common interface to sensors and actuators allows control systems to run on multiple simulated and real hardware platforms. Conversely, JavaBots supports the evaluation of competing control systems on the same hardware. In this article, we describe JavaBots and provide examples of robot systems built using it.

Read the paper:

Integrating Robotic Technologies with JavaBots

by Tucker Balch, Ashwin Ram

AAAI Spring Symposium on Integrating Robotic Research: Taking the Next Leap, Stanford, CA, March 1998

Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces

A key element in the solution of reinforcement learning problems is the value function. The purpose of this function is to measure the long-term utility or value of any given state. The function is important because an agent can use this measure to decide what to do next. A common problem in reinforcement learning when applied to systems having continuous states and action spaces is that the value function must operate with a domain consisting of real-valued variables, which means that it should be able to represent the value of infinitely many state and action pairs. For this reason, function approximators are used to represent the value function when a close-form solution of the optimal policy is not available.

In this paper, we extend a previously proposed reinforcement learning algorithm so that it can be used with function approximators that generalize the value of individual experiences across both, state and action spaces. In particular, we discuss the benefits of using sparse coarse-coded function approximators to represent value functions and describe in detail three implementations: CMAC, instance-based, and case-based. Additionally, we discuss how function approximators having different degrees of resolution in different regions of the state and action spaces may influence the performance and learning efficiency of the agent.

We propose a simple and modular technique that can be used to implement function approximators with non-uniform degrees of resolution so that it can represent the value function with higher accuracy in important regions of the state and action spaces. We performed extensive experiments in the double integrator and pendulum swing up systems to demonstrate the proposed ideas.

Read the paper:

Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces

by Juan Santamaria, Rich Sutton, Ashwin Ram

Adaptive Behavior, 6(2):163-217, 1997

A New Heuristic Approach for Dual Control

Autonomous agents engaged in a continuous interaction with an incompletely known environment face the problem of dual control (Feldbaum, 1965). Simply stated, actions are necessary not only for studying the environment, but also for making progress on the task. In other words, actions must bear a “dual” character: They must be investigators to some degree, but also directors to some degree. Because the number of variables involved in the solution of the dual control problem increases with the number of decision stages, the exact solution of the dual control problem is computationally intractable except for a few special cases.

This paper provides an overview of dual control theory and proposes a heuristic approach towards obtaining a near-optimal dual control method that can be implemented. The proposed algorithm selects control actions taking into account the information contained in past observations as well as the possible information that future observations may reveal. In short, the algorithm anticipates the fact that future learning is possible and selects the control actions accordingly. The algorithm uses memory-based methods to associate long-term benefit estimates to belief states and actions, and selects the actions to execute next according to such estimates. The algorithm uses the outcome of every experience to progressively refine the long-term benefit estimates so that it can make better, improved decisions as it progresses. The algorithm is tested on a classical simulation problem.

Read the paper:

A New Heuristic Approach for Dual Control

by Juan Carlos Santamaria, Ashwin Ram

AAAI-97 Workshop on On-Line Search, Providence, RI, July 1997

Continuous Case-Based Reasoning

Case-based reasoning systems have traditionally been used to perform high-level reasoning in problem domains that can be adequately described using discrete, symbolic representations. However, many real-world problem domains, such as autonomous robotic navigation, are better characterized using continuous representations. Such problem domains also require continuous performance, such as on-line sensorimotor interaction with the environment, and continuous adaptation and learning during the performance task.

This article introduces a new method for continuous case-based reasoning, and discusses its application to the dynamic selection, modification, and acquisition of robot behaviors in an autonomous navigation system, SINS (Self-Improving Navigation System). The computer program and the underlying method are systematically evaluated through statistical analysis of results from several empirical studies. The article concludes with a general discussion of case-based reasoning issues addressed by this research.

Read the paper:

Continuous Case-Based Reasoning

by Ashwin Ram, Juan Carlos Santamaria

Artificial Intelligence journal, (90)1-2:25-77, 1997

Case-Based Reactive Navigation: A Case-Based Method for On-Line Selection and Adaptation of Reactive Control Parameters in Autonomous Robotic Systems

This article presents a new line of research investigating on-line learning mechanisms for autonomous intelligent agents. We discuss a case-based method for dynamic selection and modification of behavior assemblages for a navigational system. The case-based reasoning module is designed as an addition to a traditional reactive control system, and provides more flexible performance in novel environments without extensive high-level reasoning that would otherwise slow the system down. The method is implemented in the ACBARR (A Case-BAsed Reactive Robotic) system, and evaluated through empirical simulation of the system on several different environments, including “box canyon” environments known to be problematic for reactive control systems in general.

Read the paper:

Case-Based Reactive Navigation: A Case-Based Method for On-Line Selection and Adaptation of Reactive Control Parameters in Autonomous Robotic Systems

by Ashwin Ram, Ron Arkin, Kenneth Moorman, Russ Clark

IEEE Transactions on Systems, Man, and Cybernetics, 27B(3), 1997. Preliminary version published as Technical Report GIT-CC-92/57, College of Computing, Georgia Institute of Technology, Atlanta, GA, 1992

Learning Adaptive Reactive Controllers

Reactive controllers has been widely used in mobile robots since they are able to achieve successful performance in real-time. However, the configuration of a reactive controller depends highly on the operating conditions of the robot and the environment; thus, a reactive controller configured for one class of environments may not perform adequately in another. This paper presents a formulation of learning adaptive reactive controllers. Adaptive reactive controllers inherit all the advantages of traditional reactive controllers, but in addition they are able to adjust themselves to the current operating conditions of the robot and the environment in order to improve task performance. Furthermore, learning adaptive reactive controllers can learn when and how to adapt the reactive controller so as to achieve effective performance under different conditions.

The paper presents an algorithm for a learning adaptive reactive controller that combines ideas from case-based reasoning and reinforcement learning to construct a mapping between the operating conditions of a controller and the appropriate controller configuration; this mapping is in turn used to adapt the controller configuration dynamically. As a case study, the algorithm is implemented in a robotic navigation system that controls a Denning MRV-III mobile robot. The system is extensively evaluated using statistical methods to verify its learning performance and to understand the relevance of different design parameters on the performance of the system.

Read the paper:

Learning Adaptive Reactive Controllers

by Juan Carlos Santamaria, Ashwin Ram

Technical Report GIT-CC-97/05, College of Computing, Georgia Institute of Technology, Atlanta, GA, January 1997

Get every new post delivered to your Inbox.

Join 2,433 other followers