Meta-Level Behavior Adaptation in Real-Time Strategy Games

AI agents designed for real-time settings need to adapt themselves to changing circumstances to improve their performance and remedy their faults. Agents typically designed for computer games, however, lack this ability. The lack of adaptivity causes a break in player experience when they repeatedly fail to behave properly in circumstances unforeseen by the game designers.

We present an AI technique for game-playing agents that helps them adapt to changing game circumstances. The agents carry out runtime adaptation of their behavior sets by monitoring and reasoning about their behavior execution and using this reasoning to dynamically revise their behaviors. The evaluation of the behavior adaptation approach in a complex real-time strategy game shows that the agents adapt themselves and improve their performance by revising their behavior sets appropriately.

Read the paper:

Meta-Level Behavior Adaptation in Real-Time Strategy Games

by Manish Mehta, Santi Ontañon, Ashwin Ram

ICCBR-10 Workshop on Case-Based Reasoning for Computer Games, Alessandria, Italy, 2010.
www.cc.gatech.edu/faculty/ashwin/papers/er-10-02.pdf

Conversational Framework for Web Search and Recommendations

We introduce a Conversational Interaction framework as an innovative and natural approach to facilitate easier information access by combining web search and recommendations. This framework includes an intelligent information agent (Cobot) in the conversation to provide contextually relevant social and web search recommendations. Cobot supports the information discovery process by integrating web information retrieval along with proactive connections to relevant users who can participate in real-time conversations. We describe the conversational framework and report on some preliminary experiments in the system.

Read the paper:

Conversational Framework for Web Search and Recommendations

by Saurav Sahay, Ashwin Ram

ICCBR-10 Workshop on Reasoning from Experiences on the Web (WebCBR-10), Alessandria, Italy, 2010.
www.cc.gatech.edu/faculty/ashwin/papers/er-10-01.pdf

Massively Multiplayer Online—Learning?

Massively Multiplayer Online—Learning?
aka, Are social networks disrupting models of education?

I spoke recently at a panel on Rebooting the University: Disruptions in Models of Learning.[1] In preparing my presentation, I found myself thinking about the topic of the panel. Are there new “models of learning”? The brain hasn’t changed all that much, has it?

The real disruption is not in models but modes of learning. Let me explain. Students today care about their education, perhaps more so than ever. In fact, 4 out of 5 students stress about their grades.[2] Yet class attendance is down. The more technology is used, the less likely students are to attend.[3] After all, why sit in a one-hour lecture when one can download the powerpoint and skim it the night before the exam? 60% of students find lectures “boring” and powerpoint “sleep inducing”.[4]

Students aren’t reading their textbooks either.[5] That’s an easy problem, you say—this is the digital generation, let’s digitize their books. Surely textbooks will be more accessible (and affordable) on their laptops, their Kindles, their iPhones? It turns out 60% of students read less when using e-textbooks instead of physical textbooks.[6] 600-page PDFs do not make the grade with today’s youngsters. Frankly, I can’t read 600-page PDFs either.

The problem starts well before the university. In the recent Silent Epidemic study[7] funded by the Gates Foundation, 47% of high school dropouts said a major reason for dropping out was that “classes were not interesting” and they were “bored”. Remarkably, 88% of dropouts had passing grades. These kids are not failing out of school; they are simply disengaging.

But wait, you say. Students are bored, they don’t go to class, they don’t read their textbooks—how in the world do they learn enough to get passing grades? That’s where modes of learning come in. Students do learn—but from Wikipedia, nearly 80% of them.[8] They learn from MIT’s OpenCourseware—50 million and counting[9], over 200 thousand visitors a month. That is a lot of engagement. And most significantly, they learn from their peers. 55% of teenagers report using IM to discuss homeworks—a larger percentage than dating.[10] Students are studying, but the web is their classroom.

But wait, you say again. Universities offer more than knowledge delivery; they offer community. As George Siemens says of Open Yale, “Great video and talented presenters. My only complaint: I’d like to interact with others who are viewing the resources. Creating a one-way flow of information significantly misses the point of interacting online.”[11] Don’t universities provide this interaction? Isn’t that their value?

Students do need community. But let’s look at where their communities are. 95% of college students are spending up to 10 hours a week in social networks[12]—blogging, updating their profiles, trading pictures, and—yes—talking about schoolwork. “With so many hunched over their laptops and cell phones”, as Preetha Ram says, “who is left on the college quad?”[13]

The college quad. The very phrase conjures up images of the walled gardens of academia, laced with ivy, filled with knowledge, brimming with students eager to absorb that knowledge. But, as my former student Chris Sprague puts it, today’s students are casting a wider net. The web is their classroom, Facebook is their community, the world is their study group. The days of walled gardens are over. That is the true disruption.

Modes of learning have changed. George Siemens talks about connectivism—the new mode learning in the digital age.[14] The university is no longer a walled garden; it is a hub that connects students to the world around them. It is open. Not just in the sense of free video lectures; rather, the community (which, after all, is the real value of the university) is open.

My colleagues and I have been building an online community called OpenStudy.[15] Funded by the National Science Foundation and the Georgia Research Alliance, OpenStudy is a kind of Facebook for learning. A place where students come, not to trade pictures and jokes, but to study. A place that connects them to other students in their university, to students in other universities, so they can study together.

We’ve seen this disruption in other areas. People collaborate online to create everything from music[16] to software[17]. Is creating knowledge any different?[18] As Rich DeMillo says, “social networks are well adapted to producing value in higher education.  The hubs and spokes of social networks reflect the long-tail effects that influencers have on learning.”[19]

I do research on games[20] and collaborative learning[21]. Anyone with a teenager at home knows how engaging massively multiplayer online games can be.[22] Stephen Downes and George Siemens are experimenting with massively multiplayer online courses.[23] OpenStudy can be thought of as a kind of massively multiplayer online learning—a world wide “guild” (if I may borrow a gaming term) of students interacting, helping, collaborating, studying together. A place for “user generated learning”, if you will.

Students get this. The world is their social graph, their gaming guild, and now, their study group. Student response to OpenStudy has been very positive. University response has also been positive, but many want to know if they can create a private network for their students. A closed network. AKA a walled garden. Universities still don’t get it.

The topic of the panel is Rebooting the University. My point is simple. The university is no longer a closed system, located in a tiny land-grant town a hundred miles from civilization. The days of isolation are over.[24] The university must be a hub for students to explore the world, expand their horizons, reach out to others. Students are doing this anyway, and if universities won’t adapt[25], students will do it without them.


[1] B Konsynski (2010), Knowledge Futures. http://halleinstitute.emory.edu/Research/knowledge_futures/2010forum.html

[2] SJ Cech (2008), Poll of U.S. teens finds heavier homework load, more stress over grades, Education Week. http://www.edweek.org/ew/articles/2008/08/13/45youth.h27.html

[3] Personally, I have abandoned technology in favor of the good old whiteboard. It is more work than flipping through powerpoints, but (speaking purely anecdotally) attendance is up, students are more engaged, grades have improved. And students seem to like it—I get more Thank A Teacher awards now J.

[4] S Mann (2009), Why do 60% of students find their lectures boring?, The Guardian. http://www.guardian.co.uk/education/2009/may/12/university-teaching

[5] Clump, Bauer & Bradley, 2004; Burchfield & Sapington, 2000; Murden & Gillepsie, 1997; McCabe, 2003.

[6] JT Rickman, J Von Holzen,  PG Klute, & T Tobin (2009), A campus-wide e-textbook initiative, EDUCAUSE Quarterly, 32(2). http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/ACampusWideETextbookInitiative/174581

[7] JM Bridgeland, JJ Dilulio Jr, KB Morrison (2006), The Silent Epidemic: Perspectives of High School Dropouts. http://www.civicenterprises.net/pdfs/thesilentepidemic3-06.pdf

[8] MH Miller (2010). Students use Wikipedia early and often, The Chronicle: Wired Campus. http://chronicle.com/blogPost/Students-Use-Wikipedia-Early/21850

[9] MIT OpenCourseWare marks 50 million visitors, The Boston Globe: Business News, 2008. http://www.boston.com/business/ticker/2008/12/mit_opencoursew.html

[10] 2007 AP-AOL Instant Messaging Trends Survey, reported in: http://www.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_view&newsId=20071115005196

[11] G Siemens (2007). Open Yale. http://www.elearnspace.org/blog/archives/003188.html

[12] National School Board Association (2008). Creating and Connecting: Research and Guidelines on Online Social—and Educational—Networking. http://www.scribd.com/doc/12836118/NSBA-Social-Networking-Study

[13] P Ram (2009). An Empty College Quad? http://preetharam.wordpress.com/2009/01/29/an-empty-college-quad

[14] G Siemens (2004). Connectivism: A learning theory for the digital age. eLearnSpace. http://www.elearnspace.org/Articles/connectivism.htm

[15] http://openstudy.com

[16] http://tunerooms.com, http://thounds.com, and others

[17] http://sourceforge.net, http://github.com, and many others

[18] D Wiley. Open source, openness, and higher education. Innovate Journal of Online Education. http://www.innovateonline.info/index.php?view=article&id=354

[19] RA DeMillo (2011). Abelard to Apple: The Fate of American Colleges and Universities in the Twenty-First Century, MIT Press, in press.

[20] https://cognitivecomputing.wordpress.com/tag/games

[21] https://cognitivecomputing.wordpress.com/tag/educational-technology

[22] Actually, the average age of gamers is 35 [ESA 2009: http://www.theesa.com/facts] so this holds for adults too. This is good; universities will need to engage adults too as they begin to address lifelong learning seriously.

[23] Massively Open Online Courses (MOOCs): http://ltc.umanitoba.ca/connectivism/?p=53

[24] It is no surprise to me that student-voted “best college towns” are no longer Ann Arbor and College Park, but places like Georgetown and our very own Emory [Princeton Review]. The campus town isn’t Emory Village, it is Atlanta, it is Washington DC, it is Greenwich Village. Students today are indeed casting a wider net, in more ways than one.

[25] Rich DeMillo (ibid.) describes one such vision: open courseware, hacked degrees, no brick walls, and above all an increased emphasis on access and a de-emphasis on selectivity and exclusion.

ICCBR-10 Workshop on CBR Startups

CBR Startups

ICCBR 2010 Workshop / July 20, 2010 / Alessandria, Italy

Please fill out a short participation survey: www.surveymonkey.com/s/S2ST588

Over the past twenty-five years, Case-Based Reasoning has matured into a full-fledged discipline within AI, with an international community, strong research momentum, and many commercial successes. However, despite its many advantages as a technology, CBR is not well known in the entrepreneurial world. In part, this is due to few startups being created by CBR researchers, who are the best people to initiate commercialization of their ideas.

This workshop will focus on the merits and challenges of creating a technology startup out of cutting-edge research in academia or research labs. We will discuss technological issues, such as the application areas best suited for CBR approaches and scalability of CBR technologies. We will also discuss practical issues, such as the tension between academic goals (e.g., publishing papers) and commercialization goals (e.g., building applications), and the different types of expertise required to create a vision (researchers), market a product (marketers), and build a company (entrepreneurs).

In true CBR fashion, we will use cases to tackle these issues.  We will hear from CBR researchers who have created CBR startups, and use their experiences to discuss different ways to commercialize CBR technologies. Some have chosen a hands-on approach, taking on a management role (CEO) or a technology role (CTO). Others have partnered with experienced business people, who have taken their ideas forward. We will also provide a forum to help participants with their startup ideas.

Agenda/Schedule

CBR Startups will be a half-day workshop with short talks by people who have spun out companies from their universities, a panel discussion with open audience questions on the merits and challenges of doing a startup, alternative ways of commercializing CBR technologies, and advice to people interested in doing this.

We will conclude with a hands-on session with 3 minute pitches by participants. Think of it as throwing down the gauntlet—a friendly competition where you pitch your CBR startup idea. Prizes will be awarded for most innovative use of CBR technology, best business idea, and idea most likely to succeed. Our intention is to provide advice and mentoring by community members who have been-there-done-that, using these ideas as case studies for all of us to learn from.

Tentative agenda:

  • Introduction and context (Ashwin Ram)
  • Short talks by CBR researchers who have done startups
  • Panel discussion with open audience questions
  • 3 minute CBR gauntlet Pitch competition

We want this to be useful to you, so please help us refine the agenda by filling out this brief survey: www.surveymonkey.com/s/S2ST588

Organizers

Invited Speakers & Mentors

[More to come. If you’d like to speak or mentor, please fill out the survey.]

Help us publicize this workshop!

Please forward this URL to others who might be interested: http://bit.ly/cbr-startups

For more information about ICCBR 2010, see: www.iccbr.org/iccbr10

User-Generated AI for Interactive Digital Entertainment

CMU Seminar

User-generated content is everywhere: photos, videos, news, blogs, art, music, and every other type of digital media on the Social Web. Games are no exception. From strategy games to immersive virtual worlds, game players are increasingly engaged in creating and sharing nearly all aspects of the gaming experience: maps, quests, artifacts, avatars, clothing, even games themselves. Yet, there is one aspect of computer games that is not created and shared by game players: the AI. Building sophisticated personalities, behaviors, and strategies requires expertise in both AI and programming, and remains outside the purview of the end user.

To understand why Game AI is hard, we need to understand how it works. AI can take digital entertainment beyond scripted interactions into the arena of truly interactive systems that are responsive, adaptive, and intelligent. I discuss examples of AI techniques for character-level AI (in embedded NPCs, for example) and game-level AI (in the drama manager, for example). These types of AI enhance the player experience in different ways. The techniques are complicated and are usually implemented by expert game designers.

I argue that User-Generated AI is the next big frontier in the rapidly growing Social Gaming area. From Sims to Risk to World of Warcraft, end users want to create, modify, and share not only the appearance but the “minds” of their characters. I present my recent research on intelligent technologies to assist Game AI authors, and show the first Web 2.0 application that allows average users to create AIs and challenge their friends to play them—without programming. I conclude with some thoughts about the future of AI-based Interactive Digital Entertainment.

CMU Robotics & Intelligence Seminar, September 28, 2009
Carnegie-Mellon University, Pittsburgh, PA.
MIT Media Lab Colloquium, January 25, 2010
Massachusetts Institute of Technology, Cambridge, MA.
Stanford Media X Philips Seminar, February 1, 2010
Stanford University, Stanford, CA.
Pixar Research Seminar, February 2, 2010

Try it yourself:
Learn more about the algorithms:
View the talk:
www.sais.se/blog/?p=57

View the slides:

Drama Management and Player Modeling for Interactive Fiction Games

A growing research community is working towards employing drama management components in story-based games. These components gently guide the story towards a narrative arc that improves the player’s gaming experience. In this paper we evaluate a novel drama management approach deployed in an interactive fiction game called Anchorhead. This approach uses player’s feedback as the basis for guiding the personalization of the interaction.

The results indicate that adding our Case-based Drama manaGer (C-DraGer) to the game guides the players through the interaction and provides a better overall player experience. Unlike previous approaches to drama management, this paper focuses on exhibiting the success of our approach by evaluating results using human players in a real game implementation. Based on this work, we report several insights on drama management which were possible only due to an evaluation with real players.

Read the paper:

Drama Management and Player Modeling for Interactive Fiction Games

by Manu Sharma, Santi Ontañón, Manish Mehta, Ashwin Ram

Computational Intelligence, 26(2):183-211, 2010.
www.cc.gatech.edu/faculty/ashwin/papers/er-09-10.pdf
www3.interscience.wiley.com/journal/123387570/abstract

Run-Time Behavior Adaptation for Real-Time Interactive Games

Intelligent agents working in real-time domains need to adapt to changing circumstance so that they can improve their performance and avoid their mistakes. AI agents designed for interactive games, however, typically lack this ability. Game agents are traditionally implemented using static, hand-authored behaviors or scripts that are brittle to changing world dynamics and cause a break in player experience when they repeatedly fail. Furthermore, their static nature causes a lot of effort for the game designers as they have to think of all imaginable circumstances that can be encountered by the agent. The problem is exacerbated as state-of-the-art computer games have huge decision spaces, interactive user input, and real-time performance that make the problem of creating AI approaches for these domains harder.

In this paper we address the issue of non-adaptivity of game playing agents in complex real-time domains. The agents carry out run-time adaptation of their behavior sets by monitoring and reasoning about their behavior execution to dynamically carry out revisions on the behaviors. The behavior adaptation approaches has been instantiated in two real-time interactive game domains. The evaluation results shows that the agents in the two domains successfully adapt themselves by revising their behavior sets appropriately.

Read the paper:

Run-Time Behavior Adaptation for Real-Time Interactive Games

by Manish Mehta, Ashwin Ram

IEEE Transactions on Computational Intelligence and AI in Games, Vol. 1, No. 3, September 2009
www.cc.gatech.edu/faculty/ashwin/papers/er-09-09.pdf

Using Meta-Reasoning to Improve the Performance of Case-Based Planning

Case-based planning (CBP) systems are based on the idea of reusing past successful plans for solving new problems. Previous research has shown the ability of meta-reasoning approaches to improve the performance of CBP systems. In this paper we present a new meta-reasoning approach for autonomously improving the performance of CBP systems that operate in real-time domains.

Our approach uses failure patterns to detect anomalous behaviors, and it can learn from experience which of the failures detected are important enough to be fixed. Finally, our meta-reasoning approach can exploit both successful and failed executions for meta-reasoning.

We illustrate its benefits with experimental results from a system implementing our approach called Meta-Darmok in a real-time strategy game. The evaluation of Meta-Darmok shows that the system successfully adapts itself and its performance improves through appropriate revision of the case base.

Read the paper:

Using Meta-Reasoning to Improve the Performance of Case-Based Planning

by Manish Mehta, Santi Ontañón, Ashwin Ram

International Conference on Case-Based Reasoning (ICCBR-09), Seattle, July 2009
www.cc.gatech.edu/faculty/ashwin/papers/er-09-06.pdf

Collaborative Information Access: A Conversational Search Approach

Knowledge and user-generated content is proliferating on the web in scientific publications, information portals and online social media. This knowledge explosion has continued to outpace technological innovation in efficient information access technologies. In this paper, we describe methods and technologies for “Conversational Search” as an innovative solution to facilitate easier information access and reduce the information overload for users.

Conversational Search is an interactive and collaborative information finding interaction. The participants in this interaction engage in social conversations aided with an intelligent information agent (Cobot) that provides contextually relevant search recommendations. The collaborative and conversational search activity helps users make faster and more informed search and discovery. It also helps the agent learn about conversations with interactions and social feedback to make better recommendations. Conversational search leverages the social discovery process by integrating web information retrieval along with the social interactions.

Read the paper:

Collaborative Information Access: A Conversational Search Approach

by Saurav Sahay, Anu Venkatesh, Ashwin Ram

ICCBR-09 Workshop on Reasoning from Experiences on the Web (WebCBR-09), Seattle, July 2009
www.cc.gatech.edu/faculty/ashwin/papers/er-09-05.pdf

Authoring Behaviors for Games using Learning from Demonstration

Behavior authoring for computer games involves writing behaviors in a programming language. This method is cumbersome and requires a lot of programming effort to author the behavior sets. Further, this approach restricts the behavior set authoring to people who are experts in programming.

This paper describes our approach to design a system that allows a user to demonstrate behaviors to the system, which the system uses to learn behavior sets for a game domain. With learning from demonstration, we aim at removing the requirement that the user has to be an expert in programming, and only require him to be an expert in the game. The approach has been integrated in a easy-to-use visual interface and instantiated for two domains, a real-time strategy game and an interactive drama.

Read the paper:

Authoring Behaviors for Games using Learning from Demonstration

by Manish Mehta, Santiago Ontañón, Tom Amundsen, Ashwin Ram

ICCBR-09 Workshop on Case-Based Reasoning for Computer Games, Seattle, July 2009
www.cc.gatech.edu/faculty/ashwin/papers/er-09-07.pdf