Posts Tagged ‘problem-based learning’

1 Oct

Make the World your Study Group

Posted by cognitivecomputing in Education, Opinion, Talks, Web / Web 2.0. Tagged: , , , .

CNN Chalk Talk: A new website called OpenStudy allows students to share resources and learn with one another from all over the world.

Click the image to watch the video (3 min.)

Read the transcript: CNN Chalk Talk, October 1, 2010

T.J. HOLMES, CNN ANCHOR: Well, coming up, calling all college students. There’s now a group online that allows you to study in a unique way. You can get help from across the globe. You don’t even need a passport.

(BREAK)

HOLMES: Well, we turn to “Chalk Talk” today, now.

We are checking out a new study group that’s geared toward helping college students succeed. It’s an online study group called OpenStudy, and it’s linking students from around the world, helping them pass some tough courses.

Joining me now is Ashwin Ram. He’s the director of Georgia Tech’s Cognitive Computing Lab, one of the founders of OpenStudy.

Sir, thank you for being here.

OpenStudy, this is a worldwide study group. Do I kind of have that right?

ASHWIN RAM, DIRECTOR, GEORGIA TECH’S COGNITIVE COMPUTING LAB: That’s right. Open Study is a match.com for studying. It’s a social learning network that enables students to connect and study together, and get help when they need it.

HOLMES: Now, you said you’ve all been thinking about this for a while, for the past couple of years. What were you trying to work out, make sure there was a market for it, or is there some complicated technology you had to work out as well?

RAM: It was actually both. We wanted to get the value proposition right for students. We spent a lot of time researching the core need that students have, and that resulted in OpenStudy.

HOLMES: What did you determine was that core need? What did you find that students out there needed?

RAM: So, students all over the world are hitting their textbooks late at night cramming for exams. Maybe they’re working on review problems, watching video lectures on iTunes or MIT.

When these students need help, who can they turn to? The core need was to be able to find someone who can help them and give them help right there, right then, no matter what time they needed that help.

HOLMES: All right. And this is, again, supposed to link students with students. Essentially a study group like at the library.

RAM: It’s a worldwide study group. Our mantra is “We want to make the entire world your study group.” So there’s always someone who can help you.

HOLMES: How does this thing work? It looks like a social network page almost here.

RAM: It does. So let’s say that you are a student, and you’re one of 10,000 students studying computer science on MIT’s web site. And you’re working on video lectures or problem sets, and you have a question.

HOLMES: OK.

RAM: What do you do? You join a study group. When you do that, you get dropped into the MIT OpenStudy Group.

As you can see, we have over 2,200 people out there. Think of them as your classmates that can help you any time you want.

I noticed that we’ve just had someone join us from Kenya.

HOLMES: Oh, wow.

RAM: We actually have students from 138 countries from around the world. That’s 71 percent of the world’s countries.

HOLMES: Now, does this cost the kids anything to sign up for?

RAM: No, it’s completely free.

HOLMES: I’ll be danged. So you can pretty much — as well, you’re talking about kids up all hours of the night. No matter — somewhere in the world somebody is going to be up, somebody’s going to be logged on, somebody’s going to be studying.

RAM: Someone will always help you. And so if you have — you can go in and help somebody, but if you have a question, or you want to just study together with someone, you click on “Ask a Question,” type some question in that you want help with, and say, “Ask Now.”

The question is posted. Everything updates in real time. And you go back to the site, and then someone will be available to start answering you.

HOLMES: Will start answering you.

All right. Are you ready for growth? Because this might catch on. Are you ready for what might come?

RAM: We are ready for growth.

HOLMES: OK.

RAM: We’ve had remarkable growth already. We’ve only been live two weeks. We have over 6,000 people already using the site.

HOLMES: All right. This is going to be the next Facebook, 500 million. Come back when you get 500 million members in there. All right?

RAM: Thank you.

HOLMES: All right.

Ashwin Ram from Georgia Tech.

Thank you so much. Cool concept.

RAM: Thank you. It was a pleasure.

15 Sep

OpenStudying the Classics

Posted by cognitivecomputing in Education, Opinion, Web / Web 2.0. Tagged: , , , .

I recently met Dr. Diana E. E. Kleiner, a distinguished professor at my alma mater and director of the Open Yale Courses initiative. We were talking about “OpenStudying the Classics”—to my knowledge, the first use of “OpenStudy” as a verb.[1] This made me think—what does it mean to “OpenStudy” something?

Some background first. In collaboration with Dean Preetha Ram of Emory University, our former student Chris Sprague from Georgia Tech’s HCI program, and experienced internet entrepreneur Phil Hill, and with funding from the National Science Foundation, the National Institutes of Health, and the Georgia Research Alliance, I’ve been working on a system called OpenStudy[2] which embodies a new way of studying. In the new millennial world of social networking, where social graphs have no geographical boundaries, professional networks are world wide, and entertainment streams from the far corners of the globe into the palm of your hand, it has always seemed odd to me that education is bounded by school walls, class interactions are limited to one teacher and a few dozen students who happened to register at the same time as you, and studying is largely a solitary activity circumscribed by so-called “collaboration policies” that typically require students to learn alone. Even “open learning” initiatives offer little more than a solitary experience watching instructional videos in your home, albeit from world famous experts.

OpenStudy, in contrast, whole-heartedly embraces the idea of “social learning”. The world is your study group, we claim. Connect with others studying the same things you are. Give and get help. The world learns as one.

But what is the “OpenStudy experience”? What will it mean, as Prof. Kleiner wonders, to “OpenStudy the Classics”? I don’t have the final answer (sic) but I do want to share my observations from a pilot with MIT OpenCourseware (OCW). For the past month, learners in three OCW courses have been given an option to “Join a study group”.[3] OCW reports their study groups are growing at a “blistering pace”—by our metrics, by about 10% a day. Learners are demanding more OpenStudy groups; if we don’t respond quickly, they create their own. What’s going on?

It’s too early for hard metrics, but permit me to share some anecdotes. MIT pilot courses include Intro CS, Calculus, and Chinese, and there are certainly interesting interactions around those topics.[4] But users are also exploring other interests. For example, there’s an active conversation about Greek Classics. What’s interesting are the participants:

  • a classics librarian at an exclusive four-year college in New England
  • a young woman considering a PhD in social sciences
  • an international student at a community college student in Georgia
  • a professor from the MIT Physics group
  • someone studying Chinese
  • a mid-career Math/CS geek from Michigan

These people did not know each other prior to their OpenStudy encounter. OpenStudy is described as the Match.com for studying together[5]—if so, this certainly seems to be working. A student from Peru came online recently, introduced himself, and apologized for his poor English. A student in the US responded in Spanish, and they struck up a conversation around their mutual study interests. Then a user from Mexico City jumped into the conversation, and off they went studying together with two users from Costa Rica. This is the new world of OpenStudying—social learning without geographical boundaries.

A homeschooled teenager recently joined OpenStudy and said “I’m new. How do you OpenStudy?” 15 minutes later, she had connected with students in an all-girls private school. She initiated a discussion on World Religions which, less than a day later, has nearly 20 participants. Half of them have contributed and half are listening. A Hindu undergraduate from India, an Orthodox Jew from Texas, and a Muslim student from Turkey are talking about what “real” Islam is like. A teenager who can’t drive, doesn’t go to school, and does not have traditional teachers or schoolmates has answered her own question. This is how you OpenStudy. You study with the world.

Every educator knows the challenge of keeping students engaged. Studying together not only improves learning, it is a lot more fun. One of the users recently emailed us saying: “Personally, I’ve come further in my development as a programmer in the month of being on OpenStudy than the previous few years struggling on my own. Being able to see how other people approach problems and considering their questions is absolutely wonderful.” A GSU professor says she is seeing 400% increase in student engagement in her required lower-division biology class due to OS.

So this, Prof. Kleiner, is how we will be able to “OpenStudy the Classics”. Students connecting with students studying the same things they are. I call it “massively multiplayer online learning”[6], a wordplay on the MMO experience we’re seeing in the gaming world. Here it has value beyond entertainment; the diversity adds to the richness of the online study group, the globalness broadens access beyond elite institutional walls, the interactivity engages today’s millennials in—of all things—study.

On average, we’re seeing 5.3 participants per “studypad” (a real-time interaction tool that facilitates conversation, discussion, or simply question answering). About 30% of the interactions occur synchronously in real time. This is quite different from a typical question site, where you post a question and wait—an hour? a day? who knows when someone might answer. OpenStudying is like a conversation in a university library or the local Starbucks, instant real-time interaction with peers—except that these peers might be halfway around the globe. The world is, after all, your social network, your professional rolodex, and, now, your study group.

Ashwin Ram
September 15, 2010

If you’ve OpenStudied and would like to share your experience, I’d love to hear about it. Please add a comment below.

[1] I’ve always been interested in the origins of words, especially new ones. Who coined the term “WebLog”, and who first shortened it to “blog? Who first used “Google” as a verb? In this day and age, surely there must be a record somewhere. To this end and with her permission, I’d like to credit Prof. Diana Kleiner with the first use of “OpenStudy” as a verb.

[2] OpenStudy is free and publicly available at OpenStudy.com, a for-profit spinoff from Georgia Tech and Emory University created via the university’s commercialization program. Our objective is to create not just an interesting research project but a sustainable product that will make a difference to thousands of learners everywhere. To accomplish this, we need to grapple with the realities of business models, lest our project die the way countless other good ideas do when their research funding runs out.

[3] Update: MIT OCW has expanded its program to several more courses: web.mit.edu/newsoffice/2010/ocw-openstudy.html

[4] Click “Join a Study Group” on the course page to see the corresponding study group.

[5] See Marc Parry’s article in The Chronicle’s Wired Campus, Start-Up Aspires to Make the World ‘One Big Study Group’, September 8, 2010: wiredcampus.chronicle.com/blogPost/Start-Up-Aspires-to-Make-the/26780

[6] My talk at the Knowledge Futures: Disrupting the University forum at Emory University, entitled Massively Multiplayer Online—Learning? aka Are social networks disrupting models of education? cognitivecomputing.wordpress.com/2010/04/26/massively-multiplayer-online—learning/

26 Apr

Massively Multiplayer Online—Learning?

Posted by cognitivecomputing in Education, Opinion, Talks, Web / Web 2.0. Tagged: , , , .

Massively Multiplayer Online—Learning?
aka, Are social networks disrupting models of education?

I spoke recently at a panel on Rebooting the University: Disruptions in Models of Learning.[1] In preparing my presentation, I found myself thinking about the topic of the panel. Are there new “models of learning”? The brain hasn’t changed all that much, has it?

The real disruption is not in models but modes of learning. Let me explain. Students today care about their education, perhaps more so than ever. In fact, 4 out of 5 students stress about their grades.[2] Yet class attendance is down. The more technology is used, the less likely students are to attend.[3] After all, why sit in a one-hour lecture when one can download the powerpoint and skim it the night before the exam? 60% of students find lectures “boring” and powerpoint “sleep inducing”.[4]

Students aren’t reading their textbooks either.[5] That’s an easy problem, you say—this is the digital generation, let’s digitize their books. Surely textbooks will be more accessible (and affordable) on their laptops, their Kindles, their iPhones? It turns out 60% of students read less when using e-textbooks instead of physical textbooks.[6] 600-page PDFs do not make the grade with today’s youngsters. Frankly, I can’t read 600-page PDFs either.

The problem starts well before the university. In the recent Silent Epidemic study[7] funded by the Gates Foundation, 47% of high school dropouts said a major reason for dropping out was that “classes were not interesting” and they were “bored”. Remarkably, 88% of dropouts had passing grades. These kids are not failing out of school; they are simply disengaging.

But wait, you say. Students are bored, they don’t go to class, they don’t read their textbooks—how in the world do they learn enough to get passing grades? That’s where modes of learning come in. Students do learn—but from Wikipedia, nearly 80% of them.[8] They learn from MIT’s OpenCourseware—50 million and counting[9], over 200 thousand visitors a month. That is a lot of engagement. And most significantly, they learn from their peers. 55% of teenagers report using IM to discuss homeworks—a larger percentage than dating.[10] Students are studying, but the web is their classroom.

But wait, you say again. Universities offer more than knowledge delivery; they offer community. As George Siemens says of Open Yale, “Great video and talented presenters. My only complaint: I’d like to interact with others who are viewing the resources. Creating a one-way flow of information significantly misses the point of interacting online.”[11] Don’t universities provide this interaction? Isn’t that their value?

Students do need community. But let’s look at where their communities are. 95% of college students are spending up to 10 hours a week in social networks[12]—blogging, updating their profiles, trading pictures, and—yes—talking about schoolwork. “With so many hunched over their laptops and cell phones”, as Preetha Ram says, “who is left on the college quad?”[13]

The college quad. The very phrase conjures up images of the walled gardens of academia, laced with ivy, filled with knowledge, brimming with students eager to absorb that knowledge. But, as my former student Chris Sprague puts it, today’s students are casting a wider net. The web is their classroom, Facebook is their community, the world is their study group. The days of walled gardens are over. That is the true disruption.

Modes of learning have changed. George Siemens talks about connectivism—the new mode learning in the digital age.[14] The university is no longer a walled garden; it is a hub that connects students to the world around them. It is open. Not just in the sense of free video lectures; rather, the community (which, after all, is the real value of the university) is open.

My colleagues and I have been building an online community called OpenStudy.[15] Funded by the National Science Foundation and the Georgia Research Alliance, OpenStudy is a kind of Facebook for learning. A place where students come, not to trade pictures and jokes, but to study. A place that connects them to other students in their university, to students in other universities, so they can study together.

We’ve seen this disruption in other areas. People collaborate online to create everything from music[16] to software[17]. Is creating knowledge any different?[18] As Rich DeMillo says, “social networks are well adapted to producing value in higher education.  The hubs and spokes of social networks reflect the long-tail effects that influencers have on learning.”[19]

I do research on games[20] and collaborative learning[21]. Anyone with a teenager at home knows how engaging massively multiplayer online games can be.[22] Stephen Downes and George Siemens are experimenting with massively multiplayer online courses.[23] OpenStudy can be thought of as a kind of massively multiplayer online learning—a world wide “guild” (if I may borrow a gaming term) of students interacting, helping, collaborating, studying together. A place for “user generated learning”, if you will.

Students get this. The world is their social graph, their gaming guild, and now, their study group. Student response to OpenStudy has been very positive. University response has also been positive, but many want to know if they can create a private network for their students. A closed network. AKA a walled garden. Universities still don’t get it.

The topic of the panel is Rebooting the University. My point is simple. The university is no longer a closed system, located in a tiny land-grant town a hundred miles from civilization. The days of isolation are over.[24] The university must be a hub for students to explore the world, expand their horizons, reach out to others. Students are doing this anyway, and if universities won’t adapt[25], students will do it without them.

[1] B Konsynski (2010), Knowledge Futures. http://halleinstitute.emory.edu/Research/knowledge_futures/2010forum.html

[2] SJ Cech (2008), Poll of U.S. teens finds heavier homework load, more stress over grades, Education Week. http://www.edweek.org/ew/articles/2008/08/13/45youth.h27.html

[3] Personally, I have abandoned technology in favor of the good old whiteboard. It is more work than flipping through powerpoints, but (speaking purely anecdotally) attendance is up, students are more engaged, grades have improved. And students seem to like it—I get more Thank A Teacher awards now J.

[4] S Mann (2009), Why do 60% of students find their lectures boring?, The Guardian. http://www.guardian.co.uk/education/2009/may/12/university-teaching

[5] Clump, Bauer & Bradley, 2004; Burchfield & Sapington, 2000; Murden & Gillepsie, 1997; McCabe, 2003.

[6] JT Rickman, J Von Holzen,  PG Klute, & T Tobin (2009), A campus-wide e-textbook initiative, EDUCAUSE Quarterly, 32(2). http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/ACampusWideETextbookInitiative/174581

[7] JM Bridgeland, JJ Dilulio Jr, KB Morrison (2006), The Silent Epidemic: Perspectives of High School Dropouts. http://www.civicenterprises.net/pdfs/thesilentepidemic3-06.pdf

[8] MH Miller (2010). Students use Wikipedia early and often, The Chronicle: Wired Campus. http://chronicle.com/blogPost/Students-Use-Wikipedia-Early/21850

[9] MIT OpenCourseWare marks 50 million visitors, The Boston Globe: Business News, 2008. http://www.boston.com/business/ticker/2008/12/mit_opencoursew.html

[10] 2007 AP-AOL Instant Messaging Trends Survey, reported in: http://www.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_view&newsId=20071115005196

[11] G Siemens (2007). Open Yale. http://www.elearnspace.org/blog/archives/003188.html

[12] National School Board Association (2008). Creating and Connecting: Research and Guidelines on Online Social—and Educational—Networking. http://www.scribd.com/doc/12836118/NSBA-Social-Networking-Study

[13] P Ram (2009). An Empty College Quad? http://preetharam.wordpress.com/2009/01/29/an-empty-college-quad

[14] G Siemens (2004). Connectivism: A learning theory for the digital age. eLearnSpace. http://www.elearnspace.org/Articles/connectivism.htm

[15] http://openstudy.com

[16] http://tunerooms.com, http://thounds.com, and others

[17] http://sourceforge.net, http://github.com, and many others

[18] D Wiley. Open source, openness, and higher education. Innovate Journal of Online Education. http://www.innovateonline.info/index.php?view=article&id=354

[19] RA DeMillo (2011). Abelard to Apple: The Fate of American Colleges and Universities in the Twenty-First Century, MIT Press, in press.

[20] https://cognitivecomputing.wordpress.com/tag/games

[21] https://cognitivecomputing.wordpress.com/tag/educational-technology

[22] Actually, the average age of gamers is 35 [ESA 2009: http://www.theesa.com/facts] so this holds for adults too. This is good; universities will need to engage adults too as they begin to address lifelong learning seriously.

[23] Massively Open Online Courses (MOOCs): http://ltc.umanitoba.ca/connectivism/?p=53

[24] It is no surprise to me that student-voted “best college towns” are no longer Ann Arbor and College Park, but places like Georgetown and our very own Emory [Princeton Review]. The campus town isn’t Emory Village, it is Atlanta, it is Washington DC, it is Greenwich Village. Students today are indeed casting a wider net, in more ways than one.

[25] Rich DeMillo (ibid.) describes one such vision: open courseware, hacked degrees, no brick walls, and above all an increased emphasis on access and a de-emphasis on selectivity and exclusion.

9 Mar

Teaching Computational Thinking to Third-Graders

Posted by cognitivecomputing in Education. Tagged: , .

From the Paideia School Newsletter, March 2009:

Computational Thinking is a new way of solving problems that derives from computer science. It involves approaching problems in a systematic, step-by-step manner, and building up solutions to complex problems from smaller pieces. Nowadays, computational thinking is a fundamental skill for everyone, not just computer scientists. Many educators argue that it should be part of every child’s education along with reading, writing and arithmetic.

Assisted by Georgia Tech computer science professor Dr. Ashwin Ram, a group of students in Kelly and Tony’s 3rd grade class at The Paideia School explored computational thinking over a four-week affinities session. The students built familiar computer games, including Etch-A-Sketch, Pong, and Tennis, and shared them with each other on a web site. They built the games not only from scratch but in Scratch, a new tool from MIT that is designed to enable young children design and build interactive computer programs.

In teams of two, the students played with algorithmic concepts including scripts, conditionals, and loops. They customized the look-and-feel of their games, built game characters called sprites, designed behaviors for their sprites, and added sounds for extra effect. They decided how their games would respond to the player and how to keep score. They tested, revised, tested again, revised again.

And when they were done, they shared their games for their classmates and anyone else to enjoy. They received some nice comments! You can try out their games yourself at scratch.mit.edu/users/kellytony.

Says Ashwin: “I received a wonderful set of thank you cards from my affinities group. What a nice surprise. One of the children wrote, “Those weeks were the best weeks of my life.” Wow. It was a great experience. The kids got a lot out of it — they learned something new, grappled with a new way of thinking about problems, and had fun at the same time. I know some of them will want to continue doing this, at least until they find their next passion! And I had a blast as well. Thank you, Kelly and Tony, for giving me this opportunity.”

Ashwin and Preetha Ram are the proud parents of three Paideia students, Naveen (3rd grade), Maya (6th grade), and Nikhil (12th grade). Ashwin can be reached by email (ashwin AT cc.gatech.edu), Twitter (@ashwinram), and LinkedIn (www.linkedin.com/in/ashwinram). Scratch is freely available at scratch.mit.edu.


8 Jul

A Cognitive Model of Problem-Based Learning and its Application to Educational Software Design

Posted by cognitivecomputing in Education, Learning, Web / Web 2.0. Tagged: , .

Problem-based learning (PBL) is a constructivist pedagogy in which students learn in small groups by working on real-world problems. Despite its many benefits, however, this pedagogy is still not widely used in K-16 classrooms, especially with large numbers of students. Traditional human-facilitated PBL places intense demands on faculty to facilitate problem-solving sessions with small groups of students; on the other hand, most educational technologies do not provide PBL’s collaborative problem-solving experience.

We propose a cognitive model of the problem-based learning process. We present a software environment called CaseBook that allows instructors to author and share problems and provides students with a pedagogically-sound PBL experience based on the cognitive model. CaseBook has been used in high school and undergraduatefrom two studies in actual classrooms.

Read the paper:

A Cognitive Model of Problem-Based Learning and its Application to Educational Software Design

by Ashwin Ram, Preetha Ram, Jennifer Holzmann, Chris Sprague

International Conference on e-Learning (eLearn-07), Lisbon, Portugal, July 2007. Also presented at Eleventh International Conference on Human-Computer Interaction (INTERACT-07), Panel on Human-Centric e-Learning, Rio de Janeiro, Brazil, September 2007.

www.cc.gatech.edu/faculty/ashwin/papers/er-07-05.pdf

15 May

CaseBook: A Problem-Based Learning Online Environment For High School Microbiology

Posted by cognitivecomputing in Education, Learning, Web / Web 2.0. Tagged: , .

Problem-based learning (PBL) is an educational approach that allows students to improve problem solving and critical thinking skills while learning science. However, PBL requires significant teacher time and expertise to develop problems and facilitate small-group problem-solving sessions. With advances in technology, PBL can be used in today’s classrooms in an effective and scalable manner.

CaseBook is an interactive computer system that allows for easy integration of PBL into the K-16 curriculum. Through a simple web-based interface, teachers enter and edit their case materials. As students work through cases, CaseBook guides them through a 3-stage process in which they analyze, learn and reflect. Students may work independently, or a small group of students may work together and share a Team Notebook, which is used to record facts, ideas, and issues about the case as they progress. Students assess their progress through self and group reflection and through teacher feedback.

We report on the use of CaseBook for a microbiology case in a high school classroom. The results suggest that CaseBook is effective for both advanced and remedial students. As the technological capacity of students and classrooms increase, it is only appropriate to use this technology to implement novel methods of teaching that will provide students the skills they need post- graduation.

Read the paper:

CaseBook: A Problem-Based Learning Online Environment For High School Microbiology

by JL Holzman, G Louizi, SC Fowler, E Lindsey, JJ Harrigan, P Ram, A Ram

12th American Society for Microbiology (ASM) Conference for Undergraduate Educators, Atlanta, GA, May 2006
www.cc.gatech.edu/faculty/ashwin/papers/er-05-06.doc.pdf
www.cc.gatech.edu/faculty/ashwin/papers/er-05-06.pdf

9 Jun

From Student Learner to Professional Learner: Training for Lifelong Learning through Online PBL

Posted by cognitivecomputing in Education, Learning. Tagged: , .

Problem-based learning (PBL) is a constructivist pedagogy in which students learn science and develop critical thinking skills by solving real-world problems in small groups. Studies have shown that PBL students are more motivated and become better learners. However, this pedagogy places additional demands on faculty. It takes time and expertise to develop suitable problems, to coach students, and to facilitate problem-solving sessions.

We are developing interactive computer systems incorporating the PBL approach which (1) help teachers design, enter, and share problems, and (2) support students and guide them through the PBL inquiry process and (3) assist teachers to continue their professional development by improving their domain knowledge. System development is guided by K-16 educators and tested in classrooms. Our goal is to enable educators to adopt this pedagogy in K-16 classrooms with minimal overhead and to assist them to effortlessly learn new technologies and new material.

Read the paper:

From Student Learner to Professional Learner: Training for Lifelong Learning through Online PBL

by Preetha Ram, Ashwin Ram, Chris Sprague

International Conference on Problem-Based Learning (PBL-05), Lahti, Finland, June 2005
www.cc.gatech.edu/faculty/ashwin/papers/er-05-03.pdf

15 May

Learning Adaptive Reactive Agents

Posted by cognitivecomputing in Agents, Learning, Robotics. Tagged: , , .

An autonomous agent is an intelligent system that has an ongoing interaction with a dynamic external world. It can perceive and act on the world through a set of limited sensors and effectors. Its most important characteristic is that it is forced to make decisions sequentially, one after another, during its entire “life”. The main objective of this dissertation is to study algorithms by which an autonomous agents can learn, using their own experience, to perform sequential decision-making efficiently and autonomously. The dissertation describes a framework for studying autonomous sequential decision-making consisting of three main elements: the agent, the environment, and the task. The agent attempts to control the environment by perceiving the environment and choosing actions in a sequential fashion. The environment is a dynamic system characterized by a state and its dynamics, a function that describes the evolution of the state given the agent’s actions. A task is a declarative description of the desired behavior the agent should exhibit as it interacts with the environment. The ultimate goal of the agent is to learn a policy or strategy for selecting actions that maximizes its expected benefit as defined by the task.

The dissertation focuses on sequential decision-making when the environment is characterized by continuous states and actions, and the agent has imperfect perception, incomplete knowledge, and limited computational resources. The main characteristic of the approach proposed in this dissertation is that the agent uses its previous experiences to improve estimates of the long-term benefit associated with the execution of specific actions. The agent uses these estimates to evaluate how desirable is to execute alternative actions and select the one that best balances the short- and long-term consequences, taking special consideration of the expected benefit associated with actions that accomplish new learning while making progress on the task.

The approach is based on novel methods that are specifically designed to address the problems associated with continuous domains, imperfect perception, incomplete knowledge, and limited computational resources. The approach is implemented using case-based techniques and extensively evaluated in simulated and real systems including autonomous mobile robots, pendulum swinging and balancing controllers, and other non-linear dynamic system controllers.

Read the thesis:

Learning Adaptive Reactive Agents

by Juan Carlos Santamaria

PhD Thesis, College of Computing, Georgia Institute of Technology, Atlanta, GA, 1996
www.cc.gatech.edu/faculty/ashwin/papers/git-cc-97-08.ps.Z