Human Modeling Approaches vs the Learner-centered Approach

1) How are the two agents similar and different in their a) design of the agent and b) interaction with the learner?

Betty's Brain

The teachable agent examined by Schwartz et al. (2009), Betty's Brain, allows students to apply interactive metacognition, where the act of teaching Betty helps learners strengthen their own metacognitive skills and results in better learning outcomes. 

Betty's Brain was "designed for knowledge domains where qualitative causal chains are a useful structural abstraction (e.g., the life sciences) (Schwartz et al., 2009, p. 5). When interacting with Betty, students teach her by "creating a concept map of nodes connected by qualitative causal links" (Schwartz et al., 2009, p. 5). Betty answers and asks questions based on her understanding of the concepts. She can also animate her reasoning process, which enables users to track her reasoning and "remediate" her knowledge if necessary. (Schwartz et al., 2009, p. 5).

Features like the ability to draw inferences, create narratives and graphics, and be customized with learner-specific personas (appearance and name), allow for the semblance of sentience and a more personalized learning experience. Perhaps most importantly, Betty "externalizes her thoughts processes," which better enables learners to apply metacognition the agent's thinking (Schwartz et al., 2009, p. 6).

Liza

In their article, Le & Wartschinski (2018), discuss the cognitive agent LIZA that is aimed at "improving the reasoning and decision making abilities of its users" (p. 45). LIZA is designed to reduce "heuristics and biases, specifically those described in established research in psychology," and would be the first "dialog oriented trainer for this specific learning domain" (Le & Wartschinski, 2018, p. 46). 

Basically, LIZA is a "virtual human that interacts with users via text-based natural language" (Le & Wartschinski, 2018, p. 46). LIZA can explain topics in the field of reasoning, guide users through a series of questions, review their understanding, and provide feedback on their mistakes. 

2) What are the theoretical underpinnings for each agent, and what is the rationale given?

Betty's Brain

According to Schwartz et al., (2009), Betty is "not only means of instruction, but rather, she provides a way to help students organize reason about the content they have learned" (p. 6). Betty's ability to appear sentient helps users to feel like they're interacting with a being who is capable of exhibiting cognitive processes. That results in increased feelings of accountability when tasked with Betty's learning. Through the process of monitoring and evaluating Betty's understanding, students also revise and cultivate their own reasoning, so that they can better teach Betty. 

Liza

As shared by Le & Wartschinski (2018), cognitive science distinguishes two types of rationality: "epistemic rationality, which is defined as how well beliefs about the world correspond to the actual structure of the world, and instrumental rationality, which is nothing more than to pick whatever behavior is best suited to reach certain goals" (pg. 46). Rational actions are those that "maximize an individual's goal fulfillment by making the best use of physical and mental resources" (Le & Wartschinski, 2018, p. 46). However, in real life, people will often trade accuracy for low effort and employ heuristics to help them make decisions, which can often result in faulty reasoning. Le & Wartschinski (2018) reason that "a method for increasing somebody's rationality would therefore be to train deliberate usage of Type 2 processes" (p. 46). 

By sharing reasoning tasks housed as story questions around 7 common heuristics and bias topics that affect rationality, LIZA aims to help learners increase their understanding of cognitive biases, raise awareness for possible fallacies, and improve their reasoning calibration (Le & Wartschinski, 2018, p. 47). LIZA frames herself as a "mentor" during her introduction to the user and asks the user questions to allow her to customize the story she shares. 

3) What are some of the benefits and limitations of each approach in the two agents? How do you think the agents will evolve with the recent trends in AI?

Betty's Brain

In the TA condition of the Gameshow, Schwartz et al. (2009) found that students "monitor[ed] and acknowledge[ed] errors more closely than [in] the Student condition" (p. 9). Students in the TA condition were also more motivated to engage in the extra work that metacognition entails, perhaps prompted by their feelings of guilt around their TA's failures. 

I was specifically interested in the idea that because TA knowledge correlates with student knowledge, it may be possible to eventually assess TA's rather than students. I could see this extra time opening up a world of opportunities for both teachers and learners in the classroom. Additionally, the classroom assessment study by Schwartz et al. (2009) seemed to indicate that long-term engagement with the TA resulted in a "steady improvement in length of causal inference," perhaps due to the consistent modeling and reinforcement by the agent's reasoning (p. 14). 

SRL Betty and the mentor agent, Mr. Davis, represent an adaptive tutoring system. Schwartz et al. (2009) found that the "self-regulation support in SRL Betty help[ed] students learn science content better" (p. 16). I thought it was particularly significant that just the belief that students are teaching an agent leads to superior learning, even when the tools and feedback are the same (Schwartz et al., 2009, p. 18). 

Liza

LIZA specifically trains humans on the following seven heuristics or biases that "frequently lead to irrational thoughts and decisions: 1) sunk cost fallacy, 2) gambler's fallacy,  3) Bayesian reasoning, 4) belief bias in syllogistic reasoning, 5) regression towards the mean, 6) co-variation detection, and 7) Wason's selection tasks (Le & Wartschinski, 2018, p. 47).

LIZA is able to provide feedback that helps to guide a user's learning and deliver recommendations at the end of the interaction. She is also designed to accommodate misunderstandings or ask for statements to be rephrased. Le & Wartschinski (2018) found that LIZA was "clearly more effective than an online course on the same topics" and that it "demonstrate[ed] significant improvement of reasoning performance in most of the tasks" (p. 53). 

In the future...

I could see both agents evolving in their available level of customization and their physical manifestation. Users may be able to customize their agents to represent a favorite teacher or have the voice of a parent. I can also see greater variety being employed in terms of available languages etc. Perhaps instead of appearing as digital entities, physical bodies may be created for these agents to allow them to incorporate embodied cognition. Additionally, agents adaptability and agility will undoubtedly increase, where agents are able to effectively respond to a wider range of student statements. 

For Betty, I could see her content being extended beyond 5th grade Science. Additionally, I could see this learning model being employed for various target audiences. For LIZA, I could see additional heuristics and biases being brought in and the same structure being applied to different topics, perhaps things like ethics or morality. 

References

Koedinger, K. R. & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive

    tutors. Educational Psychology Review, 19, 239-264. doi: 10.1007/s10648-007-9049-0. 

Le, N. & Wartschinshi, L. (2018). A cognitive assistant for improving human reasoning skills. 

    

    International Journal of Human-Computer Studies, 117, 45-54. 

Schwartz, D. L., Chase, C., Chin, D. B., Oppezzo, M., Kwong, H., Okita, S., Biswas, G., Roscoe, R., 

    

    Jeong, H., & Wagster, J. (2009). Interactive metacognition: Monitoring and regulating a teachable 

    

    agent. Handbook of metacognition in education, 340-358.