1. Introduction

Environmental uncertainty can carry relevant motivational consequences [1]. In an educational context, psychologists have known about the value of uncertain rewards in learning. Skinner demonstrated that manipulating the probability of reinforcement with behaviour ratio and time, proposing the schedules of reinforcement [2]. To further identify the unpredictable reinforcement to learning outcomes, studies adjusted varied reinforcement rates between a CS+ (a neutral stimuli before association) and US (a positive or negative stimuli before association), such as 20/80, 40/60 or 50/50 with reinforcers or without reinforcers. Evidence has shown moderate risk-taking (with 50 percent reward probability) heightens motivation, indicating that a mild uncertainty would enhance the learning process [3].

Neural psychologists also found unpredictable emotional arousal in associative learning paradigms [4]. Psychologists indicated that the predictability of an outcome influences the levels of dopamine released in mid-brain regions [5]. A primate experiment reported a peak level of dopamine in this area when the likelihood of receiving reward is about 50/50 [6]. Dopamine levels in the human mid-brain area have been studied to link with the motivation for approaching pleasures [7]. This result indicates that when dopamine level increase in the reward system of brain will produce the positive feelings during learning. Both previous neural studies and behavioural evidence confirmed the role of uncertainty during learning, encouraging psychologists to shed new light on the importance of unpredictable reinforcement in associative learning using neuroimaging techniques to determine the underlying mechanism of the association between uncertainty and learning.

We can understand this by using the method of cognitive contrast with the imaging technique, such as fMRI, providing virtual evidence. It is also important to better understand the neural statistics of environmental signal changes during the ongoing associative learning models with fMRI [8]. To image these processes, at least two experimental conditions were designed to contrast the cognitive function of interest. Neuroimaging is important to show both which brain area would be active and the amount of activation, providing scientific evidence for further research. Therefore, neural psychological studies with uncertainty contrast should be collected to show the brain activities regarding unpredictable reinforcement contracts with predictable reinforcement during learning. Whether the unpredictable reinforcement is attractive to school-aged children or as an interruption to them, this meta-analysis aims to review previous studies and provide a further understanding of the Educational value. According to the previous study, the uncertainty in associative learning is different from other paradigms such as reward uncertainty and decision-making. The uncertainty forms or removes the relationship between CS+ and US, while that in decision-making requires tolerance of an event with no particular pattern. Compared with associative learning, this uncertainty tolerance raises learners’ emotions and has an impact on their engagements. The second aim of this meta-analysis is to explore learners’ social-emotional experiences during uncertainty in learning and give suggestions for classroom practice to improve their engagement.

However, it raises a seemingly hypothetical application about the effect of uncertainty on instrumental learning in the classroom practice. In educational applications, social factors (for example, social status and self-esteem) play an important role when utilising different probabilities of reinforcement in associative learning [9]. In a previous study, students completed two cognitive risk-taking tasks with variable payoff contracts to fixed payoffs in either game or academic contexts, they exhibited a preference for low uncertainty when tackling academic tasks to avoid failure in front of teachers and peers [10].

It is also a concern that students choose high uncertainty in academic tasks in order to acquire higher recognition and esteem. Both arguments pointed out social factors may be more emotionally stimulating experiences regarding self-recognition in the school, either better or worse, which is associated with learning and then modulating the learning outcomes. Unpredictable positive reinforcement may raise children’s positive experiences, enhancing motivation during learning. Therefore, this study proposed that the arousal emotionally experienced due to uncertainty in the classroom may provide more cognitive resources to support memory formation and learning contexts.

2. Methods

The search on neuroimaging literature was conducted in August 2022 from PubMed, PsycInfo and Web of Science databases, searching for fMRI studies published from 2012. This meta-analysis focuses on commonly used learning paradigms. As few existing studies investigated the uncertainty tasks in educational circumstances because of the established “reward consistency” principle in school, not sufficient papers were found with the term “education”. Finally, the search equation applied to all the databases was: (uncertainty OR anticipation) AND (“association learning” OR “operant conditioning" OR "reinforcement") AND (“fMRI” OR "neuroimaging"). 123 studies were identified through the database searching, and 63 articles were left after duplicates were removed. The inclusion criteria used throughout this study are the following:

(1) Participants recruited in this study are physically healthy adults with no prior history of brain injury or neurological illness.

(2) Empirical studies conducted contrasts of uncertainty manipulation: uncertain vs certain, uncertain vs baseline, low uncertainty vs high uncertainty, or CS+ vs CS-.

(3) fMRI image acquisition was before and after the experiment periods.

According to these criteria, 2 reviews of previous studies were removed. 24 studies were excluded due to using patients with neurological disorders which are not matching with this research interest. This meta-analysis included fMRI studies only and excluded one EEG study and one PET study in order to maintain the homogeneity of the neuroimaging data which included in this research. Except for one article was conference abstract which cannot be accessed the full text, 34 full-text articles were assessed for eligibility. After reading, 11 articles didn’t present statistical contrasts of uncertainty manipulation as required and the associative learning task was not adopted in 14 studies. In addition, three studies reported Region of Interests (ROIs) only, rather than a whole-brain analysis result, which cannot be further analysed using an activation likelihood estimate (ALE) method. Therefore, a final total of 6 studies met the inclusive criteria for the current meta-analysis. The paper selection progress (Figure 1) is attached below.

Figure 1. the flowchart of the selection progress of studies included.

3. Data analysis

The activation likelihood estimate (ALE) algorithm is a coordinated-based meta-analytic method, utilizing random-effects analyses to identify agreements across studies and incorporates variances based on the sample sizes of each study [11]. The ALE algorithm weights studies with larger sample sizes, proposing these studies contributes more to approach the real activation foci.. Ginger ALE (version 2.3.6) is widely used for performing meta-analyses of human brain imaging studies with published coordinates in Talairach or MNI space. In the current study, these activation foci gathered from each study were transformed into MNI space to modulate a consistent format during ALE algorithm. For this study, ALE method was used to identify clusters activated and analyse the brain activation foci with the contrast between CS+ and CS- during association learning reported in the selected studies. The overlapping brain area could suggest the activation of specific part of the brain. The cluster forming threshold of the cluster level inference and type 1 error rate were both p < .05.

4. Results

The included paper were all fMRI experiments with contrast conditions: (a) Uncertainty vs. Certainty, (b) High Uncertainty vs. Low Uncertainty and (c) CS+ vs. CS-. Behavioural performances were reported by contracting the correct responses in certainty and uncertainty conditions separately. The percentage of accuracy in uncertainty conditions was significantly higher than that in certainty conditions, confirming that subjects paid attention to the uncertain reinforcers [4,12,13]. In addition, Schick [14] and [15] used aversive reinforcers and compared the subjective rating of valence and emotional arousal between CS+ and CS- conditions, concluding that conditioning ratings of the CS+ indicated significant unpleasantness compared to CS- ratings. These results demonstrated that more frequency of the reinforcers present would evoke a stronger emotional experience. Results in Schiffler (2016) found that simulations using low uncertainty reinforcement fit better with response accuracy than high uncertainty [16]. The response time in high-uncertainty conditions is much longer than that in low-uncertainty conditions, illustrating that more cognitive resources were required when facing high uncertain contexts.

Table 1. Average contribution of each experimental for significant clusters on associative learning.

This study also collected 143 activation foci from 6 previous experiments involving 162 participants, reported in Table 1. Data suggested a significant concentration of brain activation with 6976 mm³ from the left dorsal frontal lobe (-4,18,26) to the right dorsal frontal lobe (32,48,54), centred at (12.6,33.6,42.3), Z=3.48, p<0.05 at (2,26,46). The activation The analysis of uncertain and certain contrast showed significant concordance in activation primarily in medial frontal surfaces cluster that extended to bilateral dorsal prefrontal cortex. Brain activation associated with this learning pattern was found by this meta-analysis in lateral prefrontal cortex (lateral PFC), with a large cluster in bilateral medial frontal gyrus (MFG), extending dorsally into superior frontal gyrus (SFG), as well as ventral into middle frontal gyrus with a smaller cluster. However, only one study about high uncertainty and low uncertainty contrast was included. This analysis revealed no brain area exhibiting activation following this contrast. The findings illustrated that the brain area of dorsal frontal lobe were generally activated under the condition of Uncertainty vs. Certainty, and CS+ vs. CS-. No brain regions were found to be systematically engaged during High uncertainty vs. Low uncertainty (Table 2).

Table 2. Regions of overlap between the contrasts.

Note: BA = Brodmann area; R = Right; L = Left. All results are set to a threshold at P(cluster-FWE) = 0.05 with a cluster-forming threshold of P < 0.05 using 10,000 permutations.

5. Discussion

The current neuropsychological concept of "tolerance of uncertainty" predicts a physiological enhancement of incentive value with repeated stimulus [17,18]. This result provides a viewpoint that emotional responses under uncertainty are more intense compared to certainty. It remains unclear about the valence of the emotional arousal following the uncertain stimulus. Psychologists worried about the uncertainty creates frustration and feelings of unfairness, especially that negative feelings are out of control. Therefore, the established pedagogical philosophy of "reward consistency" conflicts with the idea of rewarding due to chance, which believed that frustration undermined the learners interests. A previous survey of children dialogue disclosed a new angle of this debate as students looked forward to getting an unpredictable reinforcement because it was exciting. Psychologists gradually tend to use terms of ‘liking’ and ‘exciting’ to describe the emotions children experience under uncertainty. A suggestion for future study is to explore whether the negative feelings of frustration would lead to better memory in an educational situations.

One of the selected paper by Harrison et al. [12] suggest a link between vmPFC activity and the positive affective processing of safety signals, which is likely to evoke a greater sense of reward when receiving positive reinforcement and a greater relief when the punishment is unpaired. Their results resonate with the other previous studies of vmPFC that this brain region is robustly correlated with wakeful relaxation states, such as a sense of relief, comfort and relaxation [19]. This provides a perspective to pay more attention to the emotional values during learning.

In educational practice, unanticipated rewards or encouraging feedback for students who are academic performance well influence affective response during answering questions and receiving of feedback. These positive affective responses were shown as a model of positive prediction error (PPE), simplified as ‘happy surprise’, which is the extent to which an outcome is greater than expected. Previous studies believed that PPE entails an increased neural response that can be attributed to the experience of a surprising, unexpected rewards [20]. This study put forward that the activation in PFC is associated with positive prediction error and feelings of surprise. Future studies should focus on the learning and attention process with positive feelings compared with neutral emotions.

Psychologists also found dopamine-related activities in the reward system when receiving the unpredictable outcomes [21]. Future psychologists should draw more attention on teachers feedback, and how to establish a supportive educational atmosphere, such as surprising exclamation and gestures present in classroom practice. In addition to positive emotional experience, researchers concerned more about whether chance-based uncertainty survives learning or interrupts the association between academic performance and assessment outcomes. Some researchers argued about whether the uncertainty simply provided a ‘sugar goat’ which evoked the emotional arousal restricted to unanticipated reward while the learning process remained unaffected [14]. The BOLD responses revealed that the ambiguous tones were associated with the highest activation in the anterior cingulate cortex. However, no significant correlation between prediction error and brain activities to uncertainty stimulus was found in this study. Another study showed prediction error signals in the dorsal anterior cingulate cortex (dACC) and vmPFC, which correspond with the higher cognitive functions and mood regulation [22]. These inconsistent results call for more neural evidence to find out the brain regions activated by prediction error and associated cognitive processes.

Further neural and behavioral studies need to examine whether PPE survives the learning progress rather than arouses emotional experiences only. Neural images of PFC activities should be collected when learning process is associated with surprising feedback or rewards, investigating whether students active the brain regions of learning process and attention. Due to ethical considerations, it may be less practical for children to do a brain scanning with Fmri. Children would be hard to lie still in fMRI equipment and perform unnaturally as stay in an artificial environment. Therefore, some behavioral evidence could also be important, comparing students assessment outcomes before and after intervention of unanticipated exclamation (e.g. ‘That is great!’) and gestures (e.g. Thumbs up). It is noticeable that both verbal and non-verbal praises should less related to social status and protect their self-esteem, avoiding comparison with their peers. At the same time, another consideration of fairness drawn researchers attention. Chance-based events would introduce prejudice and frustrating into learning, impacting students engagement in both behavioral and emotional. It emphasizes teachers should reduce students feeling of loss or low self-esteem associated with learning.

As lacking another overlapping clusters found in this meta-analysis, brain activities during the uncertainty process may be more variable and occur in more regions. The cognitive activities associated with unpredictable outcomes during learning remain unrevealed. Here, researchers should take social factors into consideration carefully since it takes an important role in educational environment. Researchers will need to explore the students emotional experience when receiving unanticipated positive outcomes and their subsequent interactions with teachers and peers.

6. Conclusion

All the brain activation in this meta-analysis was found in the frontal lobe which takes role of processing emotions and cognitive resources. This finding examined our hypothesis that uncertain reinforcement raises emotional experience which provides more cognitive resources to support memory formation and learning contexts. This conclusion can be applied in the educational context.

In this study, the researcher linked the prefrontal lobe with the integration of bottom-up information input and top-down predictions, which act as a mediate center that integrates prior knowledge with incoming information [4]. It responds to adopting new knowledge into individuals previous experience and anticipating the following stimulus. In educational situations, how the sense of the general attractiveness of uncertainty corresponds to both novel and repeated exposures remains discussion.