<table>
<tbody>
<tr class="odd">
<td>ARTICLE INFO</td>
<td></td>
<td>ABSTRACT</td>
</tr>
<tr class="even">
<td><em>Article history:</em></td>
<td></td>
<td>This paper discusses the process of extracting the design principles of the gamified e-assessment for low achievers in introductory programming. By focusing on the integration of gamification and assessment design principles in e-learning platforms for learning programming, this study uses comparative analysis to extract the suitable design principles among ten existing studies that can be applied in the design of the gamified e-assessments for low achievers. The comparative analysis comprises of three main phases, which are i) literature search, ii) elicitation and filtration, and iii) review, analysis and extraction of the design principles. Through this qualitative approach, the gamification principles such as the achievement, progression and rules and challenges have been extracted together with the elements of levels, points, badges, progress bars, and leader board. The assessment principles have also been derived consisting of the assessment structure and assessment composition with sub principles of problem-based instructional strategy, formative assessment, post-test strategy, assessment levels, and feedback strategy. The proposed design principles have also been demonstrated through the application in the gamified e-assessment module in an e-learning system.</td>
</tr>
<tr class="odd">
<td><p><em>Keywords:</em></p>
<p>design principles</p>
<p>gamification</p>
<p>assessment</p>
<p>e-learning</p>
<p>low achievers</p>
<p>introductory programming</p>
<p><em>DOI:</em></p></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

# introduction

In designing instructional materials for e-learning, it is important to understand the framework of e-learning. Khan (2010) proposes a framework for flexible e-learning that highlights eight dimensions of e-learning, including institutional, management, technological, pedagogical, ethical, resource support, user interface and evaluation. Each dimension has its own focus that contributes to effective learning in an e-learning environment, such as the institutional dimension deals with administrative affairs, academic affairs and others related to student services affairs (Khan, 2010; Haron et al., 2019; Bekele et al., 2022). The management aspect deals with the e-learning system’s maintenance and information distribution (Khan, 2010; Angelia et al., 2020). Meanwhile, the ethical dimension refers to social, political, etiquette, and legal

issues (Khan, 2010; Haron et al., 2019; Bekele et al., 2022). The technological aspects deal with the infrastructure, hardware and software. Resource support refers to the online support and resources required for e-learning (Khan, 2010; Haron et al., 2019). In designing instructional materials for any learning environment, the three main aspects that are crucial in enhancing user experiences are the pedagogical, user

interface and evaluation dimensions (Khan, 2010; Angelia et al., 2020). This paper focuses on the evaluation dimension where the assessment for the e-learning system for introductory programming course that are suitable for low-achieving students was designed based on the gamification and assessment principles derived from the comparative analyses of previous studies. This paper also showcases how the proposed design principles were applied in the e-assessment for introductory programming to cater to the low achievers’ learning requirements.

# Literature review

## Low Achievers in Introductory Programming

Among novices or beginners in programming, learning introductory programming has always been challenging for many, especially for low-achieving students (Kadar et al., 2021; Othman et al., 2022). Low-achieving students in introductory programming courses can be categorised as students who are slow learners, have low motivation and interest in this subject (Alshammari, 2019; Kadar et al., 2021; Othman et al.,2022). The difficulties that they are facing while learning this subject may also cause them to be less motivated and drop out of this course (Mehmood et al., 2020; Kadar et al., 2021). Other than that, the challenges in trying to grasp the fundamental concepts of introductory programming often cause them to be left behind compared to their high achievers’ colleagues (Margulieux et al, 2020; Othman et al.,2022). Despite this, over the years, academicians have strived to overcome this issue by providing the low-achieving students with different teaching and learning approaches, especially with the help of educational technologies such as mobile applications, e-learning systems, interactive multimedia, and gamification (Kamunya et al., 2020; Christopher & Waworuntu, 2021). Nonetheless, designing instructional technologies with design principles suitable for the low-achieving students in introductory programming remains unexplored, especially in designing the gamified e-assessments, where e-assessments are integrated with gamification and assessment principles. The idea of such integration comes from the fact that to increase students’ motivation and gain their interest, the assessment module in the e-learning platform can be gamified with elements that engage students’ attention and encourage them to self-monitor their progress. For example, Imran (2023) utilises elements of points, badges, levels and leader board in the gamified e-assessments as the students answer the challenges and progress through each level. The e-assessment also provides feedback mechanisms that was found useful for the students to stay motivated when feedback was given automatically and promptly (Imran, 2023). Other than that, Nadja (2022) also integrates gamification principles in the design of the e-assessment where each assessment has different levels of difficulties. Furthermore, the instructional design of the e-assessment for the introductory programming course must also uses problem-based instructional strategy where real-world problems were used to improve students’ ability in providing possible solutions (Alshammari, 2019; Nadja, 2022).

## Gamification Design in E-Learning for Programming

Gamification, defined by Deterding et al. (2011) is "the process of adding game like elements to activities that are not normally game like". Over the years, gamification in the field of instructional technology has gained many interests among academicians. For instance, Christopher and Waworuntu (2021) applied Chao (2015) Octalysis Framework in the development of an online gamified system for teaching Java programming. Chou identified eight core motives namely epic meaning and calling, development and accomplishment, empowerment of creativity and feedback, ownership and possession, social influence and relatedness, scarcity and impatience, unpredictability and curiosity, and loss and avoidance. Christopher and Waworuntu (2021) have incorporated the game elements which include leader board, level, points, and virtual goods in an Android application to teach Java programming. The app was tried on second year students and the students’ trial participation revealed and their interest in this subject has increased. Another research done by Khaleel et al. (2019) engaged the MDA framework by Hunicke et al. (2004) to develop a gamified learning website for Java programming. This paper also explained how game mechanics, which are the tools embedded in the programming content proved to improve the students’ motivation. Meanwhile, game dynamics, which are the user’s interactions with the game mechanics, and the aesthetics of the app, which are the user’s emotions during the interaction also played a role in creating a good user interface. The MDA framework was also used to develop an online learning system which incorporated several game elements such as points, badges, levels and leader board.

In designing the gamified e-assessment for introductory programming, it is essential in determining the gamification principles and elements that are suitable to be applied in the context of e-learning environment. Some of the most preferred gamification principles for e-learning systems are rules and challenges, achievement, progression, narrative and sensation (Khaleel et al., 2019; Nadja, 2022). Meanwhile, gamification elements that have been applied in many gamified e-learning systems are points, badges, levels, leader board, Avatar, gifts or virtual goods (Kamunya et al., 2020; Christopher & Waworuntu, 2021). Nevertheless, in terms of designing the gamified e-assessment that focuses on low achievers in introductory programming, it requires in depth investigation and analysis to determine whether these gamification principles and elements are appropriate for the target users. Further discussions on extracting the gamification design principles for these target users will be discussed in the next section.

## Assessment Design in E-Learning for Programming

Shalatska et al. (2020) emphasize that when designing assessments for an e-learning environment, it is important to consider the assessment structure and composition, which should align with the target audience and the objectives of the assessment. The design should outline the skills to be assessed, offer necessary knowledge and feedback, and provide clear instructions along with explicit assessment criteria, specific characteristics, and settings. The assessment structure pertains to the overall instructional strategies and assessment formats, while the assessment composition focuses on the specific content or tasks involved, including cognitive levels, formats, and feedback strategies (Shalatska et al., 2020; Scherer et al., 2020; Santos, 2023).

In designing the assessment in the e-learning platform especially to learn introductory programming for low achievers, several factors need to be addressed. For instance, the principles for assessment structures must consists of problem-based instructional strategy that emphasises on real-world case scenarios (Scherer et al., 2020; Santos, 2023). Other than that, determining the type of assessment is also important, such as providing the summative or formative assessments. As summative assessments deal with the grading and certification, formative assessments seem to be more appropriate to be implemented in the gamified e-learning environment for learning introductory programming (Pitoyo et al., 2020). Formative assessments such as e-quizzes, short-structured questions, multiple choices, sequencing and arranging, true or false questions are more suitable to assess low achievers’ knowledge and understanding in introductory programming (Pitoyo et al., 2020). Other than that, for the principles of assessment composition, factors such as post-test strategy, assessment levels and feedback strategies are also essential for the gamified e-assessment. Post-test strategy is needed as students need to be assessed after the learning have occurred (Zaharias, 2009). Meanwhile, assessment levels by utilising the Revised Bloom’s Taxonomy Cognitive Domain (RBT) are also useful in order to show students’ progress in solving introductory programming problems (Santos, 2023). Furthermore, the feedback strategies are useful to provide meaningful learning experiences through feedback mechanisms (Lane et al., 2015; Pérez-Clark et al., 2020).

# METHODOLOGY

A comparative analysis approach by Ahmad and Abdul Mutalib (2015) was adapted in the methodology. This qualitative method includes three main stages, which are i) literature review, ii) elicitation and filtering, and iii) review, analysis, and extraction of design principles.

1)  > Literature search

To find articles related to the design principles of a gamified e-assessment in an e-learning platform, an extensive literature search was conducted throughout multiple literature databases, such as Web of Science, Scopus, IEEE Xplore, and Google Scholar. For advanced searches, other keywords have been used, such as “introductory programming,” “novices,” “low achievers,” “e-learning,” “gamification,” “gamification principles,” “gamification elements,” “assessment,” “e-assessment,” and “gamified assessment”. All related articles identified must also date from 2018 until 2023 and are primarily focusing on the design principles of the gamified e-assessment for introductory programming course.

2)  > Elicitation and filtration

Initially, over 50 articles were chosen based on the previously mentioned keywords. However, to identify the most relevant articles for this research, specific themes were established. These themes required that the selected studies: i) focus on e-learning for introductory programming courses in higher education institutions, ii) involve novices or first-year students in these courses, iii) address the gamification principles and elements applied, and iv) discuss the assessment design used in the studies, in terms of assessment structure and assessment composition that include problem-based instructional strategy, formative assessment, post-test strategy, assessment levels, and feedback strategy. As a result, only ten articles met these criteria and were selected for further review, analysis, and extraction of the design principles. These ten articles thoroughly examined the use of gamified e-assessments in e-learning for introductory programming in higher learning institutions, which significantly influenced the direction of this study.

3)  > Review, analysis and extraction of design principles

Further review and analysis were conducted to ten selected articles by focusing to these pre-determined criteria, i) gamification principles, ii) gamification elements, and iii) assessment principles, with the assessment structure and assessment composition including criteria mentioned above. All the gamification principles and elements, and the assessment principles were then extracted to be applied in the proposed gamified e-assessment for low achievers in introductory programming. Next section will discuss the findings from the review, analysis and extraction of the design principles phase.

# FINDINGS

Table 1 depicts all ten selected studies after the thorough elicitation and filtration of the articles.

Table 1. All ten selected studies

| No | Author                  | Studies                                                           |
| -- | ----------------------- | ----------------------------------------------------------------- |
| 1  | Imran (2023)            | A Gamified Learning Environment Model                             |
| 2  | Nadja (2022)            | A Personalized Gamification Design Model (PeGAM)                  |
| 3  | Poonsawad et al. (2022) | Problem-based Interactive Digital Story Learning Model            |
| 4  | Alsubhi et al. (2021)   | Engagement Framework for E-learning Gamification                  |
| 5  | Kamunya et al. (2020)   | An Adaptive Gamification Model of E-learning                      |
| 6  | Winanti et al. (2020)   | Gamification Framework for Programming Course in Higher Education |
| 7  | Alshammari (2019)       | Gamification Design Model for E-learning                          |
| 8  | Khaleel et al. (2019)   | Gamification-based Learning Framework for a Programming Course    |
| 9  | Padirayon (2019)        | Gamification Application Architecture and Elements                |
| 10 | Piteira et al. (2018)   | Conceptual Framework of Gamification on Online Courses            |

Meanwhile, Table 2 shows the findings from the detailed comparative analysis that analyse the gamification principles, gamification elements, and assessment structure and assessment composition from the selected studies.

Table 1. The findings of detailed comparative analysis

<table>
<thead>
<tr class="header">
<th>No</th>
<th>Author</th>
<th>Gamification Principles</th>
<th>Gamification Elements</th>
<th>Assessment Structure</th>
<th>Assessment Composition</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>1</td>
<td>Imran (2023)</td>
<td>Progression, achievement, rules and challenges, narrative</td>
<td>Points, badges, levels, leaderboard, progress bar</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="even">
<td>2</td>
<td>Nadja (2022)</td>
<td>Progression, achievement, sensation</td>
<td>Badges, leaderboard, progress bar, virtual goods</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="odd">
<td>3</td>
<td>Poonsawad et al. (2022)</td>
<td><p>Rules and challenges, achievement</p>
<p>narrative, sensation</p></td>
<td>Points, badges, levels, leaderboard, progress bar</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>feedback strategy</p></td>
</tr>
<tr class="even">
<td>4</td>
<td>Alsubhi et al. (2021)</td>
<td>Progression, achievement, rules and challenges</td>
<td>Points, badges, levels, leaderboard, progress bar, timer</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="odd">
<td>5</td>
<td>Kamunya et al. (2020)</td>
<td>Rules and challenges, rewards, achievement, competition, altruism</td>
<td>Badges, leaderboard, progress bar, virtual goods</td>
<td>Formative assessment</td>
<td>Assessment levels, feedback strategy</td>
</tr>
<tr class="even">
<td>6</td>
<td>Winanti et al. (2020)</td>
<td>Progression, achievement, rules and challenges</td>
<td>Points, badges, levels, leaderboard</td>
<td>Formative assessment</td>
<td><p>Post-test strategy,</p>
<p>feedback strategy</p></td>
</tr>
<tr class="odd">
<td>7</td>
<td>Alshammari (2019)</td>
<td>Rules and challenges, rewards, achievement, progression</td>
<td>Points, badges, levels, leaderboard, timer</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="even">
<td>8</td>
<td>Khaleel et al. (2019)</td>
<td>Progression, achievement, rules and challenges, altruism,</td>
<td>Points, badges, levels, leaderboard, progress bar</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="odd">
<td>9</td>
<td>Padirayon (2019)</td>
<td>Progression, achievement, rules and challenges,</td>
<td>Points, badges, levels, leaderboard, progress bar</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
<tr class="even">
<td>10</td>
<td>Piteira et al. (2018)</td>
<td>Progression, achievement, rules and challenges, narrative,</td>
<td>Points, badges, levels, leaderboard, progress bar</td>
<td><p>Problem-based instructional strategy,</p>
<p>formative assessment</p></td>
<td><p>Post-test strategy,</p>
<p>assessment levels, feedback strategy</p></td>
</tr>
</tbody>
</table>

From Table 2, the design principles that are deemed appropriate to be applied in the proposed gamified e-assessment for low achievers in introductory programming are shown in Table 3, Table 4 and Table 5. Table 3 shows the extracted gamification principles that are suitable to be applied, such as principles of achievement, progression, and rules and challenges.

<table>
<thead>
<tr class="header">
<th><p>Table 3. Extracted gamification principles</p>
<table>
<thead>
<tr class="header">
<th><strong>No</strong></th>
<th><blockquote>
<p><strong>Gamification Principles</strong></p>
</blockquote></th>
<th><strong>Description</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><blockquote>
<p>1</p>
</blockquote></td>
<td><blockquote>
<p>Achievement</p>
</blockquote></td>
<td>Achievement is represented as completing each level of challenge. Digital badges, points, and leaderboards can represent a students’ achievements or accomplishments.</td>
</tr>
<tr class="even">
<td><blockquote>
<p>2</p>
</blockquote></td>
<td><blockquote>
<p>Progression</p>
</blockquote></td>
<td>Progression principle demonstrates students’ engagement in order to keep them motivated and remain interested. A student’s progress can be represented using progress bars and unlocking levels in each e-assessment.</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>3</p>
</blockquote></td>
<td><blockquote>
<p>Rules and challenges</p>
</blockquote></td>
<td>Rules are represented as defined goal settings, telling users what and how to achieve in each assessment. Assessments can also be seen as challenges or obstacles where students need to overcome by unlocking levels to progress to the next level.</td>
</tr>
</tbody>
</table></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><p>Meanwhile, the gamification elements that are identified based on the gamification principles extracted earlier are points, badges, levels, progress bar and leader board. These elements as listed in Table 4 represent the game mechanics that respond to each achievement, progression, and rules and challenges applied in the e-assessments.</p>
<p>Table 4. Extracted gamification elements</p>
<table>
<thead>
<tr class="header">
<th><blockquote>
<p><strong>No</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Gamification Elements</strong></p>
</blockquote></th>
<th><strong>Description</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><blockquote>
<p>1</p>
</blockquote></td>
<td><blockquote>
<p>Points</p>
</blockquote></td>
<td>Points are rewarded when a learner accomplishes a mission, such as completing a challenge or assessment. It can be accumulated based on the rules and challenges designed in the gamified e-assessment.</td>
</tr>
<tr class="even">
<td><blockquote>
<p>2</p>
</blockquote></td>
<td><blockquote>
<p>Badges</p>
</blockquote></td>
<td>Badges are rewarded when a learner accomplishes a mission, such as completing a challenge or assessment. It can be accumulated based on the rules and challenges set in the gamified e-assessment.</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>3</p>
</blockquote></td>
<td><blockquote>
<p>Levels</p>
</blockquote></td>
<td>Levels represent the learner’s advancement in the gamified e-assessment. When applied in the assessment, levels can be categorised based on difficulties, from easy to intermediate to hard levels of challenges.</td>
</tr>
<tr class="even">
<td><blockquote>
<p>4</p>
</blockquote></td>
<td><blockquote>
<p>Progress bars</p>
</blockquote></td>
<td>The progress bar is a virtual representation of a learner’s linear progression as he or she unlocks the challenges or assessments. It helps to engage students with their progress and achievement.</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>5</p>
</blockquote></td>
<td><blockquote>
<p>Leader board</p>
</blockquote></td>
<td>The leaderboard displays students’ achievements and compares accumulated points or badges. It invokes a sense of competition among students.</td>
</tr>
</tbody>
</table></td>
</tr>
</tbody>
</table>

Lastly, Table 5 shows the extracted assessment principles that will be integrated with the gamification principles and elements for the proposed gamified e-assessment.

Table 5. Extracted assessment principles

<table>
<thead>
<tr class="header">
<th><blockquote>
<p><strong>No</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Assessment Principles</strong></p>
</blockquote></th>
<th><strong>Sub-principles</strong></th>
<th><strong>Description</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><blockquote>
<p>1</p>
</blockquote></td>
<td><blockquote>
<p>Assessment Structure</p>
</blockquote></td>
<td>Problem-based instructional strategy</td>
<td>The structure of the e-assessment must include real-world problems to be solved, from easy to hard questions.</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td>Formative assessment</td>
<td>Using formative e-assessments such as e- quizzes with multiple-choice questions, drag and drop and short-structured questions.</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>2</p>
</blockquote></td>
<td><blockquote>
<p>Assessment Composition</p>
</blockquote></td>
<td>Post-test strategy</td>
<td>The e-assessments must adequately measure the learning objectives' accomplishment.</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td>Assessment levels</td>
<td>The e-assessments must include RBT Cognitive Domains to measure the low-achieving students’ cognitive performances according to the cognitive levels.</td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td>Feedback strategy</td>
<td>Each e-assessment must provide feedback to the students, which is given at any specific time tailored to the content being studied, the problem being solved, or the task being completed by the learner.</td>
</tr>
</tbody>
</table>

# THE application of the proposed DESIGN PRINCIPLES OF gamified e-assessment 

![](67638542150ea_media/media/image1.png)This section shows some of the examples of the application of the proposed gamified e-assessment in an e-learning system based on the extracted design principles of gamification and assessment mentioned previously. Fig. 1 demonstrates the application of gamification principles of achievement, progression, and rules and challenges with the elements of levels, progress bars, points, badges and leader board. Figure 1 also depicts the application of the assessment principles of assessment structure and composition with sub principles of problem-based instructional strategy, formative assessment, post-test strategy, assessment levels and feedback strategy in the gamified e-assessment.

Fig. 1. Samples of application of the extracted gamification principles and elements, and assessment principles in the proposed gamified e-assessment for low achievers in introductory programming.

# conclusion

Designing e-assessments that are suitable for certain group of students in the e-learning systems have always been the interest of many academicians. Nevertheless, designing gamified e-assessments for low achievers in introductory programming possess a challenge for the course instructor. Despite this, through comparative analysis approach comprising of three main phases, the proposed design principles for the gamified e-assessment focusing on the low achievers in introductory programming can be obtained. By investigating and analysing existing studies that suit the theme of the search and filtration, the gamification principles such as principles of achievement, progression , and rules and challenges can be extracted. In addition to the selected principles, the gamification elements that suit the principles can also be determined, such as points, badges, levels, progress bars, and leader boards. Other than that, as this study emphasises on the integration of gamification principles and assessment principles, through the extensive work of comparative analysis, the assessment principles have also been derived. Based on the findings, the assessment principles consisting of assessment structure and assessment composition, which then lead to the extraction of the sub principles that include problem-based instructional strategy, formative assessment, post-test strategy, assessment levels and feedback strategy. All these proposed design principles have been successfully applied in the design of the gamified e-assessment in an e-learning system as shown in the previous section. This shows that the proposed design principles with gamification and assessment principles for the e-learning system for low achievers in introductory programming can be effectively implemented. Future research will involve in the development of the prototype of the gamified e-learning system and experimental test with the target users.

# Acknowledgements/Funding

# Conflict of interest statement

# 

# Authors’ contributions

# References

Ahmad, S. Z., & Abdul Mutalib, A. (2015). Exploring Computer Assisted Learning for Low Achieving Children: A Comparative Analysis Study. Jurnal Teknologi (Sciences & Engineering) 77:29, 1–7.

Alshammari, M. T. (2019). Design and Learning Effectiveness Evaluation of Gamification in e-Learning Systems, International Journal of Advanced Computer Science and Applications (IJACSA), 10(9), 2019. http://dx.doi.org/10.14569/IJACSA.2019.0100926.

Alsubhi, M. A., Ashaari, N. S. & Wook, T.S.M.T. (2021). Design and evaluation of an engagement framework for e-learning gamification, International Journal of Advanced Computer Science and Applications (IJACSA), 12(9), http://dx.doi.org/10.14569/IJACSA.2021.0120947.

Angelia, F., Suharjito, S. & Ali, S.M. (2020). Improving English Learning by Gamification with MDA Framework. Special Issue: International Conference of Games, Game Art and Gamification (ICGGAG2018), Vol. 5 No. 2, pp. 33 – 40. .

Bekele, T. A., Karkouti, I. M., & Amponsah, S. (2022). Core conceptual features of successful blended learning in higher education: Policy implications. Education Policy Analysis Archives, 30, (156). <https://doi.org/10.14507/epaa.30.7444>.

Chao, Y, K. (2015). Actionable Gamification Beyond Points, Badges, and Leaderboard. NY: LeanPub.

Christopher, L., & Waworuntu, A. (2021). Java Programming Language Learning Application Based on Octalysis Gamification Framework. IJNMT (International Journal of New Media Technology), 8(1), 65-69. <https://doi.org/https://doi.org/10.31937/ijnmt.v8i1.2049>.

Deterding, S., Dixon, D., Khaled, R. & Nacke, L. (2014). From game design elements to gamefulness, Proc. 15th Int. Acad. MindTrek Conf. Envisioning Futur. Media Environ. - MindTrek ’11, March 2014, p.9.

Haron, H., Yusof, A. R. M., Samad, H., Ismail, N., Juanita, A., Yusof, H. (2019). The Platform of MOOC (Massive Open Online Course) on Open Learning: Issues and Challenges. International Journal of Modern Education, 1(3), 01-09.

Hunicke, R., LeBlanc, M. & Zubek, R. (2004). MDA: A Formal Approach to Game Design and Game Research. Northwestern University.

Imran, H. (2023). An Empirical Investigation of the Different Levels of Gamification in an Introductory Programming Course. Journal of Educational Computing Research, 61(4), 847-874. https://doi.org/10.1177/07356331221144074.

Kadar, R., Wahab, N. A., Othman, J., Shamsuddin, M., & Mahlan, S. B. (2021). A Study of Difficulties in Teaching and Learning Programming: A Systematic Literature Review. International Journal of Academic Research in Progressive Education and Development, 10(3), pp. 591–605.

Kamunya, S., Mirirti, E., Oboko, R. & Maina, E. (2020). An Adaptive Model for E-Learning, 2020 IST-Africa Conference (IST-Africa), pp. 1-10.

Khaleel, F.L., Ashaari, N.S. & Wook, T.S.M.T. (2019). An empirical study on gamification for learning programming language website, J. Teknol., vol. 81, no. 2, pp. 151–162, 2019.

Khan, B. H. (2010). The global e-learning framework. In S. Mishra (Ed.), E-learning stride handbook, 8th ed., pp. 42–51. Indira Gandhi National Open University.

Lane, K. L., Menzies, H. M, Ennis, R. P., & Oakes, W. P. (2015). Supporting behavior for school success: A step-by-step guide to key strategies. Guilford Press.

Margulieux, L.E., Morrison, B.B., & Decker, A. (2020). Reducing withdrawal and failure rates in introductory programming with subgoal labelled worked examples. International Journal of STEM Education, 7, 1-16.

Mehmood, E., Abid, A., Farooq, M.S. & Nawaz, N.A. (2020). Curriculum, Teaching and Learning, and Assessments for Introductory Programming Course. In IEEE Access, vol. 8, pp. 125961–125981.

Nadja, Z. (2022). PEGAM – a Personalized Gamification design Model for programming language e-courses. Doctoral thesis, RWTH Aachen University, Germany. DOI: 10.13140/RG.2.2.30798.74567.

Othman, M., Osman, A., Abdullah, N. & Ahmad, S.Z. (2022). Impact of Student’s Programming Experience on Cognitive Skills: Towards a Gamified Multimedia Learning Approach, Asian Journal of Universi-ty Education, 18(4), 944–953.

Padirayon, L. M. (2019). The Designed Gamification Application Architecture and Elements for a C\# Programming Course. Proceedings of the 2019 4th International Conference on Multimedia Systems and Signal Processing - ICMSSP 2019. doi:10.1145/3330393.3330414.

Pérez-Clark, P., Lane, K. S., Austin, K. S., Allen, G. E., Oakes, W. P., Lane, K. L., & Menzies, H. M. (2020). Instructional feedback: A step-by-step guide to virtual learning environments. Ci3T Strategic Leadership Team.

Piteira, M., Costa, C. J., & Aparicio, M. (2018). Computer Programming Learning: How to Apply Gamification on Online Courses?. Journal of Information Systems Engineering and Management, 3(2), 11. https://doi.org/10.20897/jisem.201811.

Pitoyo, M. D., & Asib, A. (2020). Gamification-Based Assessment: The Washback Effect of Quizizz on Students' Learning in Higher Education. International Journal of Language Education, 4(1), 1-10.

Poonsawad, A., Srisomphan, J., & Sanrach, C. (2022). Synthesis of Problem-Based Interactive Digital Storytelling Learning Model Under Gamification Environment Promotes Students’ Problem-Solving Skills. International Journal of Emerging Technologies in Learning (iJET), 17(05), pp. 103–119. https://doi.org/10.3991/ijet.v17i05.28181

Santos, J. S. (2023). Measuring and fostering cognitive programming skills in beginners. Ph.D. Thesis. Campina Grande, Paraíba, Brasil.

Shalatska, H., Zotova-Sadylo, O., Makarenko, O., & Dzevytska, L. (2020, November). Implementation of E-assessment in Higher Education. CEUR Workshop Proceedings.

Scherer, R., Siddiq, F., & Viveros, B. S. (2020). A meta-analysis of teaching and learning computer programming: Effective instructional approaches and conditions. Computers in Human Behavior, 109, 106349.

Winanti, Abbas, B.S., Suparta, W., Heryadi, Y. & Gaol, F. L. (2020). Gamification Framework for Programming Course in Higher Education, Journal of Game, Game Art and Gamification, Vol. 05, No. 02.

Zaharias, P. (2009). Usability in the context of e-learning: A framework augmenting ‘traditional ' usability constructs with instructional design and motivation to learn. International Journal of Technology and Human Interaction (IJTHI), 5(4), 37-59.

<table>
<tbody>
<tr class="odd">
<td><blockquote>
<p><img src="67638542150ea_media/media/image2.png" style="width:1.01042in;height:0.36111in" alt="A picture containing text, clipart Description automatically generated" /></p>
</blockquote></td>
<td>© 2024 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).</td>
</tr>
</tbody>
</table>
