**The Development of Autonomous Examination Paper Application: A Case Study in UiTM Cawangan Perlis** Noorfaizalfarid Mohd Noor\*, Nadhirah Mohd Napi, Izzati Farzana Ibni Amin Faculty of Computer and Mathematical Sciences Universiti Teknologi MARA Cawangan Perlis, Perlis, Malaysia nfaizalf@uitm.edu.my, nursakinah282@gmail.com, izzatifarzanaibniamin01@ gmail.com *Abstract* *Examination is a vital role to measure the capabilities of students in their learning. Hence, generating question paper in an effective way is a decisive job for educators in educational institution. Using traditional method, it is monotonous and time consuming. Today, Autonomous Examination Paper (AEP) is used to produce exam paper. Many researchers have proposed effective AEPs to be used by educators. This paper aims to investigate about AEP development and to construct AEP in UiTM Cawangan Perlis. As a result, Ad-Hoc Question Paper Application (AQPA) has been developed using Fisher-Yates algorithm to generate questions for exam paper in the university. Evaluation based on Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) reveal that lecturers in the university manage to interact with AQPA and willing to use it as a tool to minimize their workload. However, more improvement must be done on AQPA to be an effective AEP. To conclude, AEP brings significance to educators and can be improved with the latest technology.* *Keywords: algorithm, automated question paper, examination, Fisher-Yates, randomization technique,* **Introduction** Composing a set of exam paper is a monotonous task for instructors due to time and energy consuming, institutional exam templates, questions duplicity, syllabus requirements and questions identification (Cen et al., 2010; Hegde, Rao, & Shivali, 2018). To enhance the process, framework of automated application was proposed by researchers in order to steer efficiency to produce high quality automated examination paper. Autonomous Examination Paper (AEP) can be defined as a day-to-day application that stores collection of qualified questions in a database and use randomization techniques to generate a set of question either in printed document or for online examination by involving different level of participants (Amria, Ewais, & Hodrob, 2018; Bankar, Bhor, & Bhalerao, 2018; Bhirangi & Bhoir, 2016; Naik, Sule, Jadhav, & Pandey, 2014). Online examination application is user adaptive, friendly response time to candidates, multi-media material for questions, stored questions, use randomization, provide instant feedback and produce exam reports (Temitayo, Adebisi, & Alice, 2013). According to Cen et al., (2010), three approaches can be applied to developed automated question paper application. Firstly, all the stored questions will be displayed and then selected by experienced instructors. This approach may time consuming but the burden in producing question can be avoided due to stored questions in a database. Secondly, the stored questions will be selected using randomization techniques that embedded in the application. In this implementation, only selected and limited question will be produced automatically. It eliminates the functionality of human in generating the question paper. Finally, the randomized question can be customized accordingly to the user and seems be able to produce more version of question paper. However, the authors claimed that using randomization techniques may cause deadlock in computer and finally question paper failed to be produced. AEP as a tool in educational institution must be developed to support Bloom’s taxonomy cognitive levels such as course syllabus, difficulty level, coverage of cognitive level, marks, subject and question type (Amria et al., 2018). These are the constraints that AEP must be properly handled. According to Liu & Wang (2013), exam paper in AEP can be developed using random variable algorithm, backtracking algorithm and artificial algorithm. However, the authors concluded that artificial algorithm is the best solution to generate exam paper due to the drawback of time processing for large repository in random variable algorithm and backtracking algorithm. Researchers have already implemented artificial algorithm in AEP such as fuzzy logic, ant colony and genetic algorithm (Chavan & Mohandas, 2016; Hasimah, Teo, Bakar, Rezduan, & Rashid, 2014; Liu & Wang, 2013). Instead of the methods mentioned above, Fisher-Yates algorithm also has been used for AEP (Hegde et al., 2018). It seems that randomization is compulsory feature in AEP to make sure high quality of question paper production in educational institution. **Ad-Hoc Question Paper (AQPA)** Based on the above literature study, we have constructed an application for AEP known as Ad-Hoc Question Paper Application (AQPA). It produces a set of printed exam paper based on courses in UiTM Cawangan Perlis, Perlis, Malaysia. Instead of using word processor application to compose exam paper, AQPA offers a new approach for lecturers to create exam paper in web application using Fisher-Yates algorithm. ## To brief about AQPA, a Context Diagram in Figure 1 is used to explain the process in AQPA. From the diagram, *Administrator* and *Lecturer* are the level of participants in AQPA. The *Administrator* is a person that has responsibility to manage users in AQPA. AQPA allows lecturers to register themselves as *Lecturer*. Once registration is completed, the user needs to login using their own credential. The login page can be accessed from the main page as shown in Figure 2. Once login is done, *Lecturer* can add new exam paper, manage questions and lastly generate the exam paper as printed document. ![](105-1-274-1-2-20190531_media/media/image1.png) Figure 1: AQPA Context Diagram ![](105-1-274-1-2-20190531_media/media/image2.png) Figure 2: AQPA Front Page To start using AQPA, *Lecturer* need to enter constraints for the exam paper such as Course Code, Course Name, Semester and Exam Duration as shown in Figure 3. ![](105-1-274-1-2-20190531_media/media/image3.png) Figure 3: Exam Paper Constraints Form After completing the form, AQPA request the *Lecturer* to choose the question type either to enter Multiple Choice Question (MCQ) or Short Answer Question (SAQ) as shown in Figure 4 and Figure 5, respectively. Each of the page allows *Lecturer* to manage constraint based on Chapter and Difficulty Level. ![](105-1-274-1-2-20190531_media/media/image4.png) Figure 4: Multiple Choice Question (MCQ) Form ![](105-1-274-1-2-20190531_media/media/image5.png) Figure 5: Short Answer Question (SAQ) Form The completed questions can be viewed by *Lecturer* based on question type. For instance, Figure 6 depicts the list of MCQ question for the registered exam paper. It can be observed that user can edit and delete the current questions by clicking the appropriate icon. Figure 7 illustrates the page when *Lecturer* click *Generate Question Paper* menu. The page displays the randomized questions based on exam paper. On the top of the page, there are three buttons that allow *Lecturers* to randomize the questions, to print all the questions and to print the cover page for the exam paper. ![](105-1-274-1-2-20190531_media/media/image6.png) Figure 6: Short Answer Question (SAQ) Form ![](105-1-274-1-2-20190531_media/media/image7.png) Figure 7: Questions in Exam Paper Button *Randomize* will regenerate the questions in random order again. Fisher-Yates algorithm is used in AQPA for the questions. The code in Figure 8 was retrieved in GitHub web page and has been modified to suit with AQPA page coding. The code successfully randomized the questions in AQPA.

<?php

$totQues = array();

if(mysqli_num_rows($result)>0){

while($row = mysqli_fetch_array($result)){

$totQues[]=$row;

}

}function MyShuffle(&$totQues) {

for($i = 0; $i < sizeof($totQues); ++$i) {

$r = rand(0, $i);

$tmp = $totQues[$i];

$totQues[$i] = $totQues[$r];

$totQues[$r] = $tmp;

}

}

?>

<?php

MyShuffle($totQues, $totQues1);

$n=1;

foreach($totQues as $tot)

{

echo"<pre>";

echo"$n. ";

echo $tot['ques']."";

$n++;

echo"</pre>";

}

?>
Figure 8: Fisher-Yates Shuffle Algorithm (Statescu, 2019) Button *Print* will print all the questions as UiTM Cawangan Perlis exam paper without a cover page. To get a complete exam paper set, *Lecturer* need to print cover page using button *Coverpage*. Figure 9 shows the example of exam paper. Finally, *Lecturer* can combine the printed materials as a set of exam paper. ![](105-1-274-1-2-20190531_media/media/image8.png) Figure 9: Printed Exam Paper Cover and Question Page **User Evaluation** Evaluation for APQA is based on Technology Acceptance Model (TAM). TAM can be used to evaluate user behaviour for AQPA as an AEP (Abdullah, Ward, & Ahmed, 2016). In TAM, Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) is the main pillar to describe level of user acceptance to use application in easy and less effort and to identify level of effectiveness of application in job performance, respectively (Abdullah et al., 2016; Davis, 2013). The testing of AQPA was conducted from 1th – 17th May 2019 involving 20 lecturers from Faculty of Computer and Mathematical Sciences (45%), Faculty of Accountancy (25%), Faculty of Applied Sciences (15%) and Faculty of Business and Management (15%) in UiTM Cawangan Perlis. Table 1 shows the Likert Scale with 5 point of scales that has been used for PEOU and PU questionnaires. Table 1: Scales Description
Scale

Description

5

Strongly Agree

4

Agree

3

Neutral

2

Disagree

1

Strongly Disagree
1. ***Perceived Ease of Use (PEOU)Result*** In Table 2, there are seven questions for PEOU evaluation among the respondents. It can be observed that the lectures agree that AQPA is interactive web (Q1, Q4, Q5 and Q6) yet confused how AQPA can perform to complete their task (Q2and Q3). Most of the lecturers feel that AQPA is not easy to use. However, they willing to use AQPA as AEP to generate exam paper (Q7). Table 2: Perceived Ease of Use (PEOU) Result
Range 1 2 3 4 5
Criteria
Q1 Learning to operate the AQPA would be easy for me. 

2

(10%)

2

(10%)

16

(80%)
Q2 I would find it easy to get AQPA to do what I want to do.

8

(40%)

10

(50%)

2

(10%)
Q3 Using the AQPA enables me to accomplish tasks more quickly.

18

(90%)

1

(5%)

1

(5%)
Q4 It would be easy for me to become skillful at using the AQPA.

9

(45%)

11

(55%)
Q5 My interaction with AQPA would be clear and understandable.

6

(30%)

13

(65%)

1

(5%)
Q6 I would find the AQPA easy to use.

16

(80%)

4

(20%)
Q7 I intend to use the AQPA in the future.

3

(15%)

6

(30%)

10

(50%)

1

(5%)
2. ***Perceived of Usefulness (PU) Result*** Table 3 depicts the questions and the results for PU testing. Firstly, the lecturers hesitate and confound on AQPA as AEP that can practically assist them to generate exam paper (Q1, Q3, Q4, Q5 and Q9). However, they believe that AQPA is a functional tool for exam paper production (Q2). AQPA features are well functioned to them especially in managing MCQ and SAQ questions (Q6 and Q8). They also agree using Fisher-Yates algorithm in AQPA is working to randomize the set of questions in many times (Q7). Instead of unsure respond for some functionalities in AQPA, the lecturers agree that AQPA can be used to manage questions in exam paper preparation (Q10). Table 3: Perceived Usefulness (PU) Result
Range 1 2 3 4 5
Criteria
Q1 Using AQPA would enable to create the question paper automatically.

4

(20%)

13

(65%)

3

(15%)
Q2 Using AQPA would improve the way lecturer creates a question paper.

2

(10%)

9

(45%)

8

(40%)

1

(5%)
Q3 Using AQPA would increase university and lecturer productivity.

2

(10%)

11

(55%)

7

(35%)
Q4 Using AQPA would enhance the effectiveness in generating the question paper.

17

(85%)

3

(15%)
Q5 Using AQPA would make it easier for lecturer to generate a question paper.

8

(40%)

10

(50%)

2

(10%)
Q6 I find that by using AQPA, I am able to manage questions (add, edit or delete).

2

(10%)

17

(85%)

1

(5%)
Q7 I find that by using the AQPA, the questions are randomized well.

4

(20%)

16

(80%)
Q8 I find that by using the AQPA, I am able to produce two types of questions in the question paper.

6

(30%)

13

(65%)

1

(5%)
Q9 By using the AQPA, I am able to print and get the cover page in PDF format.

13

(65%)

7

(35%)
Q10 I would find AQPA is a useful application.

1

(5%)

6

(30%)

13

(65%)
**Conclusion and Recommendation** The term Autonomous Examination Paper (AEP) can be used to describe automated paper generator application either in mobile or web platform. The term itself includes the entire description of functional features for computer process in producing questions for exam paper. Randomization algorithm is a core element in AEP to make sure lenience usage in producing high quality exam paper. Many types of randomization can be used in AEP. AEP can be more effective if the compiled questions can be shifted as online examination instead of just an exam paper. From the TAM evaluation, AQPA is accepted among the lectures as a tool for exam paper assistant. The slight drawback from lecturers’ feedback indicate that AQPA need to improve the source codes, the interface design and the process flow. The Fisher-Yates code worked well in AQPA to randomize questions. The development of AQPA contributes interesting findings on how to produce an effective and interactive AEP for lecturers in a university. Instead of functional randomization algorithm, the design for data flow, data storage, user interface, sequence of user action and way of questions conversion are the important elements that the researcher and the developer need to fixate in future AEP development. Eventually, current technology such as information retrieval, big data and IoT could be integrated by the researchers and the developers to enhance the usability and effectiveness of AEP among educators in education institutions. **References** Abdullah, F., Ward, R., & Ahmed, E. (2016). Computers in Human Behavior Investigating the in fl uence of the most commonly used external variables of TAM on students ’ Perceived Ease of Use ( PEOU ) and Perceived Usefulness ( PU ) of e-portfolios. *Computers in Human Behavior*, *63*, 75–90. http://doi.org/10.1016/j.chb.2016.05.014Amria, A., Ewais, A., & Hodrob, R. (2018). A Framework for Automatic Exam Generation based on Intended Learning Outcomes, *1*(Csedu), 474–480.Bankar, M., Bhor, P., & Bhalerao, P. (2018). Automated generation of question paper for online MCQ test by using Shuffling algorithm, *3*(4), 219–226.Bhirangi, R., & Bhoir, S. (2016). Automated Question Paper Generation System. *International Journal of Emerging Research in Management \&Technology*, *5*(4), 28–34.Cen, G., Dong, Y., Gao, W., Yu, L., See, S., Wang, Q., & Yang, Y. (2010). A implementation of an automatic examination paper generation system. *Mathematical and Computer Modelling*, *51*(11–12), 1339–1342. http://doi.org/10.1016/j.mcm.2009.11.010Chavan, A., & Mohandas, M. (2016). Automated Question Paper Generator System using Apriori Algorithm and Fuzzy Logic, *2*(11), 707–710.Davis, F. D. (2013). Information Technology Introduction, *13*(3), 319–340.Hasimah, N., Teo, I., Bakar, N. A., Rezduan, M., & Rashid, A. (2014). Representing Examination Question Knowledge into Genetic Algorithm, (April), 900–904.Hegde, V., Rao, L. V, & Shivali, B. S. (2018). The Framework for Web - Based Automated Online Question Paper Generator through JEE, *7*(3), 1415–1419. http://doi.org/10.14419/ijet.v7i3.13573Liu, D., & Wang, J. (2013). Automatic Test Paper Generation Based on Ant Colony Algorithm, *8*(10), 2600–2606. http://doi.org/10.4304/jsw.8.10.2600-2606Naik, K., Sule, S., Jadhav, S., & Pandey, S. (2014). Automatic Question Paper Generation System using Randomization Algorithm, (12), 192–194.Statescu, A. (2019). Fisher-Yates Shuffle. Retrieved May 1, 2019, from https://github.com/thinkphp/fisher-yates-shuffle/blob/master/fisher-yates-shuffle.phpTemitayo, F., Adebisi, A., & Alice, O. (2013). Computer-Based Test ( Cbt ) System For University Academic Enterprise Examination. *International Journal Of Scientific & Technology Research*, *2*(8), 336–342.