THE IMPACT OF ONLINE TEACHING DURING THE COVID 19 PANDEMIC ON STUDENTS’ RESULTS IN COMPUTER SCIENCE SUBJECTS

Martin Misut, Maria Misutova

Abstract: The paper describes research aimed at determining the impact of various forms of teaching (full-time/online) on the success of students in the 1st year of engineering studies in computer science subjects Software Engineering I and Software Engineering II (SWE I, SWE II) at FHI. The research had two primary goals. The first was to find out whether the application of different ways of teaching (full-time/online) has a statistically significant effect (significant effect) on the success of students in computer science subjects. The second objective was to determine whether the application of different forms of teaching (full-time/online) has a statistically significant effect (significant effect) on success in the group of men and in the group of women. We used a didactic test to answer the research questions. It contained 40 tasks of various types and was created in MOODLE. Students answered the test online in a computer room in the winter semester and online at home in the summer semester. A T-test using SPSS was used for statistical verification of the established hypotheses. The obtained results indicate that the application of various forms of teaching does not have a statistically significant effect on the success of students in the subject. The results further confirmed that it is the same in the group of women and men. Although in the subject taught in the full-time form, students achieved better results compared to the subject taught online, the difference was not statistically significant. In both subjects, men performed worse than women, but the difference was also not statistically significant.
Keywords: online teaching, computer science subjects, online assessment
ACM Computing Classification System: Applied computing~Education~E-learning

Introduction
The research described in the paper is part of extensive research, the main goal of which is to design and verify a model for the teaching of computer science subjects with an emphasis on the development of creativity, independence, social skills of students as well as critical thinking.
At present, it is necessary for graduates to be able to search for, process and creatively use the essential information. [1] A creative individual with a high degree of thinking flexibility can achieve better and original problem-solving results even with less information than a perfectly informed individual whose thinking is unable to operate flexibly with the available material. [2] An important factor is a teacher [3] who ceases to be a source of information and becomes an assistant in the student’s learning. [4] This new and much more challenging task requires a deeper understanding of the issue, the will to teach experimentally [5] and not to act as an authority. [6] The current situation requires education to be made more effective. As a result, it is necessary to innovate the educational process not only in terms of content and methods but also in forms. [7]

1  Methodology
The proposed teaching model was verified in the teaching of the subjects Software Engineering (SWE1, SWE2) and Business Process Modeling (MPP) at FHI EU Bratislava. The model creation process is not over yet. We continue on its development and modify it based on the results of implemented pedagogical experiments. [8] [9]
The teaching model is of the 40-60 type, i.e. the overall evaluation consists of the assessment during the semester (40%) and the final test. (60%). The obtained points during the semester consist of points for the project and points for the evaluation of other students´ works. Students are divided into 4-member working groups. Each group is working on a joint project. The teacher does not evaluate directly. Every student is motivated to evaluate projects through extra points. A detailed description of the proposed teaching model can be found in [10]
During the Covid19 pandemic, the model was modified in terms of forms of teaching. Lectures and exercises in the summer semester in the subject SWE II took place from the 3rd week of the semester online via the MS TEAMS application. The subject SWE I was taught face-to-face in the winter semester. The same teacher taught both subjects.
The research had two primary goals. The first was to find out whether the application of different forms of teaching (full-time/online) has a statistically significant effect on the success of students in the subject. The second goal was to find out whether the application of different forms of teaching (full-time/online) has a statistically significant effect on the success of a group of men and a group of women.
The results of the last part of the research showed that students have a positive attitude towards online teaching. [11] Online lectures are perceived positively. They are more comfortable than face-to-face lectures. Students recommend online lectures in university studies, and they recognize that they can use the skills acquired in online lectures later at work, for example, when working from home. Attitudes towards online exercises are no longer so highly positive. Online practices through the MS TEAMS application do not suit students more than full-time activities, nor do they consider them more useful. The results of this part of the research also predict the influence of gender on students’ attitudes to online teaching. Women rated online education less positively than men.
Based on the results of the last part of the research and based on experience, we have established the following working hypotheses:
•	Hypothesis 1: The application of various forms of teaching does not have a statistically significant effect on the success of students in computer science subjects.
•	Hypothesis 2a: The application of various forms of teaching does not have a statistically significant effect on the success of a group of men in computer science subjects.
•	Hypothesis 2b: The application of various forms of teaching has a statistically significant effect on the success of a group of women in computer science subjects.
We used a T-test using SPSS to verify the established hypotheses.
The data describe the results of 75 FHI EU students in SWE I and SWE II in two consecutive academic years. Of these, 29 were women, and 46 were men. We assigned the corresponding values 1-3.5 to the A-FX marks. The following tables provide descriptive statistics.
Table. 1 Descriptive Statistics data from the academic year 2018/2019
	N	Minimum	Maximum	Mean	Std. Deviation	
SWEI_19	39	1,0	3,0	1,846	,5150	
SWEII_19	39	1,0	3,5	2,038	,4918	
Valid N (listwise)	39					

We used data from the academic year 2018/2019 to verify whether there is a statistically significant difference in the success of students in computer science subjects SWE I and SWE II. Both subjects in this academic year were taught by the full-time method. The reason for this verification was to obtain an answer to the question of whether the possible statistically significant difference in success is not due to the different, albeit related, the content of the subjects.
Table 2 Descriptive Statistics data from the academic year 2019/2020
	N	Minimum	Maximum	Mean	Std. Deviation	
SWEI_20	36	1,0	3,5	1,694	,5513	
SWEII_20	36	1,0	3,0	1,792	,4532	
Valid N (listwise)	36					

In 2018/19, the group of respondents consisted of 39 students, of which 23 were men and 16 were women. Students in the winter semester studied the subject SWE I, in the summer semester the subject SWE II. The same teacher taught both subjects in a face-to-face way.

Table 3. Descriptive Statistics of data from the academic year 2018/2019 by sex
Sex	N	Minimum	Maximum	Mean	Std. Deviation	
M	SWEI_19	23	1,0	3,0	1,978	,4880	
	SWEII_19	23	1,5	3,5	2,130	,5269	
	Valid N (listwise)	23					
W	SWEI_19	16	1,0	2,5	1,656	,5072	
	SWEII_19	16	1,0	2,5	1,906	,4171	
	Valid N (listwise)	16					

In 2019/20, the group of respondents consisted of 36 students, of which 23 were men and 13 were women. Students in the winter semester studied the subject SWE I, that was taught in full-time and in the summer semester the subject SWE II was taught online via the MS TEAMS application.  The same teacher taught both subjects.
Table 4. Descriptive Statistics of data from the academic year 2018/2019 by sex
Sex	N	Minimum	Maximum	Mean	Std. Deviation	
M	SWEI_20	23	1,0	3,5	1,739	,5194	
	SWEII_20	23	1,0	3,0	1,826	,4423	
	Valid N (listwise)	23					
W	SWEI_20	13	1,0	3,0	1,615	,6176	
	SWEII_20	13	1,0	3,0	1,731	,4837	
	Valid N (listwise)	13					

2  Results
(Tab. 5) shows that in the academic year 2018/2019, the average grade from the subject SWE II (2,038) is worse than in the subject SWEI (1,846) for the whole group of respondents.  However, the Paired Sample T-test using SPSS showed that the difference was not statistically significant (p = 0.058), since the value of p> 0.05.
Table 5. Paired Samples Statistics academic year 2018/2019
	Mean	N	Std. Deviation	Std. Error Mean	
Pair 1	SWEI_19	1,846	39	,5150	,0825	
	SWEII_19	2,038	39	,4918	,0788	

(Tab. 6) shows that both men and women achieved the same (worse) average mark in SWE II compared to the average mark in SWE I.
However, neither in the male group (p = 0.284) nor in the female group (p = 0.088) was the difference statistically significant. Using the T-test, we verified that in the success of students in computer science subjects SWE I and SWE II, which were in academic year 2018/2019 taught face-to-face is not a statistically significant difference.
Table 6. Paired Samples Statistics if 2018/2019 by sex
Sex	Mean	N	Std. Deviation	Std. Error Mean	
M	Pair 1	SWEI_19	1,978	23	,4880	,1018	
		SWEII_19	2,130	23	,5269	,1099	
W	Pair 1	SWEI_19	1,656	16	,5072	,1268	
		SWEII_19	1,906	16	,4171	,1043	

As can be seen from (Tab. 7), in the academic year 2019/20, the results are the same in the whole group of respondents. Students also in this academic year achieved worse results in the subject SWE II (1,792) compared to the subject SWE I (1,694). However, the results of the Paired Sample T-test again showed that the difference is not statistically significant for the whole group of respondents. (p = 0.303).

Table 7. Paired Samples Statistics academic year 2019/2020
	Mean	N	Std. Deviation	Std. Error Mean	
Pair 1	SWEI_20	1,694	36	,5513	,0919	
	SWEII_20	1,792	36	,4532	,0755	

In the academic year 2019/2020 in the group of men (p. = 0.477) and women (p = 0.461) the difference is also not statistically significant, as the value of p> 0.05. The average mark in the subject SWE II, in the group of men as well as in the group of women, is again worse than in the subject SWE I (see Tab. 8).

Table 8. Paired Samples Statistics academic year 2019/2020 by sex
Sex	Mean	N	Std. Deviation	Std. Error Mean	
M	Pair 1	SWEI_20	1,739	23	,5194	,1083	
		SWEII_20	1,826	23	,4423	,0922	
W	Pair 1	SWEI_20	1,615	13	,6176	,1713	
		SWEII_20	1,731	13	,4837	,1342	

Based on the above results, it can be stated that:
Hypothesis 1, in which we assumed that the application of various forms of teaching does not have a statistically significant effect on the success of students in computer science subjects, was confirmed. Although students achieved a better grade in the subject taught in the full-time form (1,694) compared to the subject taught online (1,792), the difference was not statistically significant. (p = 0.303)
Hypothesis 2a, in which we assumed that the application of various forms of teaching does not have a statistically significant effect on the success of a group of men in computer science subjects, was confirmed. Although men in the full-time course achieved a better grade (1,739) compared to the online course (1,826), the difference was not statistically significant (p = 0.477)
Hypothesis 2b, in which we assumed that the application of various forms of teaching has a statistically significant effect on the success of a group of women in computer subjects, was not confirmed. Although women achieved a better average grade (1,615) in the subject taught in the full-time form than in the subject taught online (1,731), the difference was not statistically significant (p = 0.461).

3  Discussion and Conclusion
The results of the research confirmed our assumption (hypothesis 1) that different forms of teaching (full-time/online) do not affect the success of students in the subject. Even though the online form of education did not have a positive effect on the success of students in the course, it has its place in teaching at the university, not only during the pandemic. Based on our observations, we can state an increase in the participation and activity of students in online lectures compared to full-time. Also, many jobs today use home-based work (home office), so students can use the skills acquired in online teaching after graduation.
We were surprised by the failure to confirm the hypothesis in which we assumed a lower success rate in a subject taught online for a group of women. In contrary, women achieved a higher average grade in a subject conducted online, compared to a mark in the subject taught in face-to-face, but the difference was not statistically significant. We established the hypothesis on the results of the previous part of the research, where women rated online teaching less positively than men. [11] One of the possible reasons for the obtained result may be the fact that although women declared a less positive attitude towards online teaching compared to men, they approached the study of the subject as systematically and responsibly as in the case of full-time education.
On the other hand, the results in the group of men indicate that men probably also approach the study of the subject in the same way whether it is taught in face-to-face or online. Although in the previous part of the research, they declared a highly positive attitude towards teaching online, this did not affect their study results. However, further research is needed to verify this assumption, also given that online teaching did not take place throughout the semester, but only from the 3rd week of the 13-week semester.
The results of the described research will be used in the process of creating a model of teaching computer science subjects with an emphasis on the development of skills in the form of competencies to increase its didactic effectiveness.

Acknowledgements
This contribution was funded by the project KEGA 009STU - 4/2018 „The innovation of teaching the subject Intelligent Control Methods at MTF STU“, and partially supported by VEGA 1/0373/18 “Big data analytics as a tool for increasing the competitiveness of enterprises and supporting informed decisions“ and by VEGA 1/0232/18 “Using the methods of multi-objective optimization in a production processes control. “

References
[1]	Andrade-Aréchiga, M., G. López, and G. López-Morteo (2012). Assessing effectiveness of learning units under the teaching unit model in an undergraduate mathematics course. Computers & Education, 2012. 59(2): p. 594-606.
[2]	Blanco, M. and M. Ginovart(2012). On How Moodle Quizzes Can Contribute to the Formative e-Assessment of First-Year Engineering Students in Mathematics Courses. RUSC, 2012. 9(1): p. 354-370.
[3]	Baneres, D. et al. (2016). Adaptive e-assessment system: A general approach. International Journal of Emerging Technologies in Learning, 2016. 11(7): p. 16-23.
[4]	Webb, M., D. Gibson, and A. Forkosh-Baruch (2013). Challenges for information technology supporting educational assessment. Journal of Computer Assisted Learning, 2013. 29(5): p. 451-462.
[5]	Fuentes, J.M. et al. (2014). Computer-based tools for the assessment of learning processes in higher education: A comparative analysis, in INTED2014: 8th International Technology, Education and Development Conference, L.G. Chova, A.L. Martinez, and I.C. Torres, Editors. 2014, Iated-Int. Assoc. Technology Education & Development: Valencia, Spain. p. 976-984.
[6]	Hostovecky, M., M. Misut, and K. Pribilova (2015). Web-based testing in science education. Lecture Notes in Electrical Engineering, 2015. 313: p. 247-254.
[7]	Daly, C. et al. (2010). Exploring formative e-assessment: using case stories and design patterns. Assessment & Evaluation in Higher Education, 2010. 35(5): p. 619-636.
[8]	Mišút, M. and M. Mišútová (2013). Evaluation of ICT Implementation into Engineering Education, in International Conference on Advances in Information Technology (ICAIT 2013), G. Lee, Editor. 2013, IERI: Jeju Island, Korea p. 260-265.
[9]	Misut, M. and M. Misutova (2017). VALIDITY OF DURING-TERM E- ASSESSMENT, in 11th International Technology, Education and Development Conference, INTED 2017, ALM L. Gómez Chova, I. Candel Torres, Editor. 2017, IATED Academy: Valencia, Spain. p. 2812-2820.
[10]	Mišútová, M. - Mišút, M. (2020) Comparison of two project-oriented teaching models in computer science subjects as viewed by students. In INTED 2020: 14th International Technology, Education and Development Conference (INTED 2020), 02. - 04. March 2020, Valencia, Spain. 1. edition Valencia, Spain: IATED, 2020, S. 8747-8754. ISSN 2340-1079. ISBN 978-84-17939-8. V databáze: WOS: 000558088808136 ; DOI: 10.21125/inted.2020.2386

 [11]	Mišút, M. and Mišútová, M. (2020) Teaching IT subjects online-experiences and students´attitudes. In DIDMATTECH 2020: proceedings of XXXIII. DidMatTech 2020 conference. New Methods and Technologies in Education, Research and Practice, June 25-26, 2020 Budapest. Budapešť: Eötvös Loránd University in Budapest, 2020, S. 149-158. ISBN 978-963-489-244-1.

Authors

	Assoc. Prof. Martin Mišút, PhD.
KAI FHI EUBA
Dolnozemská cesta 1, 852 35 Bratislava, Slovakia
e-mail: martin.misut@euba.sk
Martin Mišút obtained a PhD in the field of Industrial Automation and an associate professor title from Applied Informatics. He works as a university teacher (associate professor) at the University of Economics in Bratislava, Department of Applied Informatics. In the past, he was employed at Trnava University and the Slovak University of Technology in different positions (university teacher, head of the department, vice dean, vice-rector, rector) and at FH Koethen as Visiting Professor. 
His research interests are focused on educational paradigm change due to technological progress, information systems, data modelling and processing, as well as applying ICT into various fields of human activities.

	Assoc. Prof. Mária Mišútová, PhD.
UIAM MTF STU, 
ul. J. Bottu 25, 917 24 Trnava, Slovakia
e-mail: maria.misutova@stuba.sk
Mária Mišútová after graduating from Comenius University in the field of teaching mathematics and physics, she obtained a PhD in the field theory of teaching. In the year 2000, she received the title associate professor in the field of Theory of Teaching Mathematics. She currently works as a university teacher (associate professor) at the Slovak University of Technology at the Institute of Informatics, Automation and Mechatronics. Her research interests include Theory of teaching mathematics and applying ICT into Mathematics education.