E-Learning Performance Assessment Model Proposal for E-Learning Academies
Keywords:
E-Learning, E-Learning academies, Performance assessment, Data visualization
Abstract
Corporates provide e-learning systems to their employees to improve their knowledge and competences needed in job descriptions. This study aims to propose a model that measures and analyzes real user data in e-learning system to measure the targeted personal development and learning level of employees of corporates with e-learning academies according to different criteria, compare the success level, and evaluate the effect of training on job performance. Employees’ training activities, departments, positions, assessment and survey results, and other related data are recorded in the e-learning system and collected from the e-learning system, LMS (Learning Management System), by data mining method. The document also reviews the System Approach, Kirkpatrick’s Four Levels of Training Evaluation Model, Balanced Scorecard, KPI (Key Performance Index), and OKR (Objectives and Key Results). In order to make a performance assessment in the model, data collected from e-learning systems is used, and the academy enters its own target data into the model. The results are associated with the corporate’s KPI and OKR targets in the model. Model output is visualized for management review. The results declare that the model helps the academy have a holistic perspective for training activities associated with corporate target, a realistic review of effects of training on job performance, and possible opportunities and plans for future development of the trainings.
Downloads
Download data is not yet available.
References
Anton, C., & Shikov, A. (2018). The method of personalized corporate e-learning based on personal traits of employees. Procedia Computer Science, 136, 511-521. https://doi.org/https://doi.org/10.1016/j.procs.2018.08.253
Brandon Hall. (2019). HCM Outlook 2019. https://go.brandonhall.com/hcm-outlook-2019
Cahapay, M. B. (2021). Kirkpatrick model: Its limitations as used in higher education evaluation. International Journal of Assessment Tools in Education, 8(1), 135-144. https://doi.org/10.21449/ijate.856143
Charoenlarpkul, C., & Tantasanee, S. (2019). A proposed employee development program from objectives and key results: a case study of SG Group of Companies. ABAC ODI Journal Vision. Action. Outcome, 6(2), 133.
Chen, H. J. (2010). Linking employees’ e-learning system use to their overall job outcomes: An empirical study based on the IS success model. Computers & Education, 55(4), 1628-1639. https://doi.org/https://doi.org/10.1016/j.compedu.2010.07.005
Demirören, S. (2014). Başarım ölçütlü uyarlanabilir öğrenmenin etkililiğinin ve verimliliğinin değerlendirilmesi. Eğitim Bilimleri ve Uygulama, 13, 47-64.
Dewan, M. a. A., Pachon, W. M., & Lin, F. (2021). A review on visualization of educational data in online learning. Lecture Notes in Computer Science (15–24). https://doi.org/10.1007/978-3-030-66906-5_2.
Docebo. (2020). E-Learning trends 2020 – A new era of learning.
https://www.docebo.com/resource/elearning-trends-2020-docebo-report/
Docebo. (2020). The Future is now: Learning strategy 2020.
https://www.docebo.com/resource/future-of-work-enterprise-learning-strategy-2020-report/
Drozdova, A. A., & Guseva, A. I. (2017). Modern technologies of e-learning and its evaluation of efficiency. Procedia - Social and Behavioral Sciences, 237, 1032-1038. https://doi.org/https://doi.org/10.1016/j.sbspro.2017.02.147
Fanning, B. A. (2019). How a professional services firm has adapted learning for the new digital age (Thesis No: 13904053) [Ed.D., University of Pennsylvania]. ProQuest Dissertations and Theses Global. Pennsylvania, United States. https://www.proquest.com/dissertations-theses/how-professional-services-firm-has-adapted/docview/2285296097/se-2?accountid=11054
Fink, A. (1998). Conducting research literature reviews: From paper to the internet. Sage.
Hatipoğlu, Z. (2020). Performans yönetimi, amaçlara göre yönetim yaklaşımı ve OKR sistemi. International Journal of Arts and Social Studies, 3(4), 1-16.
Kaplan, R. S. (2009). Conceptual foundations of the balanced scorecard. Handbook of Management Accounting Research, 1253-1269.
https://doi.org/10.1016/s1751-3243(07)03003-9
Khare, K., Lam, H., & Khare, A. (2018). Educational data mining (EDM): Researching impact on online business education. In A. Khare & D. Hurst (Eds.), On the Line, 37-53. https://doi.org/10.1007/978-3-319-62776-2_3
Kurt, Ö. E. (2016). Bilgi sistemleri başarı modeli ile bir e-öğrenme sistemi değerlendirmesi. Yönetim Bilişim Sistemleri Dergisi, 2(2), 140-149.
Marr, B. (2012). Key performance indicators: The 75 measures every manager needs to know. Financial Times/Prentice Hall.
Martins, L.B., Zerbini, T., & Medina, F.J. (2019). Impact of online training on behavioral transfer and job performance in a large organization. Revista de Psicología del Trabajo y de las Organizaciones, 35(1), 27-37. https://doi.org/10.5093/jwop2019a4
Milenko, R. (2017). OKR system as the reference for personal and organizational objectives. Econophysics, Sociophysics & Other Multidisciplinary Sciences Journal, 17(2), 28-37
Minhong, W., Weijia, R., Jian, L., & Stephen, J. H. Y. (2010). A performance-oriented approach to e-learning in the workplace. Journal of Educational Technology & Society, 13(4), 167-179.
Palo, J. (2020). Setting goals for autonomous motivation using objectives and key results. [Thesis Dissertation, Aalto University]. Aalto University School of Science Master’s Programme in Information Networks.
Petersen, B. A. (2008). Measuring e-learning program effectiveness: A stakeholder approach to scorecarding performance (Publication No: 304478979) [Doctoral Dissertation, Concordia University]. ProQuest Dissertations and Theses Global.
Rojas-Chipana, I., Aguilar-Pelizzoli, N., Quiroz-Flores, J. C., Collao-Diaz, M., & Flores-Perez, A. (2021, November 3-5). Increasing profitability through the implementation of an integrated balance score card & objectives key results model in a digital agency: A research in Perú. International Conference on Industrial Engineering and Operations Management Monterrey, Mexico.
Saba, F. (2012). A Systems Approach to the Future of Distance Education in Colleges and Universities: Research, Development, and Implementation. Continuing Higher Education Review, 76, 30-37.
Saba, F. (2013). Building the future: A theoretical perspective. Handbook of Distance Education, Routledge.
Saba, F. (2014). Methods of study in distance education: A critical review of selected recent literature. Online Distance Education: Towards A Research Agenda, 151-171.
Sariman, G. (2014). Veri madenciliğinde kümeleme teknikleri üzerine bir çalışma: K-means ve K-Medoids kümeleme algoritmalarının karşılaştırılması. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 15(3), 192-202. https://doi.org/10.19113/sdufbed.41288
Saverimuttu, J. J. (2022). An exploratory study on effectively upskilling employees during COVID-19. Robert Morris University.
Sevinç, L., & Yıldırım, O. (2001). Stratejik insan kaynakları yönetiminde HR scorecard uygulaması. Dokuz Eylül Üniversitesi İşletme Fakültesi Dergisi, 5(2-1), 149-164.
Slater, S., Joksimović, S., Kovanovic, V., Baker, R. S., & Gasevic, D. (2017). Tools for educational data mining: A review. Journal of Educational and Behavioral Statistics, 42(1), 85-106. https://doi.org/10.3102/1076998616666808
Stray, V., Moe, N. B., Vedal, H., & Berntzen, M. (2023, January 3-6). Using objectives and key results (OKRs) and slack: A case study of coordination in large-scale distributed agile. The proceedings of the 55th Hawaii International Conference on System Sciences, University of Hawai'i, Manoa, USA.
Vozniuk, A., Govaerts, S., & Gillet, D. (2013, 15-18 July). Towards portable learning analytics dashboards. IEEE 13th International Conference on Advanced Learning Technologies, Beijing, China.
Warner, M. C. (2019). The corporate adult learner: A study of the effectiveness of asynchronous learning in a corporate workspace (Publication No: 13810103) [Doctoral Dissertation, Northeastern University]. ProQuest Dissertations and Theses Global.
World Economic Forum. (2020). The Future of Jobs Report 2020.
https://www.weforum.org/reports/the-future-of-jobs-report-2020/
Zhou, H., & He, Y. (2018). Comparative study of OKR and KPI, DEStech Transactions on Economics Business and Management. https://doi.org/10.12783/dtem/eced2018/23986
Brandon Hall. (2019). HCM Outlook 2019. https://go.brandonhall.com/hcm-outlook-2019
Cahapay, M. B. (2021). Kirkpatrick model: Its limitations as used in higher education evaluation. International Journal of Assessment Tools in Education, 8(1), 135-144. https://doi.org/10.21449/ijate.856143
Charoenlarpkul, C., & Tantasanee, S. (2019). A proposed employee development program from objectives and key results: a case study of SG Group of Companies. ABAC ODI Journal Vision. Action. Outcome, 6(2), 133.
Chen, H. J. (2010). Linking employees’ e-learning system use to their overall job outcomes: An empirical study based on the IS success model. Computers & Education, 55(4), 1628-1639. https://doi.org/https://doi.org/10.1016/j.compedu.2010.07.005
Demirören, S. (2014). Başarım ölçütlü uyarlanabilir öğrenmenin etkililiğinin ve verimliliğinin değerlendirilmesi. Eğitim Bilimleri ve Uygulama, 13, 47-64.
Dewan, M. a. A., Pachon, W. M., & Lin, F. (2021). A review on visualization of educational data in online learning. Lecture Notes in Computer Science (15–24). https://doi.org/10.1007/978-3-030-66906-5_2.
Docebo. (2020). E-Learning trends 2020 – A new era of learning.
https://www.docebo.com/resource/elearning-trends-2020-docebo-report/
Docebo. (2020). The Future is now: Learning strategy 2020.
https://www.docebo.com/resource/future-of-work-enterprise-learning-strategy-2020-report/
Drozdova, A. A., & Guseva, A. I. (2017). Modern technologies of e-learning and its evaluation of efficiency. Procedia - Social and Behavioral Sciences, 237, 1032-1038. https://doi.org/https://doi.org/10.1016/j.sbspro.2017.02.147
Fanning, B. A. (2019). How a professional services firm has adapted learning for the new digital age (Thesis No: 13904053) [Ed.D., University of Pennsylvania]. ProQuest Dissertations and Theses Global. Pennsylvania, United States. https://www.proquest.com/dissertations-theses/how-professional-services-firm-has-adapted/docview/2285296097/se-2?accountid=11054
Fink, A. (1998). Conducting research literature reviews: From paper to the internet. Sage.
Hatipoğlu, Z. (2020). Performans yönetimi, amaçlara göre yönetim yaklaşımı ve OKR sistemi. International Journal of Arts and Social Studies, 3(4), 1-16.
Kaplan, R. S. (2009). Conceptual foundations of the balanced scorecard. Handbook of Management Accounting Research, 1253-1269.
https://doi.org/10.1016/s1751-3243(07)03003-9
Khare, K., Lam, H., & Khare, A. (2018). Educational data mining (EDM): Researching impact on online business education. In A. Khare & D. Hurst (Eds.), On the Line, 37-53. https://doi.org/10.1007/978-3-319-62776-2_3
Kurt, Ö. E. (2016). Bilgi sistemleri başarı modeli ile bir e-öğrenme sistemi değerlendirmesi. Yönetim Bilişim Sistemleri Dergisi, 2(2), 140-149.
Marr, B. (2012). Key performance indicators: The 75 measures every manager needs to know. Financial Times/Prentice Hall.
Martins, L.B., Zerbini, T., & Medina, F.J. (2019). Impact of online training on behavioral transfer and job performance in a large organization. Revista de Psicología del Trabajo y de las Organizaciones, 35(1), 27-37. https://doi.org/10.5093/jwop2019a4
Milenko, R. (2017). OKR system as the reference for personal and organizational objectives. Econophysics, Sociophysics & Other Multidisciplinary Sciences Journal, 17(2), 28-37
Minhong, W., Weijia, R., Jian, L., & Stephen, J. H. Y. (2010). A performance-oriented approach to e-learning in the workplace. Journal of Educational Technology & Society, 13(4), 167-179.
Palo, J. (2020). Setting goals for autonomous motivation using objectives and key results. [Thesis Dissertation, Aalto University]. Aalto University School of Science Master’s Programme in Information Networks.
Petersen, B. A. (2008). Measuring e-learning program effectiveness: A stakeholder approach to scorecarding performance (Publication No: 304478979) [Doctoral Dissertation, Concordia University]. ProQuest Dissertations and Theses Global.
Rojas-Chipana, I., Aguilar-Pelizzoli, N., Quiroz-Flores, J. C., Collao-Diaz, M., & Flores-Perez, A. (2021, November 3-5). Increasing profitability through the implementation of an integrated balance score card & objectives key results model in a digital agency: A research in Perú. International Conference on Industrial Engineering and Operations Management Monterrey, Mexico.
Saba, F. (2012). A Systems Approach to the Future of Distance Education in Colleges and Universities: Research, Development, and Implementation. Continuing Higher Education Review, 76, 30-37.
Saba, F. (2013). Building the future: A theoretical perspective. Handbook of Distance Education, Routledge.
Saba, F. (2014). Methods of study in distance education: A critical review of selected recent literature. Online Distance Education: Towards A Research Agenda, 151-171.
Sariman, G. (2014). Veri madenciliğinde kümeleme teknikleri üzerine bir çalışma: K-means ve K-Medoids kümeleme algoritmalarının karşılaştırılması. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 15(3), 192-202. https://doi.org/10.19113/sdufbed.41288
Saverimuttu, J. J. (2022). An exploratory study on effectively upskilling employees during COVID-19. Robert Morris University.
Sevinç, L., & Yıldırım, O. (2001). Stratejik insan kaynakları yönetiminde HR scorecard uygulaması. Dokuz Eylül Üniversitesi İşletme Fakültesi Dergisi, 5(2-1), 149-164.
Slater, S., Joksimović, S., Kovanovic, V., Baker, R. S., & Gasevic, D. (2017). Tools for educational data mining: A review. Journal of Educational and Behavioral Statistics, 42(1), 85-106. https://doi.org/10.3102/1076998616666808
Stray, V., Moe, N. B., Vedal, H., & Berntzen, M. (2023, January 3-6). Using objectives and key results (OKRs) and slack: A case study of coordination in large-scale distributed agile. The proceedings of the 55th Hawaii International Conference on System Sciences, University of Hawai'i, Manoa, USA.
Vozniuk, A., Govaerts, S., & Gillet, D. (2013, 15-18 July). Towards portable learning analytics dashboards. IEEE 13th International Conference on Advanced Learning Technologies, Beijing, China.
Warner, M. C. (2019). The corporate adult learner: A study of the effectiveness of asynchronous learning in a corporate workspace (Publication No: 13810103) [Doctoral Dissertation, Northeastern University]. ProQuest Dissertations and Theses Global.
World Economic Forum. (2020). The Future of Jobs Report 2020.
https://www.weforum.org/reports/the-future-of-jobs-report-2020/
Zhou, H., & He, Y. (2018). Comparative study of OKR and KPI, DEStech Transactions on Economics Business and Management. https://doi.org/10.12783/dtem/eced2018/23986
Published
2023-12-30
Section
Articles
How to Cite
Yılmaz, Y., & Üstündağ, M. (2023). E-Learning Performance Assessment Model Proposal for E-Learning Academies. International Journal of Curriculum and Instructional Studies (IJOCIS), 13(2), 404-423. https://doi.org/https://doi.org/10.31704/ijocis.2023.017
