Journal Search Engine
Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1225-6692(Print)
ISSN : 2287-4518(Online)
Journal of the Korean earth science society Vol.39 No.4 pp.378-387
DOI : https://doi.org/10.5467/JKESS.2018.39.4.378

An Analysis of Korean Science Education Environment for 20 Years of TIMSS

Youngsun Kwak*
Department of Earth Science Education, Korea National University of Education, Chungbuk 28173, Korea
Corresponding author: kwak@knue.ac.kr Tel: +82-43-230-3661 Fax: +82-43-232-7176
July 27, 2018 August 15, 2018 August 27, 2018

Abstract


In this research, the change of Korean middle-school science education environments is investigated through analyzing eighth graders’ survey data collected over the past 20 years of TIMSS. We extracted educational context variables that provide meaningful information on changes of Korean science education, and have been surveyed more than 3 study cycles up to TIMSS 2015. The selected educational context variables include school resources and school climate from the school principal’s questionnaires, and teacher characteristics and instructional activities from the teacher’s questionnaires. For each context variable, we analyzed its trend over TIMSS cycles, and discussed its implications in light of Korean educational policy and curriculum changes. Based on the results, we recommended several ways that help to improve science teaching and learning in light of lab assistants, computer availability, teacher learning community, and middle school Earth science curriculum.



초록


    Introduction

    TIMSS (Trends in International Mathematics and Science Study) has measured student achievement in mathematics and science at fourth and eighth grades every four years since 1995. TIMSS is one of the representative international student achievement studies that aim to examine educational outcomes of countries all over the world, and to provide implications for improving math and science education policy. TIMSS, conducted by IEA (International Association for the Evaluation of Educational Achievement), has assessed student achievement in mathematics and science based on participating countries’ mathematics and science curricula. Korean fourth graders participated in TIMSS 1995 and then discontinued participation until TIMSS 2011.

    Korean eighth graders have participated in TIMSS for 20 years, from TIMSS 1995 to TIMSS 2015, where they have provided information for improving education policy and education quality by comparing Korean students’ math and science achievement as well as investigating and analyzing student and school level educational context variables that influence students’ achievement. TIMSS has provided information on the effectiveness of the participating country’s curriculum and school education based on students’ performance in mathematics and science (Sang et al., 2017). Korean eighth graders have ranked among the highest in mathematics and science achievement since TIMSS 1995 (Kim et al., 2012; Kim et al., 2014).

    TIMSS 2015 is the sixth assessment in the TIMSS series monitoring 20 years of trends in educational achievement, together with comprehensive data on students’ contexts for learning mathematics and science (Mullis, Martin, and Loveless, 2016). Through multi-dimensional analyses on the TIMSS results, therefore, we can examine Korean students’ achievement at the international level and deduce implications for development of Korean science education (MOE, 2016; Sang et al., 2017). Middle school students from 39 countries participated in TIMSS 2015, and the results of TIMSS 2015 study were announced at the end of November 2016. Major results of Korean eighth graders are as follows:

    The average science achievement of TIMSS 2015 is similar to that of TIMSS 2011, and Korean eighth grade students ranked the forth in science (Sang et al., 2017). Korean eighth graders have maintained highlevel performance, from the third to the fifth place.

    Although Korean eighth graders have been top performers in science achievement compared with international average, their affectional attitudes such as confidence, interest and value awareness of science have ranked the lowest level over the past 20 years, which is been a serious problem (Cho et al., 2012; Choe et al., 2013; Sang et al., 2017). Korean eighth graders took the lowest place in light of affectional attitudes towards science in TIMSS 2007, TIMSS 2011, and TIMSS 2015. We, therefore, need further studies to analyze causes for students’ negative attitudes towards science and to provide follow-up measures since Korean students like other Asian students persistently showed low levels of attitudes towards science (Kim et al., 2014; Lee, 2016).

    In this study, therefore, we will examine and reflect on changes in Korean science education environments over the past 20 years using TIMSS data, and investigate ways of future development. The goals of this research are to explore changes in Korean middleschool science education environments by analyzing eighth graders’ survey data collected over the past 20 years of TIMSS. Table 1

    Methods

    TIMSS updates and revises survey frameworks and items every cycle to reflect changing educational environments and theories. In addition, major variables affecting student achievement are surveyed for each cycle recurrently for the trend analysis. In this research, therefore, we extracted educational context variables to explore changes in Korean science education environments based on the following criteria.

    First, we extracted educational context variables that provide meaningful information on changes of Korean science education, and have been surveyed more than 3 TIMSS cycles up to TIMSS 2015 by analyzing survey questionnaires for school principals and teachers. In addition, among those educational context variables surveyed more than 3 study cycles, we excluded variables irrelevant to science education, including ‘Calculator use in math classes’, ‘Educational resources for students with learning disabilities’, etc., and variables with little changes across assessment cycles such as ‘Library resources related to science classes’, ‘Frequency of science homework’, etc. The selected educational context variables to analyze Korean science education environments are summarized in Table 2.

    To explore changes in Korean science education environments, we extracted School Resources (Science laboratory and Assistance available with science experiments), and School Climate (School emphasis on academic success, School discipline problems) from the school principal’s questionnaires, and Teacher characteristics (Gender, Teacher participation in professional development in science, Teacher collaboration activities with colleagues), and Instructional activities (Computer activities during science lessons, Percentages of students taught the TIMSS Earth science topics) from the teachers’ questionnaires. The features of TIMSS survey data and the procedure for analyzing these TIMSS data are as follows:

    Since measuring trends is a central goal of TIMSS, the sample design also aims to provide accurate measures of changes in student achievement from cycle to cycle. The basic TIMSS sample design has two stages: schools are sampled with probability proportional to size at the first stage, and one or more intact classes of students from the target grade are sampled at the second stage (Mullis and Martin, 2013). The whole students in sampled classrooms are sampled, and the science teachers teaching the sampled classrooms are also sampled. Given this multi-level sampling method of TIMSS, we need to analyze TIMSS data by applying weights reflecting sampling design to calculate valid statistics that represent Korean schools and students.

    Regarding the analysis of TIMSS data, the analysis of teacher variables is somewhat complex. TIMSS teacher samples themselves are hardly recognized as representative samples that reflect the characteristics of Korean teachers. TIMSS teacher samples are not sampled directly from the whole Korean science teachers, but they are sampled based on sampled schools and classrooms. When analyzing teacher survey data, therefore, we need to analyze survey data by applying teacher weights, and should interpret analysis results in light of characteristics of students that teachers taught. That is, analysis results of teacher variables should be interpreted as the percent of students that teachers with specific attributes taught (Foy, 2017). Teacher survey results can be attributable to the sampled schools and students.

    In this research, we used IDB Analyzer that provides statistical analysis in accordance with TIMSS study design and sampling methods to analyze survey data on the school principal’s and teacher’s responses. The IEA’s IDB Analyzer is a standalone application that creates SAS and SPSS syntax to merge and conduct analysis with large-scale assessment data, and it produces SPSS Syntax considering final weights and replicate weights. The IEA’s IDB Analyzer is a tool that allows us to conduct analysis taking into account the sample and assessment designs. When using the IDB Analyzer, we need to combine the sampling and measurement variance to obtain our standard error. The analysis results through the IDB Analyzer, therefore, need to be carefully interpreted since the IDB Analyzer produces statistically significant difference by considering final weights and replicate weights. That is, based on the sampling and assessment designs, the IDB Analyzer automatically selects the appropriate weight and replication variables for each analysis, which produces different analysis results from those produced by SPSS with unweighted descriptive statistics for all the variables in the analysis.

    Results and discussion

    1. School resources and school climate

    1) Schools’ science laboratory and assistant for science experiments

    Regarding science education resources, TIMSS has surveyed each school’s securement of science laboratory and assistants for science experiments since TIMSS 2007. Since TIMSS 2007, most of Korean schools have established science laboratories. Regarding recruitment of science laboratory, 5 top performing Asian countries such as Korea, Japan, Singapore, Chinese Taipei, and Hong Kong secured science laboratories by 98~100%, which means most of the students attended schools with science laboratories. Table 3

    Regarding recruitment of lab assistants for science experiments (lab assistants, hereafter), however, the percent of students attending schools having science assistants has gradually decreased from 46.1% in TIMSS 2007 to 35.3% in TIMSS 2015.

    Table 4 shows the recruitment state of lab assistants for science experiments in Korean primary and secondary schools up to 2014, when TIMSS 2015 main study was conducted (MOE, 2016). Data from the MOE and TIMSS survey results about securement of lab assistants are consistent. The securement of lab assistants depends on science education policies and budget conditions of the Metropolitan and Provincial Offices of Education.

    According to the previous studies, increasing science inquiry activities with greater students’ autonomy can improve students’ affectional domain, which in turn helps increasing students’ science achievement (Lim and Lee, 2016). In this context, we should prepare various conditions for science inquiry and hands-on activities by securing lab assistants since these lab and inquiry activities provide students with positive science experiences (Shin et al., 2017; Kim et al., 2017).

    2) School emphasis on academic success

    TIMSS has surveyed the school’s emphasis on academic success with school principal’s questionnaires since TIMSS 2003. In this research, we used the common items used between TIMSS 2003 and TIMSS 2015 to analyze trends in the school’s emphasis on academic success (Refer to Table 5).

    Items measuring the school’s emphasis on academic success consist of two aspects: Teacher’s and Parent’s aspects. Regarding items related to the teacher’s aspects, their average scale score has dropped consistently between TIMSS 2003 and TIMSS 2015, which means that principals reported their teachers’ emphasis on academic success has decreased over the past 12 years. The average scale score related to the parental aspects, however, has fluctuated over the past 12 years. Regardless of the fluctuation in parental emphasis on academic success, the emphasis on parental involvement in school activities has reached a certain level with the settlement of the school steering committee as a legal advisory body in the private as well as public schools since 2000 (MOE, 2016).

    3) School discipline problems

    TIMSS surveyed school discipline problems such as arriving late at school, theft, physical injury, etc., through the school principal questionnaires. TIMSS surveyed this variable over the past 20 years, from TIMSS 1995 to TIMSS 2015, but changed response scales depending on the TIMSS cycle. For example, in TIMSS 1995 principals were asked to report the frequency of each discipline problems, and since TIMSS 1999 principals were asked to report the severity of each discipline problem. Therefore, we excluded TIMSS 1995 survey results since they cannot be compared with those of the other study cycles.

    Regarding School discipline problems, principals reported average scale score with 1.19~1.73 over the past 16 years for items such as ‘Arriving late at school, Absenteeism, Classroom disturbance, Cheating, Profanity, Vandalism, Theft, Intimidation or verbal abuse among students, Physical injury to other students, Intimidation or verbal abuse of teachers or staff, Physical injury to teachers or staff’, which means most of Korean students were in schools with hardly any or minor discipline problems. In addition, depending on the TIMSS study cycle, the response results of severances of the discipline problems were fluctuating, which does not show any consistent trend with time change. In sum, Korean principals reported that Korean middle schools had hardly any or minor school discipline problem, and therefore school discipline problems do hardly affect school education environments. Table 6

    2. Teacher characteristics

    1) Gender of science teachers

    Percent of students taught by female science teachers for each TIMSS study cycle is shown in Table 7.

    Percent of students taught by female science teachers in TIMSS 1995 was 49.3%, and the percent significantly increased up to 60% in TIMSS 1999 and TIMSS 2003. In TIMSS 2007 the percent of students taught by female science teachers dropped slightly, but the percent of students taught by female science teachers has stayed at above 60% since TIMSS 2003, That is, percent of students taught by female science teachers has increased over the past 20 years. During this period, with the abolition of extra points given for men’s completion of military service in the teacher selection test and so on, percent of female candidates reached or held a majority of successful candidates in the public official recruitment exam as well as in the teacher selection test since 2001. Like mathematics, therefore, percentage of students taught by female science teachers has increased and reached more than 60% since 2003.

    2) Teacher participation in professional development in science

    Table 8 shows percent of students by teachers’ area of professional development over the four TIMSS cycles, from TIMSS 2003 to TIMSS 2015.

    Percent of students whose teachers reported their participation in professional development has consistently increased since TIMSS 2003. The significant increase in ‘Science curriculum’ activity seemed to be resulted from the abundant supply of curriculum related inservice training courses with the introduction of the 2009-revised science curriculum during this period.

    However, percent of students whose teachers reported their participation in professional development for ‘Integrating ICT into science’ dropped from 44.0% in TIMSS 2003 to about 30% in TIMSS 2007 and TIMSS 2011, and increased again up to 45.5% in TIMSS 2015, which reflects an increased emphasis on integrating technology into science lessons. This trend also reflects some impacts of national educational policies, including ‘Ways to activate primary and secondary school software education (MOE, 2014)’ and ‘Plan for human resource development for software-centered society (MOE•MSIFP, 2015), and so on, which emphasized use of software in science as well as ICT-related courses. These policies increased teachers’ demand for ICT-related expertise and made teachers participate in such professional development activities as ‘Integrating ICT into science’.

    3) Teacher collaboration activities with colleagues

    Collaborations and interactions with other teachers are teachers’ professional development activities by teachers themselves within schools. TIMSS surveyed science teachers’ interactions with other teachers over four TIMSS cycles, from TIMSS 2003 to TIMSS 2015. For these activities, percent of students by their teachers’ area of interactions are calculated by combining teachers’ responses to ‘Very often’ and ‘Often’ scale (Refer to Table 9).

    Percent of students for all three items significantly increased in TIMSS 2015. In particular, percent of students whose teachers interacted with other teachers through ‘Visiting another classroom to learn more about teaching’ increased the most significantly, from 1.4~2.8% in previous TIMSS cycles to 35.2% in TIMSS 2015. This change in teachers’ collaboration through classroom visiting seemed to be resulted from the exam-free semesters introduced in 2013 and fully implemented from 2016, official implementation of the teacher appraisal since 2010, and so on. The analysis result shows that educational context variables related to teacher characteristics have been improved over the past 20 years. Teachers' collaborations with other teachers have recently increased with the spread of the teachers’ professional learning communities (PLC) (Kwak, 2016).

    3. Instructional activities

    1) Computer activities during science lessons

    TIMSS surveys computer availability for students’ use in science lessons, and types of students’ computer activities during science lessons since TIMSS 2007. Table 10 shows the availability of computers for students’ use in science lessons. Percent of students whose teachers responded they have computers available for students to use in science lessons has consistently decreased since TIMSS 2007, from 77.2% in TIMSS 2007 to 50.4% in TIMSS 2015. From this result, Korean science classes should secure computers available for teaching and learning purposes.

    TIMSS also surveyed ‘Types of computer activities’ during science lessons. Table 11 shows the percent of students whose teachers have them use computers at least monthly. Percent of students who were asked to use computers at least monthly for various activities has significantly decreased since TIMSS 2007, from 66.4~81.4% in TIMSS 2007 to 49.7~59.0% in TIMSS 2015.

    This result seemed to be related with the decrease of available computers for science lessons since TIMSS 2007 (Refer to Table 10). With the decrease of percent of available computers for science lessons, the percent of students being able to use computers for all types of science class activities also decreased since TIMSS 2007. We, therefore, need an in-depth research on ways to secure computers for science lessons, actual conditions of computer usage, and ways to improve computer use for science learning purposes.

    2) Percentages of students taught TIMSS Earth science topics

    TIMSS surveys whether teachers have taught main topics addressed by the TIMSS science test. In this research, we compared percent of Earth science topics taught in science classes over the TIMSS cycles, which provides information on the extent to which the TIMSS Earth science assessment matches Korean curricula and teachers’ science instructions. Table 12 shows the trend in percent of Earth science topics taught in science classes over the past four cycles of TIMSS.

    In light of applied curricula, assessment topics such as ‘Earth’s structure and physical features’, and ‘Earth’s processes, cycles, and history’ have been included in the primary or grades 7-8 science curriculum in the 7th and 2009-revised curricula, which resulted in a consistent high match level between the TIMSS test and the coverage in science classes. For such assessment topics as ‘Earth’s resources, their use and conservation’ were covered in the primary school curriculum in the 7th curriculum, and were not explicitly covered in the 2009-revised middle school science curriculum. The ‘Earth’s resources, their use and conservation’ topic is, however, measured in an integrated item format combining various science content domains, and this integrated item format caused different teachers’ responses depending on teachers’ background majors over the TIMSS cycles. The ‘Earth in the solar system and the universe’ topic was covered in grades 7-8 in the 7th curriculum, and covered in ninth grade in the 2009-revised curriculum, which resulted in a significant decrease of teachers’ responses. In summary, the percent of Earth science topics taught in science classes for each TIMSS assessment topic has a better match when the topic has been included in the Korean science curriculum.

    Conclusion

    In this research, we investigated changes in Korean science education environments over the past 20 years with accumulated TIMSS data. According to the result, most of the Korean students attended schools with science laboratories while the recruitment of lab assistants for science experiments has gradually decreased since TIMSS 2007. Regarding the school’s emphasis on academic success, teachers’ part has decreased consistently since TIMSS 2003 until TIMSS 2015. Regarding teacher characteristics, percentage of students taught by female science teachers has increased and reached more than 60% since 2003. Percent of students whose teachers reported their participation in professional development as well as teachers’ interactions and collaborations with other teachers has consistently increased since TIMSS 2003. Regarding science instructional activities, computer availability for science lessons and science classroom activities using computers have consistently decreased since TIMSS 2007. Based on the results, ways to improve science curriculum and teaching & learning can be suggested:

    Frist, measures to secure lab assistants are necessary. We need to improve the lack of lab assistants to ensure students’ quality lab experiences. Although there is not an explicit relationship between the percent of staffing of lab assistants and students’ science achievement (Sang et al., 2017), Korean science teachers could hardly provide lab activities without lab assistants. Korean science teachers are overwhelmed with re-preparation of lab experiments, and cleaning of labs for the next class within a 10- minutes recess without lab assistants, which makes them hard to keep quality science lab classes (Sang et al., 2017). We should, therefore, devise solutions to secure various conditions, including lab assistants, enabling students’ positive science experiences through inquiry activities, and improvement of science achievement.

    Second, students’ accessibility to ICT should be improved in school science lessons. This research showed that lack of computers available for students use in science lessons is a serious problem. Considering the high emphasis on ICT competencies in the 2015- revised curriculum, we need an in-depth research on ways to secure computers for science lessons, and ways to use computers for quality science experiences during science lessons. Science lessons should provide students with ICT-related activities where they can conduct scientific experiments, study natural phenomena through simulations, etc. using computers and online resources so that they can adapt to a future information-intensive society. Along with increasing students’ accessibility to ICT in science lessons, we also need to improve science teachers’ expertise to use and integrate ICTs with science activities, and to develop ICT-related science teaching and learning contents.

    Third, it is necessary to emphasize teachers’ learning communities to encourage in-service science teachers continuous professional development. Regarding teacher participation in professional development in science, there has been a significant increase in teachers’ participation in curriculum related in-service training courses with an introduction of new curricula. Regarding science teachers’ collaboration activities with colleagues, there is a new trend for teachers to collaborate in the professional learning community (PLC), including interactions with their colleagues in recent TIMSS cycles. Future school education aims for professional learning communities (OECD, 2012). Korean science teachers, therefore, should develop their teaching professionalism and competencies by constructing PLCs (Kwak, 2016; Seo, 2015). Through PLCs, science teachers should collaborate to reflect on their practices and to improve science teaching, which will in turn improve student learning.

    Fourth, we need to conduct an in-depth research and improve when and how to introduce major Earth science topics in the science curriculum, especially for those topics with significant mismatch between the TIMSS test and the coverage in the Korean science curriculum. For some topics such as ‘the solar system in Earth science’, there has been consistent mismatches between the TIMSS test and the coverage in the Korean science curriculum. It is not necessary to revise Korean science curriculum in accordance with the TIMSS assessment framework, but we could update our science curriculum using the TIMSS curriculum model, which represents a higher degree of agreement with the intended curricula of the participating countries (Kwak, 2017).

    Figure

    Table

    Trends of science achievement of Korean eighth graders

    Educational context variables related to changes in Korean science education environments over the past 20 years

    Percent of students attending schools having science laboratory and assistants for science experiments

    Employment status of lab assistants of Korean primary and secondary schools between 2011 and 2014

    School emphasis on academic success

    School discipline problems

    Percent of students taught by female science teachers

    Percent of students by their teachers’ area of professional development

    Percent of students by their teachers’ area of interactions with other teachers

    Computers available for students to use in science lessons

    Percent of students whose teachers have them use computers at least monthly for various activities

    Percentages of students taught TIMSS Earth science topics in science classes

    Reference

    1. Cho, J. , Kim, S. , Kim, M. , Ok, H.J. , Lim, H.M. , and Son, S.K. , 2012, Ways of improving Korean students°Ø affective characteristics based on PISA and TIMSS results. KICE Research Report CRE 2012-4, 273 p. (in Korean)
    2. Choe, S.H. , Ku, J. , Kim, J. , Park, S. , Oh, E. , Kim, J. , and Baek, H. , 2013, Strategies for improving the affective characteristics of Korean students based on the results of PISA and TIMSS. KICE Research Report RRE 2013-8, 406 p. (in Korean)
    3. Foy, P. , 2017, TIMSS 2015 user guide for the international database. Chestnut Hill, MA: TIMSS & PIRLSInternational Study Center, Boston College, 90 p.
    4. Kim, S. , Kim, K. , and Park, J.H. , 2014, The effect of mathematics achievement on changes in mathematics interest and values for middle school students . Journal of Research in Curriculum & Instruction.18(3), 683- 701. (in Korean)
    5. Kim, S. , Park, J.H. , Kim, H. , Jin, E. , Lee, M. , Kim, J.Y. , Ahn, Y., K. , and Seo, J.H. , 2012, Findings from TIMSSfor Korea: TIMSS 2011 international results. KICE Research Report RRE 2012-4-3, 460 p. (in Korean)
    6. Ku, J. , Cho, S. , Lee, So. , Park, H. , and Ku, N.W. , 2017, OECD Programme for International Students Assessment: An in-depth analysis of PISA 2015 results. KICE Research Report RRE 2017-9, 158 p. (in Korean)
    7. Kwak, Y. , 2016, Exploration of features of cross-curricular instructional consulting in middle school science lessons through case study . Journal of the Korean Association for Science Education, 36(2), 269-277. (in Korean)
    8. Kwak, Y. , 2017, Exploration of features of Korean eighth grade students’ achievement and curriculum matching in TIMSS 2015 Earth science . Journal of the KoreanAssociation for Science Education, 37(1), 9-16, (in Korean)
    9. Lee, J. , 2016, Analysis of changes in the learning environments of middle school science classes . Journal of the Korean Association for Science Education, 36(5), 717-727. (in Korean)
    10. Lim, S.A. and Lee, J. , 2016, Affective factors as predictors of math achievement: Comparison of OECD high performing 10 countries in math . Journal of EducationalEvaluation, 29(2), 357-382. (in Korean)
    11. Martin, M.O. , Mullis, I.V.S. , Foy, P. , and Hooper, M. , 2016, TIMSS 2015 international results in science. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2015/international-results/
    12. MOE, 2014, Basic plans for the fifth educational information (2014~2018). MOE (2014.10.). (in Korean) MOE, 2016, General plans for science education (2016.2.). (in Korean) MOE•MSIFP, 2015, Promotion plan for educating human resources for SW centered society. MOE (2015.7.21.). (in Korean)
    13. Mullis, I.V.S. , and Martin, M.O. , 2013, TIMSS 2015 assessment frameworks. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
    14. Mullis, I.V.S. , Martin, M.O. , and Loveless, T. , 2016, 20 Years of TIMSS: International trends in mathematics and science achievement, curriculum, and instruction. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and IEA, 90 p.
    15. OECD, 2012, Teaching practices and pedagogical innovation: Evidence from TALIS. OECD Publishing. http://dx.doi.org/10.1787/9789264123540-en
    16. Sang, K. , Kwak, Y. , Park, J.H. , and Park, S. , 2016, The Trends in International Mathematics and Science Study (TIMSS): Findings from TIMSS 2015 for Korea. KICE Research Report RRE 2012-4-3, 342 p. (in Korean)
    17. Seo, K. , 2015, Teacher learning community: An approach for collective professional development. Seoul: Hakjisa. 294 p. (in Korean)