An examination of the assessment and evaluation questions in grade 5-8 science textbooks with respect to scientific literacy proficiency levels defined in PISA

The aim of this research was to examine the assessment and evaluation questions in 5-8 grade science textbooks in Turkey according to the scientific literacy proficiency levels defined in PISA. The possible variation of the proficiency levels with respect to the grades and the subject areas defined in the curriculum, that is Living things and Life, Matter and Change, Physical Events and the Earth and the Universe, were also examined. The research was designed as a qualitative study and carried out via document analysis. The assessment and evaluation questions in the textbooks were examined through document analysis technique and descriptive analysis was used in the analysis of the questions. Findings were presented through frequencies and percentages, and were categorized and supported with tables and graphics. In order to ensure the reliability, the questions were examined independently by the researchers of the study, and were then compared and the consistencies were calculated. While scoring, sample questions and characteristics of the scientific literacy proficiency levels published by OECD were taken into consideration. In PISA, six levels of proficiency for scientific literacy were defined. While level 1 is considered as Low Performance, levels 2, 3 and 4 are considered as Baseline Performance, and levels 5 and 6 are considered as High Performance. Within the scope of this research, a total of 1209 assessment and evaluation questions were examined according to these proficiency levels. The findings showed that, almost 60% of the questions in the 5th, 6th and 7th grade textbooks, and slightly more than 65% of the questions in the 8th grade textbook were at level 2, indicating a baseline performance level. It was found that the number of questions at high performance levels were quite low in the textbooks at all grades. When examined in terms of subject areas, it is seen that the number of questions at high performance levels was higher in "the Earth and the Universe" learning area in comparison to the others. It was observed that the questions in the "Physical Events" subject area, which has the highest number of questions, were generally stacked at the 2nd level. As a result of these findings, it is recommended that in order for better achieve in PISA and develop scientific literacy, it is necessary to include a sufficient number of high-level questions in the textbooks. It is also recommended that studies examining the curriculum and nationwide exams within the scope of scientific literacy proficiency should be conducted as further research.