Anadolu Üniversitesi Hazırlık Okulu yazılı anlatım değerlendirmesi için kullanılan iki sistemin güvenilirliğinin değerlendirilmesi

Title: Evaluation of The Reliability of Two Grading Systems for Writing Assessment at Anadolu University Preparatory School. Author: Nesrin Oruc Thesis Chairperson: Dr. Patricia N. Sullivan Bilkent University, MA TEFL Program Committee members: David Palfreyman Michele Rajotte Bilkent University, MA TEFL Program Evaluating student writing is always a difficult task for English teachers. Not only it is time consuming to read a seemingly endless stack of compositions, but it is also difficult to write a grade in red ink on the work of a student which represents so much of her/himself, and to put our students into categories such as "A" student, "C" student, or "F" student brings up the issue of the most effective ways of assessing student writing. Assessing students' writing has always been accompanied by arguments between writing instructors at Anadolu University. This is not only because of the long hours spent on grading compositions, but also because of the inconsistencies among the grades and the graders. The main reason for this problem is that at Anadolu University a Standard Grading Format (SGF) has not been used for writing assessment. Instead writing instructors have used their individual grading system to evaluate papers. The main purpose of this research study was to examine writing instructors' individual approaches to assessing writing and then to determine whether the use of a holistic scoring scale would result in an increase in the reliability of the writing assessment at Anadolu University Preparatory School.The participants were six writing instructors from Anadolu University Preparatory School teaching writing to different levels. Half of the instructors were male, half were female. The instructors had between 1 and 10 years experience in language teaching and they were all non-native speakers. The process was begun by giving 15 papers chosen to represent different grade levels from the Anadolu University Preparatory School placement exam to instructors to grade in their accustomed manner. Then, after a training given an SGF using a five band holistic scoring scale which was adapted to Anadolu University Preparatory School, instructors were again given the same 15 papers and were asked to use the new SGF to grade the papers again. Data were analyzed first by a One-way ANOVA in order to find the mean scores of each system. Then the Student T-Test was used to compare the mean scores of the systems. The results of the One-way ANOVA indicate that there is significant relationship between the grades given to the same paper by five different instructors before and after the training which means both of the systems were reliable within themselves. On the other hand, the Student T-Test results indicate that there is a large difference between the scores given to the same papers by the same instructors with two different writing assessment systems. Consequently, no clear judgement can be made as to which system is superior. With regards to the grades given by the instructors for individual papers, the results of qualitative analysis indicate that inconsistencies arise from individual instructors' writing assessment practices and that this may be lessened with holistic scoring.