Yıldız Teknik Üniversitesi Yabancı Diller Yüksekokulu temel İngilizce bölümünde yazım portföyü değerlendirme sistemi ve okuyucular arası güvenirlik

ABSTRACTWRITING PORTFOLIO ASSESSMENT AND INTER-RATER RELIABILITY ATYILDIZ TECHNICAL UNIVERSITY SCHOOL OF FOREIGN LANGUAGES BASICENGLISH DEPARTMENTTürkkorur, AsumanMA Department of Teaching English as a Foreign LanguageSupervisor: Dr. Theodore S. RodgersCo-supervisor: Dr. Susan JohnstonThis research study investigated the use of writing portfolios and their assessment by raters.In particular it compared the inter-rater reliability of the portfolio assessment criteria currentlyin use and the newSRUWIROLRDVVHVVPHQWFULWHULDSURSRVHGIRU<ÕOGÕ]7HFKQLFDO8QLYHUVLW\School of Foreign Languages, Basic English Department. The perspectives of the participantson the portfolio assessment scheme and the criteria were also analyzed. This study wascondXFWHGDW<ÕOGÕ]7HFKQLFDO8QLYHUVLW\6FKRRORI)RUHLJQ/DQJXDJHV%DVLF(QJOLVKDepartment in the spring semester of 2005.Data were collected through portfolio grading sessions, focus group discussions andindividual interviews. The participants in the study were seven English writing instructorsFXUUHQWO\ZRUNLQJDW<ÕOGÕ]7HFKQLFDO8QLYHUVLW\6FKRRORI)RUHLJQ/DQJXDJHV%DVLFEnglish Department. The instructors scored twelve student portfolios on two differentsessions using the criteria customarily used in the institution and the new analytic criteria.Focus group discussions were held before and after the grading sessions. At the end of the1grading sessions, instructors were interviewed individually. Grading sessions, focus groupdiscussions and interviews were audiotaped and transcribed.The inter-rater reliability for both of the criteria types was calculated and found to bemarginal. The results of the statistical analysis revealed that there was no difference in resultsof inter-rater reliability between the groups in both of the grading sessions. However, analysisof the focus group discussion and interviews indicated that instructors would appreciate someform of more standardized, analytic and reliable criteria for portfolio grading.Key words: Writing portfolio assessment, inter-rater reliability, alternative assessment.2