Han?Yiqiong

    【Abstract】The College English Test-Spoken English Test （CET-SET） is a nationwide spoken English test including two different bands – CET-SET 4 and CET-SET 6. It has been designed to test the oral communicative ability of university or college students in China. This article intends to evaluate CET-SET 6 in details as there are limited studies on it. With the general description of CET-SET， this article will evaluate the test in terms of its reliability and validity and give suggestions to modify CET-SET 6.

    【Key words】CET-SET 6; reliability; validity; suggestions

    【作者簡介】Han Yiqiong， Guangzhou University Sontan College.

    1. Introduction

    The College English Test-Spoken English Test （CET-SET） is affiliated to the College English Test （CET） - a criterion-related norm-referenced test in China. As one of the large-scale and high-stake standardized language test in China， CET has been more frequently researched over the past ten years （Cheng， 2008）. Researchers have studied CET from different aspects： Ren （2011） used questionnaire and interview to find out the effects of CET on English education in universities of Tianjin City; He and Dai （2006） studied CET-SET group discussion based on a corpus of test performance and found the candidates interaction is at a low degree in the group discussion; Li （2009） studied CET writing and found the teachers did not teach to the test. However， few researchers evaluate the reliability and validity of CET-SET 6 in detail， which this paper intends to do.

    2. The Description of CET-SET 6

    Authorized by Ministry of Education， the National College English Testing Committee （NCETC） has administered CET since 1987. CET aims to assess the fulfillment of College English Teaching Syllabus in college， measure the English communication ability of non-English majors at the tertiary level and provide feedback to teachers and students （Wang， Yan， and Liu， 2014）. It has been said to be a high-stake test because CET-4 certificate is a prerequisite for graduation or bachelors degree in many universities in China （He and Dai， 2006; Ren， 2011; Wang， Yan， and Liu， 2014：）. CET is comprised two levels of tests： CET-4 is to check the basic requirements in College English Curriculum Requirement （CECR）; CET-6 is to check the intermediate requirements （Du， 2012）. Before 2016， the total score was shown on CET report with three sub-scores from writing and translation， listening， and reading. Speaking was absent from the test report because speaking ability was ignored. In 1999， NCETC started CET-SET which tests English communication ability of students from higher education in China. From 2016， candidates can take CET-SET 6 without any limitation. Since the interview test was replaced by the Internet-based one， this article will only focus on the Internet-based CET-SET 6.

    The Internet-based CET-SET 6 is conducted in a small group including one virtual examiner who in fact works as an interlocutor and two candidates. The oral test consists of three parts. In the first part， candidates take turns to give a self-introduction in 20 seconds， and then they answer one question asked by the interlocutor simultaneously in 30 seconds. The question would be related to the topic of the test. The second part is individual presentation and group discussion. In this part， every candidate take turns to make a 90-second presentation with a given a visual prompt （sentences or pictures） on the screen after 60-second preparation. The information given to the candidates in the same group are about the same topic. After the presentations， candidates are instructed to take part in the group discussion on the given topic and try to reach an agreement in the end. In the third section， two candidates simultaneously answer a question about the given topic from the interlocutor in 45 seconds. The total length of the test is around 18 minutes.

    3. The Evaluation of Reliability in CET-SET 6

    If the scores would have been more similar， the test is said to be more reliable （Hughes， 2003）. In other words， reliability means the consistency in scores regardless of when and how many times a particular test is taken. The reliability of CET-SET 6 is achieved by the practices of the standardized administration procedures， the format of the interlocutors engagement， and the standardized rating procedures （Zhang and Elder， 2009）. The procedures of the testing are regulated and organized for the interlocutor and candidates so it makes no difference on the test score whether the time or the location the candidates take the test. The interviewer variability is also confirmed to have influence on the test and scores （Van Moere， 2006）. However， in CET-SET 6， the interviewers engagement in CET-SET 6 will not affect the test result， since the interviewer simply reads the instructions and questions in the test.

    Besides， the standardized scoring procedures can minimize variations in the process of scoring and the scoring criterion （Zhang and Elder， 2009）. The candidates performance in CET-SET 6 is scored by the authorized and trained raters with the formal rating scale designed on the requirements of CECR. The scoring criteria are also irreplaceable in the score reliability which is an essential component of the test reliability （Bachman and Palmer， 1996）. In CET-SET 6， the criteria are specified into three aspects： 1） accuracy and range; 2） length and coherence; 3） flexibility and appropriateness. The candidates performance is scored on a scale from 1 to 5 based on the criteria. With the scoring criteria and the standard samples for reference， rater can make a proper judgement on candidates performances. This analytical scoring can make an oral test achieve a higher score reliability （Li， 2011）. Moreover， test score in interview is inevitably subjective， which causes the inconsistency in the ratings （Bachman， 1990）. However， the potential sources of inconsistencies cannot be diminished entirely （Bachman and Palmer， 1996）.

    4. The Evaluation of Validity in CET-SET 6

    Validity means whether a test measures what it is intended to measure （Hughes， 2003）. It is the most important consideration in the test development， interpretation and use （Bachman， 1990）. Among the different types of test validity， the most common ones are content validity， criterion-related validity， face validity and construct validity （Li， 2011）.

    Content validity means the content of the test should contain a representative sample of a language skill， structures， etc. （Hughes， 2003）. A speaking test has content validity only if it tests a proper sample of the related structure such as dialogue or discussion etc.， which is easy to identify from the description of CET-SET 6. Besides， the three parts of CET-SET 6 actually test a variety of contents because candidates complete the tasks via description， negotiation， persuasion， debate， argumentation etc. which are required in CECR （Yang， 2003）. Above all， CET-SET6 has high content validity.

    Criterion-related validity is divided into concurrent validity and predictive validity. To get concurrent validity， testers need to compare the performance of a randomly sample of students with that of all students. The similarity between the two groups shows the concurrent validity of the test： the more similar they are， the higher the concurrent validity is. Predictive validity measures the degree to which a test can predict interviewees future performance. There is little research on the criterion-related validity of CET-SET.

    Face validity means the test looks as if it measures what it is supposed to measure （Hughes， 2003）. Face validity can be gained in a speaking test if the testing uses direct method such as dialogue， group discussion， role play etc.， by copying a similar context for the use of target language （Li， 2011）. CET-SET 6 tries to provide a real-life interactive context for achieving the face validity. However， it is difficult to judge whether it has high face validity. Since the test interview is conducted in a small group， each candidate has his different attitude towards the test.

    Construct validity refers to the extent of the consistency between the performance on test and test purpose （Bachman， 1990）. If the test measures the ability which it is intended to measure， it is said to have construct validity. As the performance does not fit the types of test task and result， the construct validity of CET-SET 6 is considered relatively low （Jing and Ma， 2012）.

    5. The suggestions to CET-SET 6

    With the above discussion， there are suggestions to modify CET-SET 6. Firstly， Ma （2014） concluded that the Spoken English test should be included as one inseparable part of the entire College English Test for the sake of validity of a test. This is also one of my suggestions to CET-SET 6. Secondly， CET-SET 6 needs to involve more question-and-answer interaction in the third part， which is also proposed by Lei （2019） in his study on IELTS speaking test and CET-SET 4/6. Since daily communication seldom stops with only a question and answer， the third part should have more interaction. Thirdly， CET-SET 6 needs to enrich its content by referring to the content of IELTS Speaking Test （Lei， 2019）. CET-SET 6 should choose hot issues of daily life to allow candidates to express their ideas and interact with each other so as to better assess candidates communicative ability. This will enhance the reliability and validity of CET-SET 6 and achieve the test aim as well. Lastly， raters can work with the computer-automated scoring system because the differences between raters rating are unavoidable. It may make the rating more complicated， but it helps to ensure the reliability and validity of the test score.

    References：

    [1]Bachman， L. Fundamental considerations in language testing[M]. Oxford： Oxford University Press， 1990.

    [2]Bachman， L. & Palmer， A. Language testing in practice[M]. Oxford： Oxford University Press， 1996.

    [3]Cheng， L. Y. The key to success： English language testing in China [J]. Language Testing， 2008，25（1）：15-37.

    [4]Du， H. College English teaching in China： responses to the new teaching goal[J]. TESOL in Context， 2012，22（S3）：1-13.

    [5]He， L. Z. & Dai， Y. A corpus-based investigation into the validity of the CET–SET group discussion[J]. Language Testing， 2006，23（3）： 370-401.

    [6]Hughes， A. Testing for language teachers[M]. Cambridge： Cambridge University Press， 2003.

    [7]Jing H. W. & Ma L. L. A Study of the CET-SET Validity[J]. Journal of Gansu Normal Colleges， 2012，17（6）：97-99.

    [8]Lei， X. The Implication on the Preparation and Testing on CET-SET from IELTS[J]. Overseas English， 2019，13：79-80.

    [9]Li， H. L. Are Teachers Teaching to the Test？ A Case Study of the College English Test （CET） in China[J]. International Journal of Pedagogies and Learning， 2009，5（1）：25-36.

    [10]Li， W. Validity Considerations in Designing an Oral Test[J]. Journal of Language Teaching & Research， 2011，2（1）：267-269.

    [11]Ma， F. College English Test： To Be Abolished or To Be Polished [J]. Journal of Language Teaching & Research， 2014，5（5）：1176-1184.

    [12]Ren， Y. A study of the washback effects of the College English Test （band 4） on teaching and learning English at tertiary level in China[J]. International Journal of Pedagogies and Learning， 2011， 6（3）：243-259.

    [13]Van， M. A. Validity evidence in a university group oral test[J]. Language Testing， 2006，23（4）：411-440.

    [14]Wang， C.， Yan， J. & Liu， B. An Empirical Study on Washback Effects of the Internet-Based College English Test Band 4 in China[J]. English Language Teaching， 2014，7（6）.

    [15]Yang， Y. L. On the Validity and Reliability of CET-SET[J]. Journal of Sanming College， 2003，20（4）：152-156.

    [16]Zhang， Y. & Elder， C. Measuring the speaking proficiency of advanced EFL learners in China： The CET-SET solution[J]. Language Assessment Quarterly， 2009，6（4）：298-314.

• 浅析铁厂河国有林场生态公益林管护
• 新疆裕民县天然林保护工程建设思考
• 封山育林在林业生态工程建设中的应用
• 试论营林技术在森林防火中的应用
• 植树造林技术与森林养护措施
• 林下经济发展现状及重点分析
• 智能种植技术在农业生态园的运用对策分析
• 论生态文明背景下的林业工程建设工作
• 创新林木细胞工程技术助推良种繁育产业革命
• 现代林业生态建设与林政资源管理探究
• 北方松树园林栽植与养护
• 林业工程建设中林木育苗技术要点研究
• 浅谈宁南城乡绿化树种规划与设计
• 几种生长调节剂在高海拔地区造林中的运用研究
• 林业种苗管理中常见的问题及措施研究
• 园林绿化施工质量控制措施分析
• 林政资源管理与林业生态建设探究
• 加强林政管理，促进林业持续发展探究
• 基于林政资源管理下的现代林业生态建设策略
• 林业森林资源林政管理的有效策略分析
• 北方果蔬作物贮藏、保鲜、加工的现状对策分析
• 园林花卉的无性繁殖技术
• 试论玉米胚芽油的新型脱蜡工艺技术
• 艺术设计在城市园林景观设计中的作用分析
• 当代农业发展中机械化秸秆运用技术推广应用
• well-accommodated
• well-accompanied
• well-accomplished
• well-accorded
• well-accredited
• well-accumulated
• well-accustomed
• well-achieved
• well-acquainted
• well-acquired
• well-adapted
• well-addicted
• well-addressed
• well adjusted
• well-administered
• well-admitted
• well-adopted
• well-adorned
• well-advanced
• well-advised
• well advised to
• well-advocated
• well-aged
• well-aimed
• well-aired
• 翩绵
• 翩翔
• 翩翩
• 翩翩公子
• 翩翩少年
• 翩翩巍巍
• 翩翩然
• 翩翩而舞
• 翩翩自喜
• 翩翩起舞
• 翩翻
• 翩若惊鸿
• 翩若惊鸿，婉若游龙
• 翩若惊鸿，宛若游龙
• 翩跹
• 翩跹而舞
• 翩跹起舞
• 翩跹飞舞
• 翩连
• 翩飘
• 翩飞
• 翫
• 翬
• 翭
• 翭矢