Language Testing: Test construction, administration

• All tests require time and money for preparation, administration, scoring, and interpretation.
• Practicality and the ideal testing solution may conflict due to limited resources.
• Chapter 6 further discusses this issue.
• The key testing challenge involves creating tests that are valid, reliable, practical, and have a positive influence on teaching.
• The book's next four chapters examine relevant concepts to assist readers in defining problems and finding solutions.
• The second half provides detailed advice on test construction and usage, applying the principles outlined earlier.
• Chapter 7 describes and illustrates the stages of test construction.
• Chapter 8 explores various testing techniques.
• Chapters 9-13 detail how to effectively test different language skills, particularly in educational settings.
• Chapter 14 gives straightforward advice on test administration
• Statistics are important for understanding testing and problem-solving.
• Statistical content is kept to a bare minimum and is presented simply to avoid overwhelming readers.
• Emphasis is on interpretation over calculation.
• Appendix 1 guides advanced readers with statistical operations.
• The collection of approximately 50 English language test reviews, edited by Alderson, Krahnke, and Stansfield (1987), show how well test writers address problems.
• Chapters 3, 4, and 5 are essential for a thorough understanding of the reviews.

Purposes of Language Testing

• Language testing serves to obtain information, categorized for specific situations.
• Categorization helps determine whether an existing test fits a purpose or whether new tests are needed.
• Proficiency, achievement, diagnostic, and placement tests are the 4 types discussed.

Proficiency Tests

• Designed to gauge language ability regardless of training.
• Content is not tied to specific language courses but to what is needed to demonstrate proficiency.
• 'Proficient' may mean sufficient command for a specific purpose, such as UN translator
• Tests determine if English is adequate for British university study, possibly tailored by subject area (arts, sciences).
• Content is specified early in the test's development.
• Some proficiency tests do not target a specific occupation/course and concept of proficiency is general.
• Cambridge and Oxford EFL exams are British examples (First Certificate, Proficiency, Preliminary, Higher)
• These tests assesses if candidates meet set standards for specific abilities.
• Examining bodies are independent, allowing fair comparison among candidates from various institutions/countries.
• Tests should have detailed specifications outlining what successful candidates can demonstrate.
• Users can assess test suitability/interpret results, more than "vague" proficiency notions matter.
• Proficiency tests are not based on prior courses but influence course content and teaching methods.
• Wide used tests may have harmful effects, and teachers can influence the testing organizations.
• The recent revisions to TOEFL is is a case in point.

Achievement Tests

• Unlike proficiency tests, achievement tests relate directly to language courses and their effectiveness/success
• There are two forms: final achievement tests and progress achievement tests.
• Final achievement tests are given after a course.
• Content is related to courses taught, but the relationship is debatable.
• Syllabus-content approach bases tests on detailed course syllabi/materials to make them fair.
• The downside is tests can be misleading if syllabus is bad. Good test performance doesn’t guarantee actual objective achievement.
• Alternative is basing test content on course objectives.
• Basing test content on course objectives pushes designers to clarify objectives.
• Performance then reflects objective achievement which puts pressure to keep syllabus and books aligned
• Objective-based tests counteract bad teaching, it offers reliable data on group/individual success and promotes positive change

Test Content

• Basing test content of course objectives may seem unfair if course deviates.
• Basing tests on content from bad courses misleads on students’ achievement and course quality.
• Objective-based tests give useful info and incentivize fix the unsatisfactory course.
• Objective-based tests serve students better long term due to pressure for syllabus change.
• Test should relate to language needs like proficiency ones, so expect no difference.
• Objectives and needs don't usually correlate, many test not based on course objectives.
• Users must know how tests constructed and validity/applicability
• Writers should align tests to course objectives rather than general proficiency tests.

Progress Achievement Tests

• Intended to measure students' progress.
• 'Progress' means meeting course objectives so tests also relate to them.
• Ideally, administer many achievement tests to chart increasing results, proving growth.
• Low scores discourage students/teachers early on.
• Alternative is setting well-defined short-term objectives which guide progress to the final test.
• If syllabus suits objectives, then tests fit well with lessons.
• If not, pressure is applied to make a better fit.
• The tester clarifies issues when there are syllabus faults, not on the tests.
• Teachers use informal "pop quizzes" to assess progress, keeping students sharp
• Don't follow strict assessment, should measure short-term goals like formal tests.
• Help chart teacher's route to wider objective achievements.
• Objective-based content is better than course content to fellow educators.
• Persuading people is a task because change intimidates many because it may take skill, tact and political navigating.

Diagnostic Tests

• Used to identify student strengths and weaknesses and what needs to be taught.
• Easy to find weaknesses in language skills like speaking over reading
• Tests can assess sample writing to indicate 'grammatical accuracy'.
• Grammar command hard to assess (past tense vs perfect tense) so needing multiple context examples.
• Comprehensive English test would be huge think of modal verbs.
• Few tests exist only for diagnostics due to test size.
• Lack of tests is a problem, valuable for individualized/self-instruction and showing learner gaps to target
• Readily available computers will adjust the situation so learners obtain knowledge with less time.
• Work will still need a tremendous amount of work to produce and publish them.

Placement Tests

• Place students which teaching phase suits ability
• Common to place in different level classes.
• Tests aren't recommended as concerned is not certain test suits their teaching.
• Initial assumption must be tests don't work well
• Best placement tests are situation tests, hinge on recognizing features across levels to tailor
• "In house" tests rewarded by the construction time, but save time, greater accuracy, see Chapter 7 for validation, 4 for reference

Direct and Indirect Testing

• Direct Requires performing desired skill, like composition writing.
• Direct requires tasks and text as authentic as possible, but known to be test, tasks cant be authentic, remain realistic
• Direct testing easier for productive skills to see performance.

Testing Listening and Reading

• Testers need to find way to see if candidates listening or reading been successful.
• Chapters 11 and 12 discuss methods to get accurate test results without disrupting skills.
• Many tests for productives skills deemed tough to measure due to reliability reasons which are NOT insurmountable
• Direct offers attractions, easy to form conditions to take action, gauge assessment/interpretation of skill.
• Test practice can foster practice of skills and can be helpful backwash.
• Indirect try to measure basic skill, like TOEFL section developed for indirect writing.
• Candidate identifies standard erroneous English in underlined text.
• While relates to writing ability, statistical link is weak, the action is not direct.

More on Testing

• Lado (1961) wants to test pronunciation by a paper and pencil, identifying rhyme and the other element.

Indirect Testing

• Possibly tests a representative part of abilities that underlie manifestations of the skill indefinitely.
• Representative number of grammar tests covers all scenarios with grammar.
• Direct limits on sample to tasks, possible range to grammatical structures.
• On the argument, indirect results are more generalizable.
• Main issue is connection among performance and skill tends to be weaker, unreliable.
• We don't know enough about compositions to derive its ability from measures.
• Develop measures of grammar, vocab, discourse, and punctuation.
• Cant predict composition correctly, ensure scores are representative in multiple exams.

Testing Preferences

• Now testing is preferable, proficiency linked (achievement in tests) can expect high estimates of abilities or by sampling more widely
• Direct also reinforces view to institution tests, potential beneficial Backwash
• Commit fully to tests and always include indirect to obtain diagnostic, like grammar test

Discrete Point vs. Integrative Testing

• Discrete = Test one element item for item, like tests on grammar structures.
• Integrative = Combines elements in task, writing piece, note make, writing dictation passage.
• Discernment of point will see test, tests tend to, see.
• Integratives like cloze are indirect.

Norm-Referenced vs. Criterion-Referenced Testing

• Imagine taking test, how was student performance; two forms
• The first sees the student performance compared to takers.
• The test is benchmark, relative.
• You’re not told, what student does with language.

Criterion-Referenced Testing

• Reading example from Interagency Language Roundtable (ILR)
• Sufficient listening to read print authentic materials in context.
• Able to read facts with misunderstandings.
• Can find general ideas.
• Can read prose general, news simple, bio, social, business, or context.
• Test sees ability with context, text.
• Can have general vocabulary with world cues to comprehend
• Level 1 german, speak and react with others easily.
• Greet, interact, take leave
• Swap home, school info
• Discuss, decide, plan
• Ask advice
• Ask, understand.

Reference

• Does what others cannot, can also complete tasks for benchmark test, also matters results, not who performs.
• "Those with tests encourage progress and don't make students feel doomed if weaker than others. All students successful. Don't change.
• Standard tests are meaningful, drive students to targets.
• Direct translates performance, construction from benchmark, both give similar purpose
• Assess language as needed for Uni? Tests need university analysis.
• Not able to use for results. English Michigan test, has gram, vocab.
• Tests don't tell you the need for english.

Testing Advice

• Students should be recommended in academic tables and students
• University interpret, learn.
• Testing books give advice,
• The norm references are established,
• Not to exclude those tests and should be considered.

Objective vs. Subjective Testing

• Distinction = Method.
• No judgement scorer means grading is objective.
• Multiple choice
• Judgement means grading is said to judge
• Impressive is subjective.
• Objectivity is reliability.

Objectivity in Testing

• Less subjective scorers are greater with graders also test takers.
• Ways to score good ratings, which is talked about later.

Communicative language Testing

• Discussed ability to act, which leads the testing. That results significance I am talking.