SUMMARY OF All groups
On subject
“Language Assessment Development”
By
Eka Resti Wahyuni
(100221404357)
Fitria Ningsih
(100221400412)
 |
|||
 |
|||
Topic : Constructing
Discrete-point Tests: Grammar Tests
Group : 1 (Henifah Wahyuni and Gita
Insani Maryam)
What are discussed under this
topic:
v
Grammar and Grammatical Ability in The Concept of
The Discrete-point Approach
v
The Scope of Grammar in Language Testing
v Different Types of
Tests that can be Used to Elicit Grammatical Items
v
‘Do’s and ‘Dont’s in Grammar Testing
v Weaknesses/Limitations
v Strengths/Positive
Sides
A.
Grammar
and Grammatical Ability in The Concept of The Discrete-point Approach
Definition of Grammar : rules
by which words change their forms and are combined into sentences, or the study
or use of these rules.
Definition of Grammar ability : The ability that somebody has to construct
grammatically right sentence.
According to
discrete-point concept language can be broken down into its components, so that
testing grammar based on discrete-point approach means that the test will
measure the grammar ability from its components and the test will be presented
as a kind of atomistic testing form.
B.
The Scope
These are areas that
constitute the scope of grammar testing:
·
Articles
·
Tenses
·
Phrases
·
Noun-pronoun
agreement
·
Subject-main
verb agreement
·
Adverbial-adjectival
agreement
·
Modals
·
Countable-uncountable
·
Preposition
·
Etc.
C.
Different
Types of Tests that can be Used to Elicit Grammatical Items
1) Selected-response Task:
- Multiple-choice activities
- True/False activities
- Matching activities
- Discrimination activities
- Lexical list activities
- Grammatically judgement
activities
- Noticing activities
2) Limited-production Task:
- Gap-filling activities
- Cloze activities
- Short-answer activities
- Dictation activities
- Information-transfer activities
- Some Information-gap activities
- Dialogue (or discourse)
completion activities
3) Extended-production Task:
- Summaries, essays
- Dialogues, interviews
- Role-plays, simulations
- Stories, reports
- Some-information-gap activities
- Problem-solving activities
- Decision-making activities
D. Do’s and ‘Dont’s in Grammar Testing
|
Dos
|
Don’ts
|
|
1.
Decide the essential grammatical pattern being
tested
2.
Focused on a pattern, surface forms, rote practice
for a whole test
3.
Subject-verb parallelism, syntactically accurate
4.
Repetitive questions as a means of drilling
5.
The purpose of measuring the test takers memory to
internalize the grammatical pattern is prioritized
6.
Pattern recognition is advised and sufficient
7.
The test must provide more context than only a
single sentence
8.
The test taker should understand what the
communicative purpose of the task is
9.
The test maker should also know who the intended
audience is
10.
The test maker must have to focus on meaning and
not only form to answer correctly.
|
1.
Making
a sentence fragment and comma splices
2.
Put
double negative statements (especially in MC grammar test)
3.
Give
more than a sentence to construct context
4.
Having
communicative content
5.
Giving
clues of sentence interpretation
|
E.
Weaknesses/Limitations
1.
A
considerably amount of developmental information is lost with students who have
partial knowledge.
2.
This
way of testing and scoring will underestimating students’ true ability
3.
Students’
grammatical ability may be overlapped.
4.
It
tends to de-contextualize the grammatical context in real life
Since the scope is not
really wide, there will be less energy to be
used.
F.
Strengths/Positive
Sides
1.
Students’
grammar will be more focus and can easily master the grammar ability.
2.
The
tests can be held as a mean of internal understanding of grammar pattern.
3.
Grammar
accuracy is being prioritized
It is helpful to the test-takers to make a clear
distinction of each grammatical pattern.
Topic : Constructing Discrete-Point
Test: Vocabulary Tests
Group : 2 (Eki Juvita P.W. and Isnia
Devianti)
What are discussed under this
topic:
v
Definition of vocabulary and vocabulary ability
v
The scope of vocabulary testing
v
Test Techniques
v
Example of vocabulary test
v
Should do and shouldn’t do in vocabulary testing
v
Characteristics to measure communicative vocabulary
v
Strengths and weaknesses
A.
Definition of vocabulary and vocabulary ability
Definition of vocabulary : the
set of words within a language that are familiar to that person. A vocabulary
usually develops with age, and serves as a useful and fundamental tool for
communication and acquiring knowledge.
Definition of vocabulary ability :
•
Meaning
Meaning
•
Usage According
to Harmer (1993)
•
Word formation
•
Grammar
B.
The scope of vocabulary testing
1) Semantic relations:
word groups according to meaning, synonyms, hyponyms, and opposites
2) Situational relationship:
word sets associated with particular situations
3) Collocations: word
commonly found in association, noun + preposition links and phrasal verbs (verb
+ phrasal links)
4) Relationship of form
(often reffered as word building).
C.
Test Techniques
1) Recognition
-
Multiple choice items
-
Error-Recognition items
-
Pairing and matching items
-
Gap-filling items
-
Cloze/Modified Cloze
-
Guessing meaning from context
2) Production
-
Completion items
-
Transformation items
-
Paraphrase
-
Table completion
-
Combination and addition items
-
Items involving the changing of words\
Additional:
When we should go for
vocabulary recognizition:
•
More material needs to be covered.
•
You want to test different levels of learning.
•
You have little time for scoring.
•
You are not interested in evaluating how well a test
taker can formulate a correct answer.
•
You have a large number of test takers.
When we should we go
for vocabulary production:
•
You want to evaluate a person’s ability to formulate
a correct answer.
•
You have more time to score the items
•
You want to test a person ability to apply concepts
and information to a new situation.
You have a clear idea of the aspects and concepts
that should be tested.
D.
Example of vocabulary tests:
Multiple Choices
The flight attendant asked the passengers to … attention
to the safety demonstration.
a. give
b. devote
c. pay
d. lend
E.
Should do and shouldn’t do
|
Should
Do
|
Should
not Do
|
|
Test
vocabulary by using synonyms,
antonyms
or collocations of words
|
translation
(particularly at the word level)
|
|
having
the test taker select or supply the appropriate word to fill a blank
|
Use
definitions only when the part of speech of the target word makes it
appropriate
|
|
present
the words to be tested in as similar way as possible to the way they will be
encountered in the real world
|
the
test taker have to guess which meaning of a word the test writer had in mind
|
|
distractors
must be changed to fit the context
|
Presenting
words in isolation
|
F.
Characteristics to measure communicative vocabulary
·
The test must provide more context than only a
single sentence.
·
The items must assess lexis within a limited number
of semantic fields.
·
The test taker should understand what the
communicative purpose of the task is.
·
He or she should also know who the intended audience
is.
·
He or she must have to focus on meaning and not form
to answer correctly.
·
Recognize is not sufficient. The test taker must be able “to produce
grammatical responses.”
G.
Strengths and Weaknesses
|
Strength
|
Weakness
|
|
essential building block of
language and makes sense to be able to measure learners’ knowledge of it
|
Both teachers and learners
spend lots of time and energy on vocabulary acquisition
|
|
can be utilized as a means of
motivating students to study and showing them their progress in learning new
words
|
|
|
vocabulary tests which are
part of commercial tests, such as TOEFL, attempt to provide a measure of
learners’ vocabulary size, which is believed to give an indication of overall
language proficiency (Schmitt, 1994)
|
Topic : Pronunciation Test
Group : 3 (Halim Ahmad Faizin and Faruq
Aji Subroto)
What are discussed under this
topic:
v
Definition of pronunciation ability
v
The scope of pronunciation tests
v
Test Techniques
A.
Definition of pronunciation ability
Definition: the ability to use the correct stress, rhythm, and
intonation of a word in a spoken language.
B.
The scope of pronunciation tests
§
Vowels
§
Consonants
§
Clusters
§
Syllables
§
Word stress
§
Sentence stress
§
Weak forms
§
Contractions
§
Linking
§
Intonation
C.
Test techniques
1.
Recognition:
- Dictation
- Sound discriminations
- Sound comparison
- Sound definitions
- Same sound
- Odd man out
- Gap-filling
- Silent letters
- Intonation Pattern
2.
Production:
- Word stress
- Regrouping
- Sound recognition
- Matching
Dictation
A dictation exercise may
appear in different forms. First, a whole passage incorporating
target words to be tested. It may also consist of a set of individual words
incorporating the segmental or stress features being tested.
Sound Discrimination
The testees listen to one word or sentence and
circle the one they hear.
Sound Comparison
The testees listen to a pair of words or pairs of
sentences and indicate whether they are the same or different.
Sound Definition
A word is heard, and several different definitions,
including one that is correct for the word, are given. Testees are asked to
select the correct definition for the word they heard. It implies lexis and
grammar knowledge.
Same Sound
The testees listen to a list of words and mark the
ones that are the same.
Odd Man Out
The testees listen to a list of words and mark the
one that is different.
Gap-Filling
Testees listen to a sentence and select from a set
of words the one they hear.
Sound Recognition
Testees receive a set of cards with words, the
tester pronounces them and asks the students to show the corresponding card.
Testees may also pronounce some forms.
Regrouping
The testees are given a list of words and asked to
regroup the words that have the same sound (it can be specified whether the
common sound is a vowel or a consonant).
Matching
The testees are asked to find words that have the
same sound as a given word.
Silent Letters
Testees are asked to circle silent letters (silent
letters abound in English words, and the importance of such an exercise cannot
be overemphasized).
Word Stress
In writing, the testees may be asked to use one of
the conventional ways of marking stress.
Intonation Pattern
Testees listen and identify the speaker’s intention
according to his intonation pattern.
Topic : Listening Tests
Group : 4 (Halla Al Hartik and Ita
Permatasari)
What are discussed under this
topic:
v
The meaning of listening test
v
Kinds of listening tests
v
Understanding Discourse
A.
The meaning of listening test
Definition of listening : Getting meaning from spoken
language and all the sub skills that make up listening are all subservient to
this quest for meaning.
Definition of listening ability : an
ability to receive message of oral form into written form.
B.
Kinds of listening tests
There are three kinds of listening:
- understanding single words
- understanding single sentence
- understanding longer
pieces of discourse
1. Understanding
single words:
§ Phoneme discrimination Test
§ Test of stress and discrimination
2. Understanding
single sentence
§ Visual material comprehension
3. Understanding
longerr pieces of discourse
§ Statements and dialogues
§ Talks and lectures
Topic : Spoken Interview
Group : 5 (Laely Hidayati and Hani
Kaharani)
What are discussed under this
topic:
v
Mode
v
TOEFL stages
v
Pragmatics
A.
Mode
Under this
part, there are some modes:
1.
Question and Requests for Information
Yes/ No questions should generally be avoided,
except perhaps at the very beginning of the interview, while the candidate is
still warming up. Performance of various operations (of the kind listed in the
two sets of specifications above) can be elicited through requests of the kind
:
“Can you explain to me how/why … ?” and
“Can you tell me what you think of … ?”
“Can you explain to me how/why … ?” and
“Can you tell me what you think of … ?”
2.
Pictures
Single pictures are particularly useful for
eliciting descriptions. Series of pictures (or video sequences) form of natural
basis for narration.
3.
TOEFL way
The purpose of the TOEFL test is to
evaluate the English proficiency of people whose native language is not
English. The TOEFL scores are primarily used as a measure of the ability of
international students to use English in an academic environment.
4.
IELTS way
5.
Interpreting
It’s not intended that candidates should be able
to act as interpreters (unless that is specified). However simple interpreting
tasks can test both production and comprehension in a control led way.
Situation of the following kind can be set up :
The native language speaker wants to invite a foreign visitor to his or her home for a meal. The candidates has to convey the invitation and act as an interpreter for the subsequent exchange
Situation of the following kind can be set up :
The native language speaker wants to invite a foreign visitor to his or her home for a meal. The candidates has to convey the invitation and act as an interpreter for the subsequent exchange
6.
Prepared Monologue
This technique could be appropriate in a
proficiency test for teaching assistant , or in an achievement tests when the
ability to make presentations is an objective of the course. The limitation of
this mode is frequently misused. It should only be used where the ability to make
prepared presentations is something that the candidates will need
7.
Reading Aloud
It is a way to test pronunciation separately
from the content of speech. If it is necessary to use this method of testing,
the test should at least make use of a situation where the student might
actually be reading aloud, such as reading instructions or parts of a letter to
another person..
The limitation is that this is not generally a good way to test speaking. Its backwash effect is likely to be harmful, and it is not a skill that is used much outside of the classroom.
The limitation is that this is not generally a good way to test speaking. Its backwash effect is likely to be harmful, and it is not a skill that is used much outside of the classroom.
B.
TOEFL Stages
1.
Content Review
At this stage, assessment
specialists review stimuli and items for both language and content.
2.
Fairness Review
ETS Standards for
Quality and Fairness (2002) mandates fairness reviews. This fairness review must take
place before using materials in a test. Because attention to fairness is such
an integral part of the test design, all assessment specialists undergo
fairness training — in addition to item writing training — relatively soon
after their arrival at ETS. As part of their training to develop TOEFL test
materials, item writers must become familiar with the ETS Guidelines for
Fairness Review of Assessments (2009) and the ETS International
Principles for Fairness Review of Assessments (2007) and use them when
reviewing items and stimuli. The content review process itself, therefore,
always includes fairness as an aspect of development.
3.
Editorial Review
All TOEFL test materials receive an
editorial review. This review’s purpose is to ensure that language in the test
materials (e.g., usage, punctuation, spelling, style, and format) is as clear,
concise, and consistent as possible. Editors ensure that established ETS test
style is followed. In addition, when warranted, editors check facts in stimuli
for accuracy or to ensure that the stated facts are currently true; in areas
such as physics or geography, for example, changes in facts occur periodically
C.
Pragmatics
1.
The Bilingual Syntax Measure
2.
Illyn
Ilyin interview is the more typical and the more
pragmatically oriented than Upshur.
3.
Upshur
Upshur
and his collaborators set up a test to assess productive communication ability
as follows:
- Examinee
and examiner are presented with four pictures which differ in certain
crucial respects on one or more ‘conceptual dimensions’
- The
examinee is told which of the four he is to describe to the examiner.
- The
examinee tells the examiner which picture to mark and examiner makes a
guess.
The
number of hits, that is, correct guesses by the examiner is the score of the
examinee.
Group 6 :Constructing Integrative Test:
Cloze Procedure
(Tests of Grammar and Reading
Communicative)
(Eka Resti Wahyuni)
1. Cloze
Procedures Test
v Definition
Cloze Procedures Test is
a technique that can be used to assess students’ understanding of a text. It
consists of a text several words of which have been left out, multilated, or
deleted. It is believed to be as an effective means that can be used to assess
global language abilities.
v Strategies
of Deletion
There
are two words that can be utilized to delete words from the original text.
a.
Fixed
Deletion Strategies
In this
strategy, the rule is that every nth
word in the text is deleted constantly. For example, a word is deleted in every
7th word, the deletion will
be applicable to the 7th word encountered from the first sentence in
a reading text. The range within which words may be deleted is between 5 the
least and 12 the most.
b.
Rational
Deletion Strategies
The
rational strategies apply random word deletion. It also uses selective deletion
strategies. The rules used to delete words are based on the types of words to
be deleted.
There
are possibilities, such as function words only, function or grammatical words:
prepositions, articles, conjunctions and content words only, content words:
nouns, verbs, adverbs, adjectives.
v Some
Rules to Ponder in Word Deletion
a.
When
cloze procedures test applied to a paragraph, deleting words should not be made
on the first sentence also the last sentence. It is because normally the first
sentence of a paragraph serves a topic sentence where the main idea is located.
b.
When
cloze procedures test applied to an essay. As an essay consists of several
paragraphs, the first paragraph should be left undeleted as the first paragraph
functions as an introductory paragraph where the thesis statement, a sentence
having a similar role to that of a topic sentence, is located. The minimum
number of the words in the text should be 125 words which comes from this
computation: 5x25.
v Presentation
of Blank Spaces
There are several ways we can
do to present the blank spaces in cloze procedures:
a.
Full
blanks, probably with dots.
Numbers may be presented along with the blank
spaces. There are three ways: they may be placed at the earlier, the middle,
and the end parts of the blank spaces respectively and these numbers may be put
between brackets(.....), or they are left open. As the following suggest:
(1)...................,...............(2)...............,
and........................ (3).
b.
Using
underlines for the words deleted. The length should also be the same. The
number can also be placed in the middle part to vary the presentation of the
numbers for each item. The length should be the same.
c.
Using
cued blank strategy. Blanks are provided cues with some letters of the deleted
words. The letter cues may be one, two, or three letters depending on the
number of the letters in the words deleted. It can be placed in the earlier
part, the middle or the end parts or any combination of these. The length is
not applicable.
v Presentation
of Answer
There are three variations of
presenting the answers:
a.
Providing
no answer
b.
Provision
of answers in one list
c.
Provision
of answers with a multiple choice format.
v Input
Text and Deleted Parts
Input
texts are those texts that serve as the source of problems from which several
words will be deletes. Originally input texts in cloze tests can be a paragraph
or an essay. A strategic deletion is also applicable to paragraphs. Deletion of
a paragraph may include the introductory paragraph, body paragraphs, or the
conclusion paragraph.
Deletion Parts:
a.
Words deletion
b.
Sentence deletion
c.
Paragraph deletion
2. C-Test
Procedures
C-test is another technique that utilizes mutilation of a reading
text.
What makes Cloze procedures and C-Test different is that the principles of mutilation.
v Principles for multilation in C-test:
a.
The mutilation of words starts from the second word of a second
sentence of a text. The second half of every other word is deleted. The last
sentence is left intact, not mutilated.
b.
Presentation of the blank spaces is like the fixed-length blank
strategy for all words deleted or it may follow the number of letter deleted in
that for each letter deleted, it is replaced with the numbers of letters that
is multilated.
c.
Normally there are about 8 paragraphs to be mutilated of which
topics are different. The first sentence of each paragraph is also not
multilated as to provide the students with context.
3. Cloze-Elide
Procedures
v Definition
The
cloze elide is an objective language test task whereby superfluous, incorrect
words are inserted into a text and must be identified by the test taker within
a limited time. The cloze-elide procedures have the opposite strategies. Into a
reading text, several words are placed in the text in several spots, thus
creating ungrammatical in their neighboring contexts.
v The
Principle of making cloze-elide tasks:
Not
any words can serve as the words to be inserted.
The
requirements of the superfluous words that can be inserted into the text is
“damaging grammatically and not too conspicuous by their placement and to make
sure they were varied in their syntactic placement.
4.
Reading
Comprehension
v
Definiton
Reading
is essentially as a form of communication between writers and readers which
mediated through a written text. In reading event principally these aspects are
involved, namely the writer, text and reader.
v
Reading Abilities
A
person who has suffcient knowledge of semantic and syntactic system of the
language and background knowledge of the topic of reading text.
v
Focus
of Reading Tests:
a. Text comprehension
is important because comprehension is the reason for reading.
b. Text comprehension
is purposeful and active.
c. Text comprehension
can be developed by teaching comprehension strategies explicitly, through
cooperative learning, and by helping students use strategies in combination and
flexibly.
v
Test of Reading skills
Typical questions
used to teach each of the above skills:
a.
Topic
Topics
may be asked in conjunction with particular paragraph or the whole passage.
Questions requiring topics of a paragraph or the whole passage may have an
answer in the form of a word, which is a very rare case, or phrase. A topic in
phrase form thus implies ideas than one in the form of phrase. A topic in
phrase form thus implies ideas that are very specific. The longer is the more
specific the ideas contained in it.
b.
Main
Idea
While
topics deal with ideas or subjects of discussion in general, main idea refers
to the thought about which a topic of a passage is being expressed. A main idea
contains a topic: a topic is the subject being talked about in the main idea.
Often times main ideas contain the writer’s point of view, attitude, or
feelings toward the topic under discussion in the passage. In addition to the
difference, a topic may be in the form of a word or phrase, a main idea is
commonly expressed in the form o f a sentence.
c.
Specific
Information
Identification
of specific factual information relates to questions with “what”, “where”, and
“when”.
d.
Detail
Information
Identification
of detailed factual information responds to questions with “why”, and “how”.
e.
Reference
Questions
eliciting reference skills deal mostly with deictic pronouns with refer to a
word or a phrase located prior the pronouns (anaphora) or the word or the
phrase located after pronouns (cataphora).
f.
Inference
Unlike
factual questions, be they specific of detail, questions of which answer
require the making of inference or implication do not ask factual information
explicitly stated in the passage. Test takers are skillfully required to infer
based on the available stated information in the passage.
v
Type of test
Reading
comprehension test based is divided based on the level difficulties:
The
first level, literal comprehension, is the most obvious. Comprehension at this
level involves surface meanings. At this level, teachers can ask students to
find information and ideas that are explicitly stated in the text. In addition,
it is also appropriate to test vocabulary.
The second level or strand is interpretive
or referential comprehension. At this level, students go beyond what is said
and read for deeper meanings. They must be able to read critically and analyse
carefully what they have read. Students need to be able to see relationships
among ideas, for example how ideas go together and also see the implied
meanings of these ideas. At this level, teachers can ask more challenging
questions such as asking students to do the following:
* Re-arrange the ideas or topics discussed
in the text.
* Explain the author's purpose of writing
the text.
* Summarize the main idea when this is not
explicitly stated in the text.
* Select conclusions which can be deduced
from the text t!hey have read.
Then,
the third level of comprehension is critical reading whereby ideas and
information are evaluated. At this level, students can be tested on the
following skills:
* The ability to differentiate between
facts and opinions.
* The ability to recognize persuasive
statements .
* The ability to judge the accuracy of the
information given in the text
There are kinds of test
possible:
a. Short Story Reading
Comprehension
Readers are
tested on their ability to perform interpretations, make deductions, and infer
the meaning of vocabulary words based on a short story
b. Informational Passages
Reading Comprehension
In
these reading comprehension, students are asked questions about information
they have read about a specific topic. Each passage reads similar to a
newspaper of journal article, and provides interesting information about some
aspect of history, nature, mechanics, science, art, and more. Questions involve
critical thinking with a focus on logic and inference.
c. Technical Reading
Comprehension
In
these reading comprehension, students are asked questions about the meaning,
significance, intention, structure, inference, and vocabulary used in each
passage. Each passage reads like an encyclopedic or technical journal article.
Answers for worksheets in this section can be found at the end of each
individual worksheet.
d. Role Play Reading
Comprehension
In
these reading comprehension, students can increase their understanding of
colloquial and idiomatic expressions and get a feel for conversational English.
e. Dual Version Reading
Comprehension
In
each of these reading comprehension, the same story is told, but with two
versions: one that is basic, and one that is more advanced. This allows
students to make direct comparisons between the advanced version to the more
basic one, and makes for a powerful learning experience.
GROUP 7 : COMMUNICATIVE
TEST; SPEAKING, WRITING AND READING
SKILL
(Javas
Afrizal & Fuad Fajar)
v Communicative Testing
Communicative competence is the development of learners' communicative
competence is defined as "expression, interpretation, and negotiation of
meaning involving interaction between two or more persons or between one person
and a written or oral text".
Communicative testing is
a learning tool, providing evaluative information to both learner and teacher. The purpose of communicative testing is to Measures learners' ability to translate their
competence (or lack of it) into actual performance in 'ordinary' situations. CLT,
the tests have to be communicative as well. Incommunicative language tests (CL
Tests), a test has to measure the CC realized in the four language skills of
listening, reading, speaking, and writing each of which is led to other skill
to make the test more integrative in manner.
v Characteristic of Communicative Testing
Brown
(2005) suggests five core characteristics for designing a communicative
language test. These include meaningful communication, authentic situation,
unpredictable language input, creative language output, and integrated language
skills.
First,
the purpose of language learning is communication so language learners’
communicative ability should be measured. In other words, language tests should
be based on communication that is meaningful to students and meets their
personal needs. Authentic situations can help increase meaningful
communication. The usefulness of authentic situations in increasing meaningful
communication is emphasized by Weir (1990) when he states that, ‘language
cannot be meaningful if it is devoid of context’ (p.11). By using ‘unpredicted
language input’ and ‘creative language output’, Brown (2005) means that in real
situations it is not always possible to predict what speakers say
(unpredictable language input) so learners need to prepare for replying
(creative language output). The last characteristic is integrated language
skills. A communicative test should require test takers to show their ability
of combining language skills as in real life communication situation.
v Kinds of Communicative Testing
1.
Communicative Testing in
Listening skills
Communicative listening
tests design requires (1)authentic texts e.g. conversations, interviews,
broadcasts, telecasts, extended talk, and entertainment; (2) tasks e.g.
transcoding, and scanning; (3) channel through which messages are conveyed from
the sender to the receiver and (4) response mode which is usually oral but in
some instances, could also be written or nonverbal. For example:
v Information Gap
v Dictation
2.
Communicative
Testing in Speaking skills
The activities that involve speakers in
using language for the purpose of achieving a particular goal or objective in a
particular speaking situation. For example:
v Role Play
v Interview
v Problem Solving
3. Communicative testing in writing skills
Some tests combine reading and writing in
communicative situations. Testees can be given a task in which they are
presented with instructions to write a letter, memo, summary, etc., answering
certain questions, based on information that they are given.
For
example;
v Business Letter
v Personal Letter
GROUP 8
: AUTHENTIC ASSESSMENT: PORTFOLIO
(Istianah, Indri Wijaya)
v Definition
Portfolio assessment is
the procedure used to plan, collect, and analyze the multiple sources of data
maintained in the portfolio.
v Characteristics of portfolio assessment
1.
Comprehensiveness
2.
Predetermined
and Systematic
3.
Informative
4.
Tailored
5.
Authentic
A.
Generally
Portfolio fall into three categories
1.
Working
2.
Showcase
portfolio
3.
Record-keeping
Portfolio
B.
Guideline
in establishing portfolio
v State objective clearly
v Give guidelines on what materials to include
v Communicate assessment criteria to students
v Designate time within the curriculum for
portfolio development
v Establish periodic schedules for review and
conferencing
v Designate an accessible place to keep portfolios
v Provide positive washback-giving final
assessment
C.
Advantages
of portfolio
v Foster intrinsic motivation, responsibility, and
ownership
v Promote student-teacher interaction with the
teacher as the facilitator.
v Individualize learning and celebrate the
uniqueness of each student.
v Provide tangible evidence of students’ works
v Facilitate critical thinking, self-assessment,
and revision processes.
v Offer opportunities for collaborative work with
peers, and
v Permit assessment of multiple dimension of
language learning.
D.
Disadvantages
of portfolio
v Much time and money needed to score portfolio
v Wonder whether the teacher will be able to
assess all their students fairly, accurately, and comprehensively.
E.
The
following are examples of what might become a part of a portfolio
v Written work from different academic areas
v A video tape showing the quality of interaction
in committee work
v A cassette to indicate the quality of oral work
done, such as in reading and speaking endeavors
v Snapshots of construction experiences as they
relate to ongoing lessons and units of study
v Drawings, diagrams, charts, and graphs develop
by the learner.
GROUP 9: DESIGNING AUTHENTIC ASSESSMENT: PROJECT (INCORPORATING
READING WRITING AND SPOKEN LANGUAGE)
(Evi Rani,
Hari Prasetyo)
A.
Project in Authentic
Assessment Movement
The
project in authentic assessment is student works with other students as a team to create a
project that often involves multimedia production, oral and written
presentations, and a display.Student projects usually call
for students to create something to demonstrate learning. Some projects may reflect individual work as
well as group work.
B.
Characteristics of Project
as a Form of Authentic Assessment
v
Require
students to perform, create, produce, or do something.
v
Often
presented orally or written report.
v
Use
real-world context and integrative.
v
Allow
students to be assessed on what they normally do in class everyday.
v
Focus
on process as well as product.
v Tap into thinking and problem solving.
C.
Procedures to Establish
Project as a Form of Authentic Assessment
These
steps are adopted from procedures that are suggested by Baker (1993) and
Herman, Aschbacher, and Winters (1992). The procedures are follow:
v
Built a
team.
v
Determined
the purposes of authentic assessment in form of project.
v
Specify
the objectives.
v
Prepare
students for project in stepwise progressions.
v
Use
reliable evaluation form, checklist, rating form, and questionnaire.
v
Treat
projects as opportunities for giving feedback and provide feedback
systematically.
On
the other hand, J. Michael O'Malley have listed characteristics of student’s
project that should be considered in authentic assessment.
v Constructed
Response
v Higher-Order
Thinking
v Authenticity
v Integrative
v Process
and Product
v Depth
in Place of Breadth
D.
Constructing a Project Assessment Scheme to Assess Students’
Language Skills
A project is meaningful if it
fulfills two criteria:
v
First, students must perceive the work as
personally meaningful, as a task that matters and that they want to do well.
v
Second, a meaningful
project fulfills an educational purpose. Well-designed and well-implemented
project-based learning is meaningful in both ways.
E.
Conduct student Strengths
and Weaknesses
v
Strengths :
·
Connection to real life skills.
·
Require students to solve complex
problems or produce multi-step projects, often in collaboration with others.
·
Attempt to combine teaching,
learning, and assessment to promote student motivation, engagement, and
higher-ordered learning skills.
·
Teacher and students share an
understanding of the criteria for performance; in some cases, students even
contribute to defining the expectations for the task
v
Weaknesses :
- Subjectivity
in scoring, the costliness of administering and scoring, and the narrow
range of skills that are typically assessed.
- Inter-rater
agreement is increased with clearly defined criteria, including exemplars
and non-exemplars and initial and on-going training of the evaluators.
- The
emphasis on assessing knowledge in-depth or in application, often limits
the amount of content knowledge that is assessed.
GROUP 10 : DESIGNING AUTHENTIC ASSESSMENT: EXTENDED RESPONSE
(INCORPORATING READING, WRITING AND SPOKEN LANGUAGE)
(Fitri Handayani, Irma Rosadi, Ulfa Jihadil Fitri)
v Definiton
Web definition : An answer to an essay item
which asks or implies a question which has no definite limits to restrict the
student response.
Extended-response questions are writing
prompts or questions that give students the opportunity to prepare a written
answer, often a short phrase, a list, or a more substantial composition such as
a multipage essay. Extended-response questions are usually open-ended so that
students can demonstrate the extent of their knowledge and skills in the area
under assessment.
v Characteristic of extended response test
1.
assess
a student's ability to select, construct, and analyze a graph.
- assess
a student's ability to analyze a complex problem and to generate a model
or other solution to a particular problem or issue.
- assess
a student's ability to communicate information collected from a reading,
from an investigation, or from other sources.
v Develop questions from the upper levels of
Bloom’s Taxonomy
1. Remember
2. Understand
3. Apply
4. Analyze
5. Evaluate
6. Create
v Writing item guidelines
The item should
1.
clearly
tell students what they are to do and what is expected of them.
- clearly
tell students where they are to write their response.
- use
simple but authentic vocabulary and good sentence structure. It should be
clear and concise.
- identify
the information or material that students should use when preparing their
response.
- clearly
indicate the scientific skill or process that should be demonstrated in
the response (see CLG 1 and higher order thinking skills).
- provide
proper cueing to direct the student's thinking and identify expectations.
v Instructional Strategies for Extended Response
a.
Directed
Reading - Thinking Activity (DR-TA)
b.
Say
Something
c.
Think aloud
d.
Writing
about reading
v Assessment
1.
Responses
will be assessed on content only
- Developmental
expository writing does not happen overnight, but rather with practice
over time.
- Diagnostic
assessment will take place as the student begins to write. Has comprehension taken place? Is the
student on task, engaged, thinking, and responding in writing?
- Formative
assessment takes place as the student writes, not just on one occasion,
but over time
- Summative
assessment will take into consideration progress over time when
conferencing with the student and
comparing expository writing samples with those completed earlier
in the year.
v Several issues of Extended Response
1.
Reading Extended Response is NOT a writing test
2. Students should NOT write a summary of
the passage
3. Students are NOT scored on conventions
(punctuation, spelling, etc.)
GROUP 11: DESIGNING NON-TEST IN
LANGUAGE LEARNING:
QUESIONNAIRES, CHECKLIST AND
RATING SCALE
(Fitri
Nurul H, Evi Purwanti)
v KINDS OF NON-TEST
A.
QUESTIONNAIRE
A
questionnaire is simply a ‘tool’ for collecting and recording information about
a particular issue of Questionnaires should always have a definite purpose that
is related to the objectives of the research, and it needs to be clear.
Questionnaires are commonly used to:
v To collect factual information in ordes to
classify people and their circumstances
v To gather straight forward information relating
to people’s behavior
v To look at the basic attitudes/opinions of a
group of people relating to a particular issue
v To measure the satisfaction of customer with a
product or service
Even questionnaires are used to collect some information to
the certain situation, but questionnaires should not use to:
v To explore complex issues in great depth
v To explore new, difficult or potentially controversial
issues (NB: longer, relatively
unstructured depth interviews would be more appropriate here)
v As an ‘easy’ option which will require little
time or effort (a common error)
In order to gather useful and relevant information it is
essential that careful consideration is given to the design of your
questionnaire. A well-designed
questionnaire requires thought and effort, and needs to be planned and
developed in a number of stages:
1.
Initial
consideration
2.
Question
Content, phrasing and response format
3.
Question
sequence and layout
4.
Equalities
5.
Confidentiality
6.
Piloting the questionnaire
7.
Final Questionnaire
B.
CHECKLIST
A checklist is a tool for identifying the presence or absence of
conceptual knowledge, skills, or behaviors'. Checklists are appropriate when
the teacher is looking for the presence of specific elements in the product or
performance, and all elements are generally weighted the same.
1. Characteristics of checklist
v have criteria for success based on expected
outcomes
v be short enough to be practical (e.g., one sheet
of paper)
v have tasks chunked into logical sections or flow
from start to finish
v highlight critical tasks
v have sign-off points that prevent students from
proceeding without approval, if needed
v be written with clear, detailed wording to
minimize the risk of misinterpretation
v have space for other information such as the
student’s name, date, course, examiner, and overall result
v be reviewed by other instructors
2. Designing Checklist
In
developing checklist, use the following steps :
1. Review
the learning outcome and associated criteria for success.
2.
State the level of success required for the checklist to be considered
completed. In most cases, all items
must be checked.
3.
Decide on the response such as “Yes” or “No”, or simply have a box to
be checked once the item has been
completed.
4.
From a procedure, process, or task description list, pick those items
that
are required for a good performance or
product.
5.
Group similar items or order them sequentially—keep as short as
possible.
6.
Highlight critical steps, checkpoints, or indicators of success.
7.
Write clear instructions for the observer.
8.
Review the task descriptions for details and clarity.
9.
Format the checklist.
10. Ask for feedback from other instructors
before using it with students.
3. Advantages Of Using Checklists
v Easy to use and update
v Require little training
v Available whenever evaluation is needed
v Flexible and can be used with a variety of
assessment strategies
v Behaviors can be recorded frequently
4. Disadvantages of Using Checklists
v Can be time consuming
v Teachers find it difficult to adapt teaching and
evaluation behaviors to include checklists
v If there are too many checklists, the teacher
can be overwhelmed with assessment and record keeping
v Teachers may not consider assessments with
checklists as valid measures
v Checklists do not indicate how well a student
performs
C.
RATING
SCALE
A rating scale is a
tool used for assessing the performance of tasks, skill levels, procedures,
processes, qualities, quantities, or end products, such as reports, drawings,
and computer programs.
1.
Characteristics of rating scales
Rating scales should:
v have criteria for success based on
expected outcomes
v have clearly defined, detailed
statements
v have statements that are chunked
into logical sections or flow sequentially
v include clear wording with numbers
when a number scale is used
v have specific, clearly
distinguishable terms
v be short enough to be practical
v highlight critical tasks or skills
v indicate levels of success required
before proceeding further, if applicable
v Sometimes have a column or space for
providing additional feedback
v have space for other information
such as the student’s name, date, course, examiner, and overall result
v be reviewed by other instructors
2. Types of Rating Scale
v Graphic rating scales
v Numeric Rating Scales
v Descriptive Rating Scales
3. Designing rating scales :
In
developing rating scales, use the following steps :
1. Review the learning outcome and associated
criteria for success.
2. Determine the scale to use (words or words
with numbers) to represent
the levels of success.
3. Write a description for the meaning of each
point on the scale,
as needed.
4. List the categories of performance to be
assessed, as needed
5. Clearly describe each skill.
6. Arrange the skills in a logical order, if you
can.
7. Highlight the critical steps, checkpoints, or
indicators of success.
8. Write clear instructions for the observer.
9. Review the rating scale for details and
clarity.
10. Format the scale.
11. Ask for feedback from other instructors
before using it with students.
4. Advantages of Using Rating Scales
v Quick and easy to complete
v User can apply knowledge about the student from
other times
v Minimum of training required
v Easy to design using consistent descriptors
(e.g., always, sometimes, rarely, or never)
v Can describe the student’s steps toward
understanding or mastery
5. Disadvantages of Using Rating Scales:
Reliability
v Highly subjective (rater error and bias are a
common problem)
v Raters may rate a student on the basis of their
previous interactions or on an emotional, rather than an objective, basis
v Ambiguous terms make them unreliable: raters are
likely to mark characteristics by using different interpretations of the
ratings (e.g., do they all agree on what “sometimes” means?)
0 comments:
Post a Comment