A.
KINDS OF NON-TEST
Non-tests are made to discover how
well the students understand the information taught and to determine how much
growth they are making in developing their academic skill. By assessing through
alternative means, teachers can ensure that tests anxiety or an ability to
perform on tests do not lead to misunderstanding of a student ability. There
are many kind of non-test but in this part, we will discuss questionnaire,
checklist, and rating scale.
A.
Questionnaire
A questionnaire is simply a ‘tool’
for collecting and recording information about a particular issue of
interest. It is mainly made up of a list
of questions, but should also include clear instructions and space for answers
or administrative details. Questionnaires should always have a definite purpose
that is related to the objectives of the research, and it needs to be clear.
Questionnaires are commonly used to:
v To collect factual information in ordes to classify
people and their circumstances
v To gather straight forward information relating to people’s behaviour
v To look at the basic attitudes/opinions of a group of
people relating to a particular issue
v To measure the satisfaction of customer with a product or
service
Even questionnaires are used to collect some
information to the certain situation, but questionnaires should not use to:
v To explore complex issues in great depth
v To explore new, difficult or potentially
controversial issues (NB: longer, relatively
unstructured depth interviews would be more appropriate here)
v As an ‘easy’ option which will require
little time or effort (a common error)
In
order to gather useful and relevant information it is essential that careful
consideration is given to the design of your questionnaire. A well-designed questionnaire requires
thought and effort, and needs to be planned and developed in a number of stages:
1.
Initial consideration
·
Firstly,
it is important to be clear about the type and nature of information you need
to collect and exactly who is your target population.
·
Second, we also need to decide on the most
appropriate method for administering the questionnaire and consider
your approach to sampling.
·
Finally,
it is useful to consider how the findings will be analyzed as this may have an
impact on the design of the questionnaire.
2.
Question Content, phrasing and response format
It is the most
important, aspect of questionnaire design relates to the questions
themselves. You need to make sure that
each question:
• Adds
value. If it is just ‘nice to know’ and
does not add value, leave it out.
• Is clear and
easy to understand.
• Asks what you
think it is asking and does not cause confusion.
Writing
questions is a creative process and there is no standard format for a ‘good’
question.
3.
Question sequence and layout
Questions should
be numbered and ordered in a way that is logical to the respondent, with similarly themed questions
grouped together. A technique known as ‘funneling’
begins with general questions before focusing down to more specific
questions. Simple questions are often
placed at the beginning to put respondents at ease. Some questions may require ‘routing’, (e.g.
if ‘no’, go to Q4), but be careful not to make this too complex. It is also important to include clear
instructions for the respondent or interviewer.
4.
Equalities
When
designing your questionnaire you also need to remember to account for
equalities issues. For example:
·
Questions
on gender, age, ethnic origin and disability are sometimes used to monitor
whether a representative cross-section of the population was reached (but it is
bad practice to ask these questions and simply store the data rather than using
it for a specific purpose).
·
You
may need to include a paragraph translated into minority ethnic languages for
people who do not speak/read English as their first language. This should
explain the focus of the survey and allow them to request a translated copy of
the questionnaire.
·
You
may also need to include a statement in large print indicating that respondents
can request a copy of the questionnaire transcribed into Braille or Large Text.
5.
Confidentiality
Respondents
need to be reassured that the information they provide on the questionnaire is
confidential. This means that their identities or personal details must not be
disclosed to others, except for research purposes, and any data used in the
report will not be linked to any respondents. If the questionnaire is not
exclusively for research (for example, if it is used to update a database that
is used for purposes other than research), then this should be made clear to
respondents and the questionnaire cannot be described as confidential survey
research.
6.
Piloting the questionnaire
It is good practice to ‘pilot’ or pre-test your
questionnaire with a small sample of respondents before use. The pilot should
check people’s understanding and ability to answer the questions, highlight
areas of confusion and look for any routing errors, as well as providing an
estimate of the average time each questionnaire will take to complete. Any
amendments highlighted by the pilot should be made to the questionnaire before
issuing a final version.
7.
Final Questionnaire
In this step people can start use the questionnaires after
following the previous step.
8.
The example of Questionnaire
The
example of questionnaires that can be used as self assessment tool :
9.
The
advantages and disadvantages of using questionnaire:
B.
C.
D.
E.
F.
B.
CHECKLIST
1. What is checklist?
A checklist is a
tool for identifying the presence or absence of conceptual knowledge, skills,
or behaviors'. Checklists are appropriate when the teacher is looking for the
presence of specific elements in the product or performance, and all elements
are generally weighted the same.
2.
Characteristics
of checklist
Checklists
should:
ü have criteria for success based on
expected outcomes
ü be short enough to be practical (e.g.,
one sheet of paper)
ü have tasks chunked into logical sections
or flow from start to finish
ü highlight critical tasks
ü have sign-off points that prevent
students from proceeding without approval, if needed
ü be written with clear, detailed wording
to minimize the risk of misinterpretation
ü have space for other information such as
the student’s name, date, course, examiner,
and overall result
ü be reviewed by other instructors
3.
DESIGNING CHECKLIST
In developing checklist, use the following steps :
1. Review the learning outcome and associated
criteria for success.
2. State the level of success required for the
checklist to be considered
completed.
In most cases, all items must be checked.
3.
Decide on the response such as “Yes” or “No”, or simply have a box to
be checked
once the item has been completed.
4. From a procedure, process, or task
description list, pick those items that
are
required for a good performance or product.
5. Group similar items or order them
sequentially—keep as short as
possible.
6. Highlight critical steps, checkpoints, or
indicators of success.
7. Write clear instructions for the observer.
8. Review the task descriptions for details and
clarity.
9. Format the checklist.
10. Ask for
feedback from other instructors before using it with students.
4.
THE EXAMPLE OF CHECKLIST
5.
ADVANTAGES OF USING CHECKLISTS
Easy
to use and update
Require
little training
Available
whenever evaluation is needed
Flexible
and can be used with a variety of assessment strategies
Behaviors
can be recorded frequently
6.
Disadvantages
of Using Checklists
Can
be time consuming
Teachers
find it difficult to adapt teaching and evaluation behaviors to include
checklists
If
there are too many checklists, the teacher can be overwhelmed with assessment
and record keeping
Teachers
may not consider assessments with checklists as valid measures
Checklists
do not indicate how well a student
performs
C.
Rating Scale
1. What is a rating scale?
A rating scale is a tool used
for assessing the performance of tasks, skill levels, procedures, processes,
qualities, quantities, or end products, such as reports, drawings, and computer
programs. These are judged at a defined level within a stated range. Rating
scales are similar to checklists except that they indicate the degree of
accomplishment rather than just yes or no. Rating scales list
performance statements in one column and the range of accomplishment in
descriptive words, with or without numbers, in other columns. These other
columns form “the scale” and can indicate a range of achievement, such as from poor
to excellent, never to always, beginning to exemplary,
or strongly disagree to strongly agree. Some tasks, such as
procedures and processes, need to be observed in order to be assessed.
2. Characteristics
of rating scales
Rating scales should:
•
have criteria for success based on expected outcomes
•
have clearly defined, detailed statements
This gives more reliable
results. For assessing end products, it can sometimes help to have a set of
photographs or real samples that show the different levels of achievement.
Students can visually compare their work to the standards provided.
•
have statements that are chunked into logical sections or flow sequentially
•
include clear wording with numbers when a number scale is used
As an example, when the
performance statement describes a behavior or quality, 1 = poor through
to 5 = excellent is better than 1 = lowest through to 5 =
highest or simply 1 through 5. The range of numbers should be the
same for all rows within a section (such as all being from 1 to 5).
The range of numbers should
always increase or always decrease. For example, if the last number is the
highest achievement in one section, the last number should be the highest
achievement in the other sections.
•
have specific, clearly distinguishable terms
Using good then excellent
is better than good then very good because it is hard to
distinguish between good and very good. Some terms, such as often
or sometimes, are less clear than numbers, such as 80% of the
time.
•
be short enough to be practical
•
highlight critical tasks or skills
•
indicate levels of success required before proceeding further, if applicable
•
Sometimes have a column or space for providing additional feedback
•
have space for other information such as the student’s name, date, course,
examiner, and overall result
• be reviewed by other
instructors
3. Types
of Rating Scale
a.
Graphic
rating scales let you mark a point on a line or continuum to indicate your
rating, as shown in the example below.
b.
Numeric
Rating Scales are used sometimes in an effort to quantify results. Note that
here, as with the 1-10 scale, we have assigned the highest value to what we
assume would be the desired response.
c.
Descriptive
Rating Scales are those in which each rating level is defined, often in detail
and is not necessarily assigned a point value. Having good descriptions for
rating levels alleviates some of the problems identified for graphic scales and
does not force a teacher to quantify performance, if that is not appropriate.
4. The
Examples of Rating Scale
5.
Designing
rating scales :
In developing rating scales, use
the following steps :
1. Review the
learning outcome and associated criteria for success.
2. Determine the
scale to use (words or words with numbers) to represent
the levels of
success.
3. Write a
description for the meaning of each point on the scale,
as needed.
4. List the
categories of performance to be assessed, as needed
5. Clearly
describe each skill.
6. Arrange the
skills in a logical order, if you can.
7. Highlight the
critical steps, checkpoints, or indicators of success.
8. Write clear
instructions for the observer.
9. Review the
rating scale for details and clarity.
10. Format the
scale.
11. Ask for
feedback from other instructors before using it with students.
6. Advantages of Using Rating Scales
Quick
and easy to complete
User
can apply knowledge about the student
from other times
Minimum
of training required
Easy
to design using consistent descriptors (e.g., always, sometimes, rarely, or
never)
Can
describe the student’s
steps toward understanding or mastery
7. Disadvantages of Using Rating Scales:
Reliability
Highly
subjective (rater error and bias are a common problem)
Raters
may rate a student on
the basis of their previous interactions or on an emotional, rather than an
objective, basis
Ambiguous
terms make them unreliable: raters are likely to mark characteristics by using
different interpretations of the ratings (e.g., do they all agree on what
“sometimes” means?)
D.
Tips
for Developing Questionnaire, Checklists and Rating Scales
— Use questionnaire,
checklists, and
rating scales relation to outcomes and standards.
— Use simple formats that can be
understood by students and that will communicate information about student
learning to parents.
— Ensure that the characteristics and
descriptors listed are clear, specific and observable.
— Encourage students to assist with
constructing appropriate criteria. For example, what are the descriptors that
demonstrate levels of performance in problem solving?
— Ensure that questionnaire, checklists, and rating scales are dated
to track progress over time.
— Leave space to record anecdotal notes or
comments.
— Use generic templates that become
familiar to students and to which various descriptors can be added quickly,
depending on the outcome(s) being assessed.
— Provide guidance to students to use and
create their own checklists, rating scales and rubrics for self-assessment
purposes and as guidelines for goal setting.
E.
Semantic Differential Scale
The Semantic Differential (SD)
measures people's reactions to stimulus words and concepts in terms of ratings
on bipolar scales defined with contrasting adjectives at each end. An example
of an SD scale is:
Usually, the position marked 0
is labeled "neutral," the 1 positions are labeled
"slightly," the 2 positions "quite," and the 3 positions
"extremely." A scale like this one measures directionality of a
reaction (e.g., good versus bad) and also intensity (slight through extreme).
Typically, a person is presented with some concept of interest, e.g., Red
China, and asked to rate it on a number of such scales. Ratings are combined in
various ways to describe and analyze the person's feelings.
A number of basic considerations
are involved in SD methodology:
1) Bipolar adjective scales
are a simple, economical means for obtaining data on people's reactions. With
adaptations, such scales can be used with adults or children, persons from all
walks of life, and persons from any culture.
(2) Ratings on bipolar
adjective scales tend to be correlated, and three basic dimensions of response
account for most of the co-variation in ratings. The three dimensions, which
have been labeled Evaluation, Potency, and Activity (EPA), have been verified
and replicated in an impressive variety of studies.
(3) Some adjective scales
are almost pure measures of the EPA dimensions; for example, good-bad for
Evaluation, powerful-powerless for Potency, and fast-slow for Activity. Using a
few pure scales of this sort, one can obtain, with considerable economy,
reliable measures of a person's overall response to something. Typically, a
concept is rated on several pure scales associated with a single dimension, and
the results are averaged to provide a single factor score for each dimension.
Measurements of a concept on the EPA dimensions are referred to as the
concept's profile.
(4) EPA measurements are
appropriate when one is interested in affective responses. The EPA system is
notable for being a multi-variate approach to affect measurement. It is also a generalized approach, applicable to any concept or stimulus, and thus it permits comparisons of affective reactions on widely disparate things. EPA ratings have been obtained for hundreds of word concepts, for stories and poems, for social roles and stereotypes, for colors, sounds, shapes, and for individual persons.
(5) The SD has been used as
a measure of attitude in a wide variety of projects. Osgood, et al.,
(1957) report exploratory studies in which the SD was used to assess attitude
change as a result of mass media programs and as a result of messages
structured in different ways .
v The example of semantic
differential scale :
F.
Conclusion
Questionnaires,
checklist, and rating scales are tools that state specific criteria and allow
teachers and students to gather information and to make judgement about what
students know and can do on relation to the outcomes.
0 comments:
Post a Comment