August 27, 2003
SAT = Stupid Annoying Tests
Posted by Jess in
Day to Day
Apparently SAT scores are up. Higher than they’ve been in 20 years, actually. I don’t really know what that means.
I’ve always been quiet in the whole concept of how accurate certificate testing programs are, almost afraid to voice my opinion. There’s a reason.
I don’t think standardized tests test anything except how good one is at taking tests.
It’s an art form. It really is, and one slighted towards those left-brained individuals who make up the majority of the world. I’m not one of those people.
I would like to think that I’m Not Entirely Stupid; I made honor roll every semester in High School, and was even on the National Honor Society. I took the SAT’s twice, and even did the Pre-SAT’s.
I never scored over 900 total. I don’t even want to tell you how many scholarships and financial aid it cost me.
For some people, multiple choice is just another test. You pick the best answer, and go on your merry way. For others, it’s a nightmare. We agonize over wording and tenses. We agonize over the clock that’s counting down in the corner of our peripheral vision. We agonize because we can't visualize what we are suposed to be answering correctly. What happens when two choices are right? Who judges which one is “The Best”? And why don’t I get any credit for choosing the one that was at least second closest to being The Best?
What’s the worst that could happen by allowing me to write in my own best possible answer? Someone would have to go over my exam by hand. That means they would have to employ more test-graders, which means they would be creating more jobs. That’s good, right? I’d pay more to take each test if meant I’d be able to pass it the first time… I’d still end up paying less!
Permalink
|
TrackBack (0)
Comments
As a former college instructor, and as the designer of hundreds of excruciating multiple-guess exam bank questions, I have to take some satisfaction in the knowledge that what I did was cruel and painful to at least a few of you. Not that I would have purposely directed any of my evil to you personally, Jess, but it gives me a chance to laugh my evil laugh...
The biggest single reason for multiple guess is not that it's quicker to mark, but that "right" and "wrong" are not arguable (assuming the questions are well-formed). You'd be amazed at the time one spends listening to arguments trying to prove that an essay answer stating the polar opposite of the truth is at least partly correct, and is therefore deserving of some points. At least a B-minus. C'mon. The filling in (or not) of a small rectangle with a graphite and clay mixture, on the other hand, is a somewhat more concrete concept (except as it applies to paper ballots used during elections held in the State of Florida, due to the influence of the Bermuda Triangle).
There are sure-fire tricks for multiple guess tests, though. The first on the list of "Things that actually work" is, "When in doubt, pick 'C'." Really. The distribution of correct responses is not random, nor is it even -- psychologically-based answer distributions will be biased slightly towards "C" being the correct response in either one-of-four or one-of-five selections. The bias is slight, but it is there.
The second is a little harsher. Get blind drunk during your cram session. The kind of drunk that stops you from feeling your face altogether. If you can lie down on the front lawn and don't have to hold on to keep from falling off, you aren't there yet. DON'T LEAVE THIS TOO LATE. You don't want to be unconscious during the exam period, and you definitely don't want to be drunk while writing the test. What you want is the sort of hangover that makes incorrect answers hurt down to your toenails. While you may be sorry for the rest of the day, you'll find that the correct answers don't hurt nearly as much to read as the incorrect ones. Even "slightly less correct" answers will be excruciatingly painful. I have seen this work too many times to ignore it -- people who had no business being in post-elementary courses can get an easy B this way. Of course, they're idiots again once the pain is gone, but by then they've got the frameable placemat, and there's nothing anyone can do about it....
OK, Jess...
A. Get over it!
B. Get over it!
C. Get over it!
D. All Of The Above!
LOL!
Stan - You have one of the most extensive and fascinating histories of anyone I've ever met (not to mention you are one of the funniest people!). And I mean that with the utmost respect and incredible admiration. Get out of here, I didn't know you were a college instructor! May I ask what subject?
I understand completely how right and wrong are not arguable... my point actually was how different people process coming to the conclusion of differentiating what is completely right and what is right, but in a different way.
Though the 'when in doubt, pick C' concept definately made it's way to me, myself and everyone I know has long followed that idea.
(that, and the much less well-known "D is for Diploma". Just kidding.)
I wish I knew your second method much sooner, I always saved that for AFTER the exam. :-)
Tom -
Two trains are coming towards each other at a speed of 45 miles per hour. Jess can guzzle 3 cups of Maxwell House Columbian Supreme coffee in 1.3 hours. How long will it her to finish paying back her student loans? :-)
LOL!!!
Oh, and Stan, I could have written you a KILLER essay. :-)
Two points, one experiential, one logical.
Experiential: Of those whom I know who took the SATs when I did (early 90s), their scores on various tests and pretests were generally pretty consistent relative to each other. Also, those whom we all expected to do well did indeed do well (including students who did poorly in school, but for reasons other than aptitude). In other words, the tests were measuring something — a something that we all knew to correlate with academic success.
Logical: If SAT scores measure anything that correlates with collegiate success (which is something for the universities to decide, although their decision seems, recently, to have been to ignore evidence that the correlation is there), then it doesn't really matter what that something/anything is; the tests remain useful for discerning whom to admit to an academic program.
My own feeling is that the intellectual arguments often made against standardized tests actually speak for them. What would it mean that somebody is "just good at testing"? It could mean that the person can remember large amounts of information, which would seem to be an obvious plus in education. It could also mean that the person is able to discern correct answers based on the limited information offered by the individual questions and or the tests as a whole (including the likely construction of the test). The fact that it is an "art," as you say, hardly suggests that it is not related to the "art" of learning.
The increasing scores in conjunction with deteriorating educations suggests to me that the tests are being watered down in response to the real objection to the SAT: that it might make those who don't test well feel bad. But if a person can't admit his or her own strengths and weaknesses, then I'd say that person has bigger problems than getting into a higher collegiate tier.
I can definitely admit my own weaknesses, math being one of them. :-P
So isn’t that the reason that one goes to college? To improve their weakness and turn it into a strength? If I’m not allowed in because of that weakness, then doesn’t that defeat the point?
I don’t think doing well or not on tests is a question of making me feel bad, it’s a question of making me feel frustrated. I don’t feel bad because I DO know the material. This is my entire point. PROVING I know the material through tests is what’s in question. Coming back to your suggested idea of how someone can become a good test-taker, I am unable to look at the information in front of me and discern the correct answer. I find the limited choices very hard to discern which one is the right one. For example, I found this sentence of yours very confusing (please don’t take that the wrong way, I’m actually happy because it’s a great example for me to show you what I mean!)
“The fact that it is an "art," as you say, hardly suggests that it is not related to the "art" of learning.”
I actually had to re-write that sentence again in a positive sense and THEN reverse it to understand what you said. I had to re-write it as “the fact that it is an ‘art’ as you say suggests that it is related to the art of learning”, and then reverse it after I understood what you are trying to say.
Now picture having to do that with every question while the clock is ticking.
I guess it’s just a matter of different styles of learning. I definitely agree with you that someone who is good at testing probably has an innate ability to remember large amounts of information. I’m wondering if it’s HOW the large amounts of information that gets into their head that makes the difference.
I believe that people learn differently. Some people can read a textbook, memorize what they read, and be tested on it successfully. I’m visual. I need to go to the museum and view the artifact, and THEN I’ll understand what they are talking about.
My main point of this topic, actually, was that it seems that the fact that people learn differently is becoming the focus of many studies lately. I was just wondering why they can’t focus a study on how people TEST differently. :-)
Thanks to everyone that has responded so far by the way, I'm really enjoying this discussion!
Jess, I was an electronics technology instructor who also taught mathematics for people on different diploma/certificate paths (community college-style diplomas, rather than degrees -- in Canada the terminology is different, and there is a definite separation between the two academic streams). It started when I was unceremoniously yanked from my comfortable hangar and thrust into the classrooom during my military career (I want to say "Air Force", but we have an integrated force in Canada -- I wore a light blue uniform and had a bird on my hat, but the airplanes I worked on were Navy craft, and the communications college belonged to the Army). Did you ever change jobs and find out that what you'd gotten into suited you perfectly? Just before the government stopped funding anything, I was teaching at a college, putting Workers' Compensation clients through high school (some, but not all, for the second time), and doing some "distance education" work in Law for a university.
There's no doubt that people test differently. Different styles of exams can make a huge difference in just about every field that can be tested -- and there are people who can develop an intimate, complete and practical familiarity with the subject matter yet will never pass a formal "test" of that knowledge. I've had students like that; people who can lead classroom discussions and ask questions in class that demonstrate a level of understanding way beyond the "for dummies" model being used to communicate the knowledge to the masses, yet whose academic records are sullied by low passing marks and occasional retests to get credit at all. Yet, with what I was teaching (electronics, with an emphasis on avionics, while I was in the military), one couldn't throw students into the working world without testing and certification. If something goes wrong with a flight system, how do you explain to the families of the crash victims that you heard from a guy who heard from a guy that this technician seemed to know what they were doing?
Oh, and Justin --- there is a relatively large "increase your SAT score" industry out there these days. Students spend years teaching themselves how to get a high score on the styles of questions asked on the SAT. The fact that they are able to score well indicates nothing but their ability to write the SAT: I've had students with amazing scores, who were apparent genii when answering questions on a multiple-choice test, but who were missing things at a fundamental level. A student who has a few formulas memorized and knows how to pick key words out of an exam question can tell you the precise air pressure at a given altitude, but that will not stop the same person from asking why the altitude should increase as the airplane gets further from the ocean surface (true story). The SAT score may be a predictor of academic success, but academic success is no better a predictor of the ability to practically apply knowledge than the SAT itself.
Wow, Stan, electronics and mathematics. Sounds a lot like my dad. Isn't it funny how your life can take such a turn like that, but you find out it's what you really meant to do?
If Matt had continued on his current course (no pun intended), he'd be a pharmacist right now, not the genius programmer he discovered he really is!
And you make a truly excellent point about the issue of personal responsibility, I had not thought about it like that at all. Seems I'm far too used to the Rhode Island way of "I know a guy who knows a guy..." :-)
OK, Jess, true confession time. I am, for some unknown reason, lucky with multiple guess tests. It makes a LOT more sense for tests to be more practical (e.g., "Here's a problem; write some code to address the issue"). But because I'm lucky with tests, I like the mutiple guess. On several occassions, I've completed a certfication test wondering if I passed (because of the poorly worded questions, multiple answers that are correct, etc.). Then the score pops up as 94% or something. I thank the testing gods for being happy that day and go on my way. Yeah, of course I study my brains out, but tests have never really bothered me. In school my goal was always to be the first one done with the test and to blow the grading curve.
It's not fair. Not at all. Because the ability or luck to do well on multiple guess tests is NOT related to the ability to code well or apply other skills. It is related to studying, yes, but also to happening to be comfortable with the test format. Geez,and then if one gets nervous with multiple guess tests ANYWAY, it must REALLY be torture to take an ADAPTIVE test!
Stan,
When effective, the SAT prep courses teach students a very specific application of the knack for discerning answers based on limited information. The way in which they do this is by offering students information that is external to the questions. One example of a knack that makes for a good test taker is understanding what it is that the person who designed the test is trying to discover, including what erroneous thinking the incorrect answers are attempting to catch.
Phrasing it as "external information" points to the concept that underlies my argument: the qualities that make for a good SAT taker also apply to learning — as well as the application of knowledge. Examples of these qualities might be remembering broad amounts of specific information, being able to bring broad-view concepts to bear, or possessing the motivation to study and do extra work to prepare.
Your air-pressure genius might need to be taught the definition of altitude, but he'll likely remember it once told. Somebody who understands broad concepts might need to learn memorization techniques, or at least how to find specific answers quickly. A motivated worker might need to spend extra time on a project. But the standardized test on which all of these people did well will give some indication of a baseline capability.
Of course, this also illustrates that standardized tests are not sufficient of themselves. Each of the hypothetical people in the previous paragraph must be assessed and taught in such a way as to discover and address the related shortcoming. But if a university has found that incoming students with SAT scores below a certain level tend to fall behind (that an adequate baseline is not there), then it serves both the school and the applicant to suggest that he or she look elsewhere.
Let's just put it this way -- I left Mensa because I don't suffer fools lightly. The abilty to perform well on a specific style of test indicates nothing about a person's ability to practically apply any knowledge or skills they may develop to the real-world situations for which the knowledge was imparted. The "air pressure genius" is highly representative of a breed of learner who is interested only in learning the means of solving problems likely to be found on a test. Since "altitude" is merely the long form of a symbol in an algorithm, it's actual meaning is irrelevant and uninteresting to the student. Both the learning style and the attitude are engendered and fostered by the primacy of a specific style of standardized testing as a means of determining who will be selected for possible academic progress.
Far more valuable in the real world is a technician or an engineer who understands the nature and ramifications of a practical problem and may need to keep a reference handy to look up a particular formula. Standardized tests have not so far been able to test this sort of thing, so they test for the ability and desire to memorize formulae and pick out keywords from highly formulaic problem statements (things that can easily be tested using nothing but multiple choice questions).
Make no mistake here. I am not apologizing for my own abilities. I cannot recall ever having scored less than ninety-eight percent on a multiple-choice test (except a Senior mathematics contest written when I was in the ninth grade), and I am well aware of my own ability to learn in depth and practically apply what I have learned. I am also acutely aware of the inadequacy of standardized testing (or any multiple-choice testing) as a means of determining probable real-world performance, both as an instructor, and as a supervisor of field technicians.
You keep coming back to academic performance. I have stated, and will state again, that academic performance alone is no more reliable an indicator of probable real-world performance than the SAT itself. "Objective" academic performance (as measured by multiple-guess examinations) can be used as one of a set of indicators, but it cannot and should not be the sole indication used for any purpose. It is not entirely irrelevant, but absent concurrent subjective evaluation it is not a true indicator of what is, in the end, the primary purpose of education.
I can't believe I put an apostrophe in "it's" [sic] in that first paragraph. I must really be riled up.
Stan,
OK, we can put this component aside: we agree that standardized tests are insufficient measures of themselves. Moving on:
"The 'air pressure genius' is highly representative of a breed of learner who is interested only in learning the means of solving problems likely to be found on a test."
— It is up to the university, as part of its curriculum, to ensure that the knowledge that is important for students to learn is taught and tested. Furthermore, it would be crucial for an employer to ensure that new hires know or can readily grasp important concepts. Nobody has suggested that corporations blindly hire based on SAT score.
"Both the learning style and the attitude are engendered and fostered by the primacy of a specific style of standardized testing as a means of determining who will be selected for possible academic progress."
— Again, this indicates that other components of an application ought to be considered in proportionate measure. I do believe, as I stated above, that a minimum SAT score can be efficient in a general sense (when slots are much less numerous than applications), but then, I also believe that inability to begin a writing sample with a grammatically correct sentence is similarly useful, as would be straight Cs throughout high school.
"Far more valuable in the real world is a technician or an engineer who understands the nature and ramifications of a practical problem and may need to keep a reference handy to look up a particular formula."
— As I argued above, such people have strengths to which they can play for standardized tests, as well, and if the fact of the test requires them to hone those strengths, then they have not merely improved their ability to take tests. To use your language, they learn to spot and interpret various methods of applying the formula that they have learned to be behind the questions. At any rate, an interesting tangential line of thought is that businesses could and should begin exploring ways in which to find and educate these folks either to obviate higher education or to offer the new employee incentive to specify his/her studies.
"You keep coming back to academic performance. I have stated, and will state again, that academic performance alone is no more reliable an indicator of probable real-world performance than the SAT itself."
— Actually, I was referring to "academic performance" in the context of standardized tests' ability to predict it. Your point about subjectivity is well taken; neither standardized tests, nor academic performance, nor even past professional performance can entirely predict future productivity, and there will be fully competent and worthwhile people who slip through the cracks of all three. However, with a big pile of resumes or applications, relatively efficient steps for discarding the unmanageable portion of the pool must be taken.
Bottom line: in the real world, practical steps must be taken. Perhaps a purely academic scenario would help to illustrate the point: a rich, eccentric scientist has the resources to take on one "student," whom he intends to train and to whom he intends to pass the torch for future advancement of his studies when he is gone. 10,000 applications arrive. The scientist must compromise in his search in such a way as to not spend the rest of his life interviewing candidates. He must also factor in the amount of training he'll have to do and indicators of applicants' amenability to such training.
Not to get off topic, but I take it to be one of the travesties of our overly secular age that we battle imperfect standardized measures as if the possibility that they might be useful would detract from the value of those who perform poorly on them. It's not so, even of people who DON'T have some compensatory knack that a given test fails to discover.
so, i have a story about the SAT. i was in high school. making decent grades i seem to remember, but not stellar. basically a little above average, 3 point something. no honor roll or super genius clubs. just pimples and uniforms (catholic school) mostly. :-)
anyway, when i went to take the SAT i was suffering from one of the worst hangovers of my life. i had to go out of the room and throw up in a water fountain. when i came back, the english part of the test was over, they were doing math. i didn't know, and continued doing english. i was just about finished with the english when it dawned on me... i asked the proctor, she said, yeah you should be doing math now.
doh!
wearily, dizzily, blearily, i started on the math.
i scored exactly 1,000. which was good enough for the state school i applied to.
and that was the SAT... (no pre tests, no subsequent tests) ...i really have no idea why i got drunk the night before. i reeeally don't think it helped. but maybe?
*shrug*
and that's my SAT story. :-)
hehe, that is a funny story! I took my SATs stone cold sober, and never got above 980 (took the test 3 times, took the PSATs twice, and took about two or three SAT Prep classes in highschool). So NOW I know the secret -- get drunk the night before! :-)
PSAT... that sounds familiar... i think i did take that. and i seem to remember something called the CLASS? CLAST? can't seem to clear enough cobwebs out of the old noodle.
i'd always thought what happened that day was basically just a set of stupidly tragic circumstances, but according to Stan it looks like it was a good thing to do...
;-)
the thing is, i was so clueless at that time, i could have cared less what i scored, or really even where i went to school. well, i did care, but i didn't know why. i should have stayed away from the university for a few years and learned something about life before attending. i went for 2 years, made crappy grades, dropped out. but when i went back i was very excited to be there. i learned tons, made good grades and ten years after originally starting, finally graduated. and of course now i'm doing something that has mostly nothing to do with what i studied, which was finance. i finally got a good dose of programming when working at chase manhattan (vba in excel, but it stuck) and here i am... writing code.
anybody ever feel like life is the theatre of the absurd? and you are the main character?
