KH6HZ wrote:
"KC4UAI" wrote:

It seems that a lot of folks are "memorizing" the test
questions and not mastering the material.

I've been stating this very thing for close to 10 years now.

Given the number of questions in the pool, it's not
impossible to memorize just the questions and not know the
concepts.

It is important to put "memorize" in quotes, because (as others have
mentioned) it is highly unlikely someone memorizes verbatim the question and
exact answer. More likely what actually happens is people become familiar
enough with the question pool after drilling long enough that they simply
recognize the correct answer -- no real "memorization" per se of the actual
question or answer. Much like the same way you become familiar with, say,
streets along your daily commute, even though you probably do not have a map
memorized in your head.

Exactly - you may not even consciously know the street names, but you
know
the route.

Here's another "memorization" example:

Way back when "Trivial Pursuit" first came out, somebody gave me the
game as a present. I kept the card sets out and carried a small bundle
around with me. I'd glance
at them at odd moments - waiting for/riding the elevator, during TV
commercials, while waiting for the washer or dryer to complete a load,
etc. In a relatively short time I had gone through the first box of
cards twice.

I didn't try to consciously memorize the questions and answers on the
cards, I just read the questions, tried to guess the answers, and then
looked to see if I was right.

The end result was that I was near-unbeatable in a game *if* they
started with the first box of cards. I hadn't really "memorized" all
the Q&A, or even most of them, but having seen them before put me way
ahead of most other players.

I'd argue that this is very short sighted

It is, IMO, very short sighted, because people who pass the examinations
using this method do not, IMO, meet the goals of the ARS as outlined in 97.1

Perhaps, but that's not the big issue.

What I see as the big issue is that such testing may actually do a
disservice to the amateurs themselves, because they wind up with a
license but not the basic knowledge
on how to set up a station and operate it.

so one wonders what the solution here is... I suppose we
could increase the question pool by 10 fold or so and make
it easier to learn the material than memorize the
questions?

My proposed solution is to eliminate question pools entirely, and instead
have a computerized question pool which is entirely randomly generated based
on various parameters. For example, take this question from the Tech
license:

T7B10 (B)
What is the satellite sub-band on 70-CM?
A. 420 to 450 MHz
B. 435 to 438 MHz
C. 440 to 450 MHz
D. 432 to 433 MHz

Now, rather than having 4 set answers, why couldn't we simply have a
computer program generate the correct answer and 3 distractors
automatically?

Because that wouldn't help the situation at all - at least not in the
above example.

First off, a regulations question is essentially a memorization
question. A Ph.D. in EE,
a pile of patents and the Nobel Prize in physics won't help a person
answer that question if they don't know the relevant rules in Part 97.

Second, if the exact questions are publicly available, figuring out the
correct answer is pretty easy. Then all the person has to do is
"memorize" the correct answer enough to recognize it. Changing the
distractors doesn't make any difference. In fact, if one intends to
"memorize" the pool, the first step is to blank out all the distractors
and only look at the right answers!

Where such an approach would have an effect would be in questions like
Ohm's Law, where the values could be randomly generated.

Some people have argued that my idea makes the test too "hard", or makes it
appear as a "graduation exam".

From the perspective that the exam is harder, that is probably true. You
would actually need to know the material, rather than simply become familiar
enough with the question pools to pass the examination.

Maybe not. I think that, in the long run, it is actually easier to
learn the material.
It's the short run that is the problem.

However, in no way do I support (or suggest) that we make the examination
"harder" from a material perspective. If an applicant is supposed to "know"
ohms law on an examination, is it too much to ask that they really
demonstrate they "know" it, rather than simply "know" what the answer to the
question is, with no real understanding of the theory behind the question?

Agreed! The issue is *not* how "hard" the tests are, but how good they
are.

Like Cecil once said... The examinations are not supposed to be graduation
exams, nor do I support any type of proposal to make them more difficult,
from a content perspective. My suggestion, however, which I've posted for at
least 6-7 years, is to simply make the question pools computerized to
eliminate the ability of applicants to "memorize" the Q&A's, and ensure that
applicants actually know the material they are tested on.

Agree again!

In practice, however, not much can be done other than to enlarge the
question pools
and possibly have computer randomization of values. The FCC is clearly
not going to
take over the testing jobs that have been done by unpaid volunteers for
more than 20
years.

73 de Jim, N2EY