What makes a good question in your opinion?

[Hlud]

In a textbook i use for my AP class, i have come across the following question, "A man enters a tall tower, needing to know its height. He notes that a long pendulum extends from the ceiling almost to the floor and that its period is 15.5 s. How tall is the tower?"

Physics classes are rife with these types of questions, yet i think these types of questions are horrible. For one, it is only nominally 'real-world'. My colleague would joke that the smart student faced with finding the height of the building would just ask the building manager. The result is also meaningless, at least to the student. The period of 15.5 s could have very well be 14.7 s, or 13.2 s, or 16.1 s. To me, giving the student a random number tests their ability to use their calculator, more than anything else.

Ever since i started teaching, i have been trying to articulate what qualities make up a good question. What makes a good question in your opinion? If this is a well-written question, what do you like about it, and other questions like it? If not, what else makes this, and other questions like it, poorly written?

 

[Dr.D]

What makes a good question depends to some extent on the student level. For an introductory level, the question given is not bad in that it asks the student to connect the height of the tower with the length of the pendulum and the length of the pendulum with the period of the pendulum. The student must also recognize that gravity is a factor in this problem.

The more advanced the student it, the more things a good question asks of him. By degrees, we want to ask the student to synthesize new knowledge with previous knowledge to address the question posed. We don't want a question that requires no previous knowledge at all, and we don't want a question that is based only on the most recent topic of discussion. It is the integration of old and new knowledge that provides value.

 

[ZapperZ]

In a textbook i use for my AP class, i have come across the following question, "A man enters a tall tower, needing to know its height. He notes that a long pendulum extends from the ceiling almost to the floor and that its period is 15.5 s. How tall is the tower?"

Physics classes are rife with these types of questions, yet i think these types of questions are horrible. For one, it is only nominally 'real-world'. My colleague would joke that the smart student faced with finding the height of the building would just ask the building manager. The result is also meaningless, at least to the student. The period of 15.5 s could have very well be 14.7 s, or 13.2 s, or 16.1 s. To me, giving the student a random number tests their ability to use their calculator, more than anything else.

Ever since i started teaching, i have been trying to articulate what qualities make up a good question. What makes a good question in your opinion? If this is a well-written question, what do you like about it, and other questions like it? If not, what else makes this, and other questions like it, poorly written?

I don't quite get the issue that you are having with this question. Why shouldn't this question be asked, and why shouldn't there be a quantity attached to a variable? This isn't just a mindless plug-and-chug, because they have to reason out the significance of the period of oscillation that is given.

Zz.

 

[Andy Resnick]

Ever since i started teaching, i have been trying to articulate what qualities make up a good question. What makes a good question in your opinion?

A good question is one that allows the instructor to accurately assess the student's mastery of some amount of course content.

Heh... that's the rub, right? Students will often consider a question 'good' if they can get the right answer; that is, it is matched to their skill level and has an unambiguous, single, answer that (in physics) is generally either a number ± units or an equation/expression/'formula'. This approach primarily considers homework or test questions.

By contrast, instructors may feel that a good question is thought-provoking, designed to combine and synthesize ideas, and upon solution (if one exists!), should elicit more questions or foster an open-ended discussion. This approach considers plug-n-chug a low-level skill acquired on the way to higher levels of analysis.

Ideally, there should be a mix of the two extremes; I like to use the first type of question as a warm-up exercise, building up to the second type of question.

 

[Mark44]

My colleague would joke that the smart student faced with finding the height of the building would just ask the building manager.

This reminds me of an interesting post I saw many years ago about a student who was tasked with determining the height of a building, and given a barometer to use to make the calculation. Apparently the student was bored with these types of problems, so instead came up with a long list of possible solutions.

• Use the height of the barometer as a sort of measuring stick, and walk down the stairs keeping track of how many barometer heights he marked off.
• Drop the barometer from the top of the building, noting how long the barometer took to reach the ground, and calculating the height from this time interval.
• Attach the barometer to a long rope and lower it to the street. The length of the rope is the height of the building.
• Go to the building superintendant, and say, "If you tell me the height of this building, I will give you this fine barometer."
• And others...

See https://en.wikipedia.org/wiki/Barometer_question.

 

[symbolipoint]

I...

Ever since i started teaching, i have been trying to articulate what qualities make up a good question. What makes a good question in your opinion? If this is a well-written question, what do you like about it, and other questions like it? If not, what else makes this, and other questions like it, poorly written?

The question can be clearly understood, ideally requiring no assumptions.
The question is directly related to the book section where the question is placed, although should be allowed to rely on earlier-learned material.

 

[mpresic]

Your joking friend is on the wrong track. What makes him (or her) think the building manager would have any better idea of the height of the building than your friend has. It could be the building manager might tell him, the pendulum has a period of 15.5 seconds, and your friend should take the trouble to use the well known physics equation for the period of the pendulum in terms of the gravitational acceleration and the length.

The building manager (in the real world ) might also suggest that building managers are more likely to know this physics equation, than the height of the building that (s)he manages. The manager might also remind your friend that the equation derived assumes a small angle approximation. The manager of the building usually doesn't have blueprints, or architectural drawings of the buildings he manages. (The same can be true for the building superintendent).

Back to the main point. I do not see anything wrong with the question as stated. The point that the period measured could be 16.1 sec, or 13.2 seconds etc.
It could be.

I might be a little bit put off, but if a student were to use his own number for the period (within reason), I would repeat his calculation with his values to verify he was correct consistent with these values, (OK, I might take off 1 point out of 10, for either not reading the problem correctly.) After all, I am more interested in whether the student understands the concepts, than the height of the (fictional) building.

For students in ninth grade, the height of the building might be determined from the length of it's shadow, and an angle sighted with a protractor taped to a soda straw. Dr D is correct that a good question should consider the level of the student tested.

For freshman physics, the height can be determined by a pendulum, and in a later chapter, a barometer. Most students will recognize that the information given in the problem should be used to get the answer, so they might get fewer points (or no points) if they were to suggest they would use the shadow and angle and elementary trigonometry to find the height of the building rather than the period of the pendulum.

 

[mpresic]

TThinking back at the OP's question. I think the best questions test more than the ability to use equations to get specific numbers unless these numbers are relevant or important. For example, I think it might be better to ask the student the period or length of a pendulum, with a length equal to the radius of the Earth. The teacher could remark in an afterword that this period is equal to the period of a satellite if it had this radius from the Earth center.

It might be interesting to calculate the period of a pendulum with a length hanging from the Empire State Building (for NYC students), or the Golden Gate Bridge, (for San Francisco students).

I do like "human interest" stories. I used the Guinness book of Records to ask students, what velocity a baseball achieved when Joe Sprintz attempted to catch it dropped from a zeppelin (the height was in the Guinness book off records.). I added as an afterword, it knocked out several teeth. One edition of the record book mentioned the record for catching a grape in one's mouth. (I believe it was more than 300 feet.) What was the minimum velocity for such a remarkable grape. What is the minimum velocity for the longest champaign bottle cork flight listed in the Guinness book of records.

Most of my tests even at the introductory level did not ask for numbers but to derive the relevant equations. Specific numbers were of secondary importance.

 

[Hlud]

I don't quite get the issue that you are having with this question. Why shouldn't this question be asked, and why shouldn't there be a quantity attached to a variable? This isn't just a mindless plug-and-chug, because they have to reason out the significance of the period of oscillation that is given.

I don't like the question because it is almost mindless plug-and-chug. The majority of the time spent on the problem is not spent on doing physics. As Dr. D suggested, the student has to connect length of string with height of building (well, the question does that for you) and then the period with the length of the string (which is suggested by the equation). Because the physics is so little (at least in comparison to the math being done), i deem it a poor question.

Now, a variance to this question would be to cut the string as the bob is at the lowest point on its path. Dr. D suggests doing this, as it connects different concepts. IN my opinion, that doesn't solve the issues, as the math done (working with the model) is still far greater than the physics done (relating the model to the actual phenomenon). All the kinematics done will, as well, be almost entirely plug-and-chug. The only physics done, i believe, is realizing the direction of velocity at the bottom of its path, and that gravity is the only force remaining after the cut, hence projecting this problem into the kinematics unit. Again, the mathematics portion is much greater than the physics portion.

 

[Hlud]

A good question is one that allows the instructor to accurately assess the student's mastery of some amount of course content.

I believe this is one of my biggest frustrations from college that i still carry over. All of my college courses would talk about the equations we derived. What was the scope of the equation, etc... All of my textbooks would do the same. There were more words than equations. When we got homework questions (and by extension, test questions), all i, and anyone i knew, would have on my paper was equations. Just the math. And no professor would comment negatively on that. Clearly, there is some disconnect between course content and effective assessment writing.

By contrast, instructors may feel that a good question is thought-provoking, designed to combine and synthesize ideas, and upon solution (if one exists!), should elicit more questions or foster an open-ended discussion. This approach considers plug-n-chug a low-level skill acquired on the way to higher levels of analysis.

Ideally, there should be a mix of the two extremes; I like to use the first type of question as a warm-up exercise, building up to the second type of question.

I agree with your point of increasing difficulty. But, if the warm-up question isn't thought provoking at all, then i fail to see the point of it as a step to the thought-provoking question.

What makes a thought-provoking question in your opinion?

 

[Hlud]

TThinking back at the OP's question. I think the best questions test more than the ability to use equations to get specific numbers unless these numbers are relevant or important. For example, I think it might be better to ask the student the period or length of a pendulum, with a length equal to the radius of the Earth. The teacher could remark in an afterword that this period is equal to the period of a satellite if it had this radius from the Earth center.

I agree with this. The number is a lot of times irrelevant, as in this case. Maybe an addendum like "The man really wanted to visit a building that is at least X high, did he get achieve his dreams?" would at least make the number relevant, but the question becomes silly (sillier actually, how many tall buildings do you walk into with a pendulum just hanging from ceiling to floor?).

But, does a derivation alone satisfy the requirements of a good question, in your opinion? From my experience, a derivation can be good, but is not enough.

 

[Dr. Courtney]

Ever since i started teaching, i have been trying to articulate what qualities make up a good question. What makes a good question in your opinion? If this is a well-written question, what do you like about it, and other questions like it? If not, what else makes this, and other questions like it, poorly written?

A good question is unambiguous and allows the student to apply the principles of the section or chapter to solve a problem quantitatively. It is of a level of appropriate difficulty for the course, which in college physics (or AP) should usually mean more than simply identifying and plugging numbers into a single three letter formula.

Poorly written questions have one or more of the following qualities: ambiguous, encourage formula roulette, are too advanced or too simple for the course level, and/or cannot be easily related to the principles in the section where they appear.

I don't worry much about simplifying assumptions that may be unrealistic in most "real world" applications. Part of a good physics course is teaching and learning how those simplifying assumptions might be justified and how much inaccuracy may be introduced if the simplifying assumptions are less than perfect.

 

[mpresic]

I cannot say for sure because I have not visited the museum for a while but I think the Smithsonian, Chicago Field Museum, Boston, or one of the museums does indeed have a Foucault;s pendulum that extends from the roof (or perhaps slightly below the roof) to the floor. Would the same question be less silly if you said, the Foucault's pendulum at the Smithsonian Natural History Museum has a length of (look up the length, I'm sure you can find it), what is the period.

Another possibility that would be harder for the student would be: The pendulum inside a grandfather's clock on the Earth has a period of 1 second. If you take the pendulum to Mars, where the surface gravity is 0.38 that of Earth, what would be the period of the pendulum.

Before you think this is routine plug in chug, keep in mind that the GRE's the gatekeeper of many of out undergraduate's entry into grad schools contains about maybe 50 of such questions. The other half ask for algebraic answers, but these are quick answers too.

I remember when I had to take the GRE's I envied schools that addressed these easy answer in a minute or two questions. My school asked us to solve problems like where to aim a rifle if you want to shoot a deer from the Merry-go-round. Account for the Coriolis force (of course). We also had easy questions like what is the angular velocity of the second hand on a watch. The first question might be worth 30%, and the second 3%. Most good physics courses contain a mixture of questions at varying levels.

I do agree with your concern about (perhaps) your main point. I examined the standards of learning for high schools in a particular state a while back. One standard said:

The student should be able to calculate the moment of inertia of a spinning disk with the equation I = 1/2 M R squared, and L = 1/2 I w squared
I asked myself, why. If the student computes the moment of inertia, on the way to using the conservation of angular momentum or energy, as a step in a more complicated problem, this is OK. If the student understands the angular momentum is the total momentum of all particles in the disk around the fixed axis, this is OK too.

Calculating physical quantities for the sake of knowing what the values are without any regard to their meanings is not OK.

Let me finish with a mea culpa.
At one time I realized the center of population of the US moved (perhaps 40 miles) from 1970 to 1980. I asked my students assume the average human weighs 100 lb, and there were an average of 300 million humans in the US during this decade. What is the momentum of the US population. (I dressed up the problem a bit by declaring where the centers of population were geographically). I thought I was being clever.

Thirty years later I find myself considering who cares what the momentum or energy is. Neither the momentum or the energy are conserved in any meaningful analysis which could be done over time. The value is just an idle curiosity. It cannot be applied to anything

 

[Andy Resnick]

I agree with your point of increasing difficulty. But, if the warm-up question isn't thought provoking at all, then i fail to see the point of it as a step to the thought-provoking question.

What makes a thought-provoking question in your opinion?

Warm-up questions, IMO, simply help wake up the student- 8am classes need some warm-up time.

For me, a thought-provoking question doesn't have a single correct answer, and maybe not even a single answer. They should involve more than 1 concept. In the context of intro physics courses, some of these questions could be like:

"Explain the connection between Newton's third law and conservation of momentum"
"I observe an acorn fall about 10 meters out of a tree, hit the ground, and bounce back up about 1 meter. How much mechanical energy was lost, and where did that missing energy go?"

Here's a question from one of my previous exams:

“Tommy John” surgery is the name given to a surgical procedure used to repair a torn ulnar collateral ligament (UCL), an elbow injury common to professional baseball pitchers. You will analyze the forces and torques acting on the elbow occurring during a baseball pitch.

1) What is the acceleration of the baseball corresponding to a 90 mph (40 m/s) pitch? Hint: this problem is simpler to solve if you assume the ball travels about 4m during a pitch.
2) Since, in addition to the 145g baseball, the pitcher must accelerate his arm, what force does the pitcher’s body generate? Hint: a human arm weighs about 6% of total body weight.

We just assumed that the entire arm accelerates as fast as the ball. This isn’t exactly true, but proceed as if it were.

3) Since there is rotation about the shoulder and elbow, this force creates torques on both the shoulder (rotator cuff) and elbow (UCL). Given your answer to (2), estimate the amount of force acting only on the ball and lower arm (below the elbow).
4) This is a distributed force, so let’s simplify and assume this (partial) force acts entirely on the center of mass of the lower arm + ball, approximately 0.2 m from the elbow, and perpendicular to the arm. What is the torque applied to the elbow that the UCL must stabilize?
5) Bonus (2 pts): why is correct ‘follow through’ motion so important, not just in baseball but any other rapid motion (kicking, punching, etc)?

How did you do:
A study estimated the average peak torque to be 64 Nm (Newton-meters), with a normal range between 52 and 76 Nm. A second study showed an average UCL failure load of 32 Nm. This indicates that pitching frequently results in stress that is far in excess of the observed failure load of the UCL."

 

[PeroK]

Warm-up questions, IMO, simply help wake up the student- 8am classes need some warm-up time.

Here's a question from one of my previous exams:

“Tommy John” surgery is the name given to a surgical procedure used to repair a torn ulnar collateral ligament (UCL), an elbow injury common to professional baseball pitchers. You will analyze the forces and torques acting on the elbow occurring during a baseball pitch.

1) What is the acceleration of the baseball corresponding to a 90 mph (40 m/s) pitch? Hint: this problem is simpler to solve if you assume the ball travels about 4m during a pitch.
2) Since, in addition to the 145g baseball, the pitcher must accelerate his arm, what force does the pitcher’s body generate? Hint: a human arm weighs about 6% of total body weight.

We just assumed that the entire arm accelerates as fast as the ball. This isn’t exactly true, but proceed as if it were.

3) Since there is rotation about the shoulder and elbow, this force creates torques on both the shoulder (rotator cuff) and elbow (UCL). Given your answer to (2), estimate the amount of force acting only on the ball and lower arm (below the elbow).
4) This is a distributed force, so let’s simplify and assume this (partial) force acts entirely on the center of mass of the lower arm + ball, approximately 0.2 m from the elbow, and perpendicular to the arm. What is the torque applied to the elbow that the UCL must stabilize?
5) Bonus (2 pts): why is correct ‘follow through’ motion so important, not just in baseball but any other rapid motion (kicking, punching, etc)?

How did you do:
A study estimated the average peak torque to be 64 Nm (Newton-meters), with a normal range between 52 and 76 Nm. A second study showed an average UCL failure load of 32 Nm. This indicates that pitching frequently results in stress that is far in excess of the observed failure load of the UCL."

That sounds more like a Physio than a Physics exam to me!

 

[ZapperZ]

I don't like the question because it is almost mindless plug-and-chug. The majority of the time spent on the problem is not spent on doing physics. As Dr. D suggested, the student has to connect length of string with height of building (well, the question does that for you) and then the period with the length of the string (which is suggested by the equation). Because the physics is so little (at least in comparison to the math being done), i deem it a poor question.

I don't believe it is simply a plug-and-chug. That's what I was arguing about. The student has to make a connection between (i) the period of oscillation and (ii) the length of the pendulum.

Now, you may argue that this is simply looking up the formula in the book. But for me, being aware that there IS such a relationship is itself knowledge! I'd be very happy if the student learns that there is a relationship between variable A and variable B.

For many of us, we find many things to be patently obvious or trivial in basic, intro physics. But for many students, especially for non-STEM students taking such a class, this is not so. One of the course objectives that I have for such a class is to give students the skill to convert "word problems" into mathematics. Now, for many of us, this is a trivial exercise, but for many of these students, it is a daunting task.

So sure, if this question was given to 2nd year physics major, then I'd say this is way too easy and simple. But if this was given to a physics class where the students are non-physics or non-STEM students, then I see zero issues with this. In fact, it might easily be a topic of discussion.

Now, a variance to this question would be to cut the string as the bob is at the lowest point on its path.

This would be silly because one might as well simply measure the length of the string and get it over with.

Zz.

 

[Andy Resnick]

That sounds more like a Physio than a Physics exam to me!

It's forces and torques. Basic intro physics.

 

[Hlud]

A good question is unambiguous and allows the student to apply the principles of the section or chapter to solve a problem quantitatively.

A question that i really like is "A person in a boat floating in a small pond throws an anchor overboard. What happens to the water level of the pond? Defend your response." Would you classify this as a good question?

I don't believe it is simply a plug-and-chug. That's what I was arguing about. The student has to make a connection between (i) the period of oscillation and (ii) the length of the pendulum.

I would like to ask you the same question, regarding the anchor in the pond.

I would definitely classify it as a good question because you are spending more time and thought with the physical phenomenon than with the mathematical construct. I am not doubting that the question in the OP spends zero time with the physical phenomenon, but much less time than with the mathematical construct (that is, doing the algebra, where the physics, generally, is irrelevant at this point).

This would be silly because one might as well simply measure the length of the string and get it over with.

This is my mistake. If the pendulum was above the ground, then when cut, the bob's motion could then be described by the kinematics these students learn in a previous chapter.

 

[Hlud]

“Tommy John” surgery is the name given to a surgical procedure used to repair a torn ulnar collateral ligament (UCL), an elbow injury common to professional baseball pitchers. You will analyze the forces and torques acting on the elbow occurring during a baseball pitch.

For the most part, i would definitely classify this Tommy John surgery question as good. My two issues are a) it can be argued that someone unfamiliar with baseball has a disadvantage; and b) i would make that bonus question required.

In fact, i think a good follow up would be to compare pitching styles of today, with the pitching styles of yesteryear. In that, the students would have to make a case whether or not the older wind-up reduced the stress put on the UCL. That may be over their heads, though.

 

[Hlud]

Would the same question be less silly if you said, the Foucault's pendulum at the Smithsonian Natural History Museum has a length of (look up the length, I'm sure you can find it), what is the period.

I think my issue with that is the Foucalt pendulum is intended to demonstrate the rotation of the earth. I would want to have the students explain the physics of that, but then it may (or may not) become more difficult than intended.

The pendulum inside a grandfather's clock on the Earth has a period of 1 second. If you take the pendulum to Mars, where the surface gravity is 0.38 that of Earth, what would be the period of the pendulum.

I might want to rephrase the question to remove the numbers. 'Would this grandfather clock run slow, fast, or still keep time? Defend your response.' I don't see the purpose of these numbers. But, yes, i think this is a better question, because the physics involved has increased, albeit slightly, in proportion to the math done.

Calculating physical quantities for the sake of knowing what the values are without any regard to their meanings is not OK.

I feel like this is explains my issues with most physics problems i come across. Edward Redish would argue that there is a hidden curriculum in physics, one that goes beyond the computed answer. I think most physics teachers agree that the computed answer is just a small part of the learning. However, when a student spends most of her time doing the math to get to that answer, she is right to hold that answer in higher regard. I certainly fell into that category when i was a student.

I am not against having numbers in problems, per se, but i would want those numbers to have significant meaning. Labs are a great place for that. Outside of labs, i would be much more cautious in including random numbers.

At one time I realized the center of population of the US moved (perhaps 40 miles) from 1970 to 1980. I asked my students assume the average human weighs 100 lb, and there were an average of 300 million humans in the US during this decade. What is the momentum of the US population. (I dressed up the problem a bit by declaring where the centers of population were geographically). I thought I was being clever.

Thirty years later I find myself considering who cares what the momentum or energy is. Neither the momentum or the energy are conserved in any meaningful analysis which could be done over time. The value is just an idle curiosity. It cannot be applied to anything

I definitely still do this from time to time. I think it is very difficult to change old habits, though. But, if i were to compare myself now from when i first started teaching, i am definitely improving!

 

[rootone]

What makes a thought-provoking question in your opinion?

A question which provokes thoughts.

 

[Dr. Courtney]

A question that i really like is "A person in a boat floating in a small pond throws an anchor overboard. What happens to the water level of the pond? Defend your response." Would you classify this as a good question?

I think it depends on the learning objectives of the course. I tend to design physics courses as vehicles to improve quantitative problem solving skills which include sequences of deliberate problem solving steps, unit analysis, and careful assessments.

Cutesy problems where two competing effects exactly cancel out don't really give students the opportunities I like to practice and grow in those things, but rather depend on them seeing a trick of clever physical insight. Great question for courses designed more to build qualitative physical insight. But for my purposes, I guess it depends how the quantitative problem solving is balanced in the rest of the course.

 

[Hlud]

I think it depends on the learning objectives of the course. I tend to design physics courses as vehicles to improve quantitative problem solving skills which include sequences of deliberate problem solving steps, unit analysis, and careful assessments.

Cutesy problems where two competing effects exactly cancel out don't really give students the opportunities I like to practice and grow in those things, but rather depend on them seeing a trick of clever physical insight. Great question for courses designed more to build qualitative physical insight. But for my purposes, I guess it depends how the quantitative problem solving is balanced in the rest of the course.

I don't quite follow. Because the explanation to the question requires an equation, i would argue that it is a balance of quantitative and qualitative reasoning (you are reading an equation, and using words to apply it to your scenario). I think you are arguing that quantitative reasoning requires the use of actual numbers, which are absent in my question. Then, would you argue that undergraduate classes become qualitative only at junior and senior year, because they are almost entirely derivation based?

And, i don't quite understand why it is a cutesy problem. AP Physics 1+2 are now heavily reliant on these types of questions, and students are doing awfully on these exams. My experience with teachers is that they haven't changed their practices much after the AP change. They may teach slightly different material, but they are generally using the same quantitative-only methods. If these problems require clever physical insight (which i think is a great thing to have), then they clearly aren't getting that insight using the methods currently in use.

 

[symbolipoint]

All this "cutesy" and qualitative stuff has not been explained very clearly. There was once a professor who gave instructions on a test once, " Assign variables to ALL numbers, and solve each problem purely in variables - NO substitution of any actual numeric values". That person was a Physics professor.

 

[Dr. Courtney]

I don't quite follow. Because the explanation to the question requires an equation, i would argue that it is a balance of quantitative and qualitative reasoning (you are reading an equation, and using words to apply it to your scenario). I think you are arguing that quantitative reasoning requires the use of actual numbers, which are absent in my question. Then, would you argue that undergraduate classes become qualitative only at junior and senior year, because they are almost entirely derivation based?

If you think an equation is required, perhaps you are reading

"A person in a boat floating in a small pond throws an anchor overboard. What happens to the water level of the pond? Defend your response."

differently from me. There is no explicit requirement for an equation in the statement of the problem, so my expectation from years of teaching is that most students I've had would be unlikely to give one.

Quantitative thinking does not always require the use of actual numbers, but if you are thinking that the above problem requires the use of an equation, it is a BAD question, because the above question is guilty of ambiguity. A better question would specify how the response should be defended: "Defend your answer by quantifying any change in the pond's water level in terms applicable quantities: the density of water, the density of the anchor, etc."

 

[Dr. Courtney]

And, i don't quite understand why it is a cutesy problem. AP Physics 1+2 are now heavily reliant on these types of questions, and students are doing awfully on these exams. My experience with teachers is that they haven't changed their practices much after the AP change. They may teach slightly different material, but they are generally using the same quantitative-only methods. If these problems require clever physical insight (which i think is a great thing to have), then they clearly aren't getting that insight using the methods currently in use.

I would never make a determination regarding whether the teacher or the students are more at fault in poor exam performances without knowing how much of the reading and homework assignments the students are doing. Teachers cannot and should not shoulder the blame for poor exam performance when students who are not doing the assigned reading and homework exercises score poorly on the exams.

I am also not a big fan of AP courses. I've known too many cases where students perform very poorly in downstream courses having managed to get through the AP courses without learning (or perhaps not retaining) very much. I'm not sure what you mean by "quantitative only" methods. Most quantitative homework and exam problems require a lot of qualitative thinking to draw the right picture, identify the right physical principles involved, and assess the reasonableness of the numerical answer and the units. Sure, students may be tempted to approach problems with formula roulette, but it's part of a teacher's job to steer them away from that and to assign questions that do not reward that approach.

 

[Andy Resnick]

For the most part, i would definitely classify this Tommy John surgery question as good. My two issues are a) it can be argued that someone unfamiliar with baseball has a disadvantage; and b) i would make that bonus question required.

Anyone can nit-pick any question ad infinitum: most of my students don't know how to drive a car and/or have never been on a roller coaster; I suppose that puts them at a disadvantage for those types of questions.

In fact, i think a good follow up would be to compare pitching styles of today, with the pitching styles of yesteryear. In that, the students would have to make a case whether or not the older wind-up reduced the stress put on the UCL. That may be over their heads, though.

Since my central point is that 'good' questions are thought-provoking and lead to open-ended discussion, I'm glad that you are able to further explore this question!

 

[Dr. Courtney]

All this "cutesy" and qualitative stuff has not been explained very clearly. There was once a professor who gave instructions on a test once, " Assign variables to ALL numbers, and solve each problem purely in variables - NO substitution of any actual numeric values". That person was a Physics professor.

OK, that's a fine approach for some courses.

Knowing that many downstream courses require lots of practice with numbers and units, this is not an approach I would take in introductory courses I've taught. Many students in intro courses are still coming up the learning curve with things like orders of magnitude, reasonable accelerations and velocities, significant digits, and things that are better practiced by substituting in actual numeric values. Taking that approach would leave them under-prepared for downstream science and engineering courses.

In these courses, my approach has tended to be to emphasize solving problems in terms of variables as far as possible, and then substituting in numbers and units at the end - performing the required math operations on the units as well as the numbers to use whether the units obtained agree with the expectations as an additional assessment of the reasonableness of the final expression before substituting in numbers.

 

[ZapperZ]

In all of this, there appears to be a lack of appreciation of the type of students one is teaching to.

I'm teaching two very different groups of students: (i) science/engineering majors and (ii) non-STEM majors.

It will be very silly to teach the same thing and the same way to this very different groups of students, not just because they have different preparations, but also because they have different set of interests. That is why I stated that the original question stated by the OP may be inadequate for group (i) but highly appropriate for group (ii).

But even "bad" questions can easily be a learning point. I consider it a bonus if a student brings up an ambiguity in a question in which he/she could come up with a different scenario, interpretation, or even answer. Only someone who has either an understanding of the concept involved, or someone who has made an effort to tackle the problem, will be able to discover this, and either one of those is very encouraging. How many times have we ourselves in this forum found someone asking a question which the member thought was "clear" or "simple", and yet, we found to be vague, confusing, or can have varying interpretation? Our background and expertise allow us to see the bigger picture and all the possible alternatives.

There is no such thing as a perfect question. As Andy has stated, we can always nitpick something about any question being asked. There will always be some unintended inherent bias in many physics questions, because we often have to appeal or refer to something "familiar", which may not be familiar to others. I once tried to find something to replace "projectile motion" as popular example of basic 2D motion in intro physics, because many female students told me that they don't usually play with "cannons" or throw balls up in the air. However, a female professor cautioned me that if I bring in a more "feminine" examples, I may be stereotyping the female students. That essentially put a stop to that.

So again, I see no issue with the original question of this thread. I'd love to see how a student would tackle it and see his/her line of thought.

Zz.

 

[Hlud]

There is no explicit requirement for an equation in the statement of the problem, so my expectation from years of teaching is that most students I've had would be unlikely to give one.

I think it depends on how you model answering the question. It could be argued that there is no explicit requirement for an equation for the question posed in the OP. I could expect some students trying to pull a fast one with "Really tall." Both questions relate answers to quantifiable things (height of the tower or water level). If i model both types of questions (one with defined values, and the other without) using equations, i would definitely expect of students to answer these types of questions using an equation.

I would be hesitant to use your suggestion, unless i snip it off at 'applicable quantities.'

 

[Hlud]

Anyone can nit-pick any question ad infinitum: most of my students don't know how to drive a car and/or have never been on a roller coaster; I suppose that puts them at a disadvantage for those types of questions.

I apologize. Upon moving to a new country, i have had a lot of students truly struggle with some of those types of questions, and it has made me more cautious in question writing.

 

[Hlud]

I would never make a determination regarding whether the teacher or the students are more at fault in poor exam performances without knowing how much of the reading and homework assignments the students are doing. Teachers cannot and should not shoulder the blame for poor exam performance when students who are not doing the assigned reading and homework exercises score poorly on the exams.

Well, i wouldn't want to put the entire blame on the teacher as well, but i don't think the students would suddenly jump in blame over the course of one year. There was a clear drop in performance after the change, and still has a lower achievement rate (even considering the change to scoring) now, compared to AP B years. A huge part of this drop is the quick shift in expectations.

Nonetheless, a lot of teachers i have discussed these changes with do not care for and struggle implementing qualitative responses. The textbook may use words to explain the physics. The teacher may use words to explain the physics. Why is the student restricted from using words to explain physics? You may argue this is not the case, but student work suggests otherwise. You may see a picture; it would be rare to see a written explanation of the physical insight to solving the problem; either way it becomes all math from there. You rarely see a written explanation after a numerical solution.

What i am asking for is what qualities of a question would better generate these types of responses. Ones where emphasis is placed on physical insight. The question in the OP has, in my opinion, low physical insight, and hence why i don't consider it to be a good question.

And, by no means am i suggesting the abolishment of math in physics (i say this because a lot of teachers and professors believe that conceptual physics is physics without math). I am arguing for the inclusion of more qualitative explanations in addition to that math.

 

[Dr. Courtney]

Well, i wouldn't want to put the entire blame on the teacher as well, but i don't think the students would suddenly jump in blame over the course of one year. There was a clear drop in performance after the change, and still has a lower achievement rate (even considering the change to scoring) now, compared to AP B years. A huge part of this drop is the quick shift in expectations.

Nonetheless, a lot of teachers i have discussed these changes with do not care for and struggle implementing qualitative responses. The textbook may use words to explain the physics. The teacher may use words to explain the physics. Why is the student restricted from using words to explain physics? You may argue this is not the case, but student work suggests otherwise. You may see a picture; it would be rare to see a written explanation of the physical insight to solving the problem; either way it becomes all math from there. You rarely see a written explanation after a numerical solution.

What i am asking for is what qualities of a question would better generate these types of responses. Ones where emphasis is placed on physical insight. The question in the OP has, in my opinion, low physical insight, and hence why i don't consider it to be a good question.

And, by no means am i suggesting the abolishment of math in physics (i say this because a lot of teachers and professors believe that conceptual physics is physics without math). I am arguing for the inclusion of more qualitative explanations in addition to that math.

It seems like you are wanting to solve with questions issues that can be addressed with a grading rubric.

Both me and several departments I know awarded most points on physics problems based on factors other than the mathematical solution.

20% was for drawing and labeling a picture or diagram
20% was for identifying the important physical principle (Conservation of Energy, for example, or Newton's 2nd law)
20% was for writing down an orderly sequence of steps to solve the problem
20% was for the numerical solution
20% was for a written assessment for whether and why the numerical solution was correct

This grading rubric awards 80% of the points for stuff on the paper other than math.

 

[symbolipoint]

It seems like you are wanting to solve with questions issues that can be addressed with a grading rubric.

Both me and several departments I know awarded most points on physics problems based on factors other than the mathematical solution.

20% was for drawing and labeling a picture or diagram
20% was for identifying the important physical principle (Conservation of Energy, for example, or Newton's 2nd law)
20% was for writing down an orderly sequence of steps to solve the problem
20% was for the numerical solution
20% was for a written assessment for whether and why the numerical solution was correct

This grading rubric awards 80% of the points for stuff on the paper other than math.

Most of those "20%" items ARE the Mathematics; but this is a matter of interpretation.

 

[Hlud]

It seems like you are wanting to solve with questions issues that can be addressed with a grading rubric.

Both me and several departments I know awarded most points on physics problems based on factors other than the mathematical solution.

20% was for drawing and labeling a picture or diagram
20% was for identifying the important physical principle (Conservation of Energy, for example, or Newton's 2nd law)
20% was for writing down an orderly sequence of steps to solve the problem
20% was for the numerical solution
20% was for a written assessment for whether and why the numerical solution was correct

This grading rubric awards 80% of the points for stuff on the paper other than math.

I have used a similar rubric for grading in the past. However, i did not include the last step, as you have it. How would you model the written assessment of why the numerical solution is correct?

I have abandoned grading this way because i am trying to expand on the student's giving me sufficient physical insight. I think using Newton's 2nd Law does not satisfy that.