What makes a good question in your opinion?

[Mmm_Pasta]

I think having students take limiting cases could be one way for a student to rationalize why there solution is correct/reasonable. As for numerical answers: order of magnitude, comparison to real world scenarios, etc. are other ways of rationalizing that the numerical answer makes sense. I took a class in undergrad where the instructor had us make an explanation for whether our answer was reasonable or not. From personal experience it is very difficult to guide students into developing this intuition. However, it's not an easy thing to develop in the first place either.

 

[Andy Resnick]

I apologize. Upon moving to a new country, i have had a lot of students truly struggle with some of those types of questions, and it has made me more cautious in question writing.

Being cautious is good- it means you are being thoughtful! Slightly off-topic, but when my students ask for study tips, I often suggest they invent 'test-like' questions because of the mental effort involved. Those that do often remark how effective that strategy is.

 

[Dr. Courtney]

I have used a similar rubric for grading in the past. However, i did not include the last step, as you have it. How would you model the written assessment of why the numerical solution is correct?

That depends on the topic at hand. I usually make significant efforts to teach topic-appropriate assessment methods throughout the course, so that students have ample instruction and lots of practice. But in general, I emphasize that good assessments have three components: a double check on magnitude of the number, the direction (or sign), and the units. Units tend to be similar across topics. A student should do the math on the units when they substitute numbers and units of quantities into their final symbolic expression. An assessment on units can be as simple as "the units of the answer are as expected for an acceleration, m/s/s."

If the answer is a vector, it might be a good assessment of direction to say, "the direction of the acceleration is the same as that of the net force." Or if it is a scalar, "It makes sense that the final velocity is negative, because the ball is falling at the end, and the positive direction was defined to be upward."

Assessing the numerical magnitude tends to be more specific to the topic. But when working with Atwood machines and objects sliding or rolling down inclined planes with gravity as the only external force, I point out that the magnitude of right answers is always between 0 and 9.8 m/s/s. Numbers above 9.8 m/s/s in these kinds of problems need a lot of extra scrutiny and are probably wrong.