Thanks for delving into this - this touches on the crowdsourcing science part of the project (one aspect I particularly enjoy).
Please see inline responses...
The really cool thing about signal detection theory (SDT) is that if some basic assumptions are met, it can provide independent measurements of discrimination sensitivity and bias. The response strategy often boils down to bias. In other words, when I have uncertainty about what I saw, am I more inclined to answer "stalled" or "flowing"?
Yes! These four values are often collectively referred to as a confusion matrix (for 2-alternative forced choice tasks). These values are also, respectively, referred to as true positives, false positives, false negatives, and true negatives.
SDT does not offer any specific value in terms of helping participants to choose a response strategy. SDT's measure of sensitivity (d-prime) is based on the idea that regardless of whether you err on the side of misses or false alarms (regardless of your bias), your sensitivity will remain constant, which is what is reflected by the "blue tube". Your best strategy in all circumstances is to simply give your best answer based on what you see and what you think about what you see.
Really apt question. People would be expected, on average, to be more sensitive to a very clear stimulus than to a noisy one. And to the extent to which our perceptual filters work similarly for all movies and to the extent to which our expertise about movies is unidimensional (if it tends to be more or less, rather than having different kinds of expertise), then more sensitive people would tend to detect stalls better than less sensitive people. And I think that is generally true, though we haven't investigated this question. So for example, it is conceivable that you and Guy both have the same sensitivity, but that Guy is better at detecting stalls when vessels are straight, and you are better at detecting stalls when vessels are curved. We have not explored individual differences like that.
Exactly! In Stall Catchers, we have set up the scoring rubric to try to neutralize bias. In other words, we have tried to make both kinds of mistake equally costly, or both kinds of successes (hits and correct rejections) equally rewarding, taking into account the approximate frequency of stalled and flowing vessels that are shown. Despite this, each catcher has her/his own individual bias toward responding one way or the other, and there appears to be a (normal) distribution of such biases that is roughly centered around neutral responding (which is consistent with a neutral reward system).
There is really no strategy to optimize the blue tube behavior other than simply responding as best you can. Even though the cost of False Negative might be higher than that of a False Positive, if you tried to adjust your bias accordingly to respond, on average, with more positives (Stalls), the False Positive rate would increase much faster because of the prevalence of flowing vessel movies and the blue tube would drop just as quickly as if you biased yourself in the other direction. In other words, neutral responding is optimal and there is really no way to game the system (which, of course, is by design).
That said, for research purposes, we'd rather see a false positive than a false negative (we don't want to miss any stalls if we can help it).
Thanks again for the probing questions - I hope this helps clarify!
(p.s. Sorry for my late reply - I actually replied to most of this about a week ago, but somehow left it unfinished until now. Guy kindly brought this to my attention!)