Please give us your feedback about Stall Catchers!

Bugs

Thanks for looking at Stall Catchers! We beta-tested it with ~100 people and hopefully we've caught all the bugs, but if you find anything that shouldn't be happening, please let us know is this thread.

General feedback form

You can also leave anonymous feedback and rate the game via this form:
https://goo.gl/forms/uqdHmKn1ZApTQ5X52

Thanks much for helping us improve Stall Catchers!

Other problems?

If you're having difficulties that aren't related to bugs - for instance, you need help on an image or can't find out how to do something - please make a new topic for that.

Hi -- I'm getting odd results sometimes.

For example, I'll view a series and decided to choose 'Flowing'. The response back is sometimes, "Not correct!" But the "Expert Answer:" says, "FLOWING", and the comment says, "Vessel is flowing."

Or, I click "Stalled" and choose my stall point, and the response is, "Correct! You get 0 points!" with, "Expert Answer: Flowing."

I'm using Chrome. I can share some screenshots if that would help. I have tried logging out and back in, but I would still get these types of responses periodically.

Hi devon,

that sounds strange indeed! Yes, screenshots would be really helpful - could you forward those to ?

And - what OS & browser are you using?

We'll try to figure out what's happening here & fix it asap! Thanks for letting us know

Hi Devon,

So sorry for the frustrating system behavior, devon. We encountered similar issues in an early prototype, but this is the first I've heard about those behaviors since we launched this version.

Are you on a Mac or PC (the following assumes PC):

In addition to the screen shots, the next time that happens could you please send the page source?

You can access it by going to the View Menu in Chrome, and selecting Developer -> View Source.

Then select (Ctrl-A) copy (Ctrl-C) and paste (Ctrl-V) that text into an email to .

Thanks again - as @seplute mentioned, we will get to the bottom of this!

Best wishes,
Pietro

I'm having the same problem! Yesterday I clicked "FLOWING" on two movies. Got the reply,"Not Correct." Movie is FLOWING from both. I typed a message outlining the problem in comment box at bottom of movie. I quit Catching and shut the site down. Signed back in today. First two movies went fine. Then. I again clicked "FLOWING." I was judged "Not correct" again. Rather than rack up wrong scores, I have quit "Catching" until this problem is solved! I don't know what else to do. I have iMac, 10.11.6- el capitan. Working with Safari browser. I would love to change to chrome browser, and if you can supply me with the proper URL, I will make the change. Thanks, Evelyn R. Smith

Dear Evelyn,

Thank you very much for reporting this problem. It's quite frustrating, I'm sure. We are still investigating and haven't been able to reproduce it in our testing. However, we have been testing on the PC, so we will now perform similar testing on a Mac and across various browsers and hope to get to the bottom of this quickly.

Our developer reports that, despite the feedback you were given, your answer was recorded correctly. Minor consolation, I'm sure

We will let you know as soon as we have more information about this and thank you again - both for solid annotation effort, and for reporting this.

One question - have you annotated any movies since you last experienced the error? This will help us investigate.

Best,
Pietro

@devon and Evelyn,

I just learned that we pushed a change to Stall Catchers that we think will fix this issue. The change was implemented after Evelyn's most recent comments. However, because we have been unable to reproduce the issue, we can't be certain that this will resolve the issue.

Could we implore you to try again and see if the issue recurs? And if so, please stop annotating immediately and let us know. That way, we will easily be able to see which one gave you the problem.

Thanks for your patience with our growing pains. We will get to the bottom of this!

All the best,
Pietro

No, I haven't Pietro. I'll do some Catching again, and stop annotating immediately as you suggested. I really love being a part of this research. I don't want to drop out. Thank you for your help. Evelyn

Thanks so much for your kindness and support, Evelyn!

Update: we found the real problem! It is not what we originally thought, but we have fixed it and are now trying to finish up some other updates that were in progress so we can push it to the server.

If you annotate now you will likely encounter the same problem again. But we will let you know as soon as the issue is fixed.

Thanks again,
Pietro

Oh, Pietro---I think you have fixed it! I examined several movies and the program appears to be working fine---no more error messages. But,while eyesonalz.com is apparently back working well---this Catcher is not. I'm an early morning person. My brain goes to sleep at sunset. So, I'll say goodnight to all of you and will back annotating early am tomorrow. I really appreciate the effort you put into solving this problem. Alz@home is brand new to me---but I already feel right at home and part of the family.

Hi Evelyn,

I'm glad you had good results tonight, though I wonder if you just happened to get lucky We think this issue is occurring for prolific contributors because you are occasionally seeing movies you have previously annotated. If your new answer disagrees with your old answer you might see this error. Now that we understand it, we can fix it, but it probably won't be resolved until later tomorrow.

By the way, for very difficult vessels it is normal to get a distribution of responses from the same contributor. For example, if a person sees the same vessel 10 times, they might say stalled 3 times and flowing 7 times. And that can actually be useful to the analysis. Once we get more smoothly underway, though, seeing the same movie twice will be almost unheard of.

Anyway, we are so glad to have such a strong contributor and communicator in our budding community Looking forward to many shared adventures and successes to come!

Best,
Pietro

Warning
Error saving movie answer
Please continue to next movie.

Sometimes I get this error when I choose an answer for the movies. Could this be a server glitch?

Hi @sean4046!

Thanks for alerting us to this. I think this sometimes indicates a network connection issue, but I will forward this to our developers so they can advise.

How often does it happen? And is it happening more frequently now than in the past?

Thanks again,
Pietro

Devon and Evelyn,

Thanks again for all your great feedback on the "Incorrect Answer" bug! We think it is fixed. Please let us know if you encounter any more issues.

Very best,
Pietro

Oh, dear, I'm deep into the do-do. Missed so many easy ones....and then when I tried to go to a new movie, I got the message--"Error saving movie answer. Please continue to next movie." Well, I've tried that over and over, and instead of getting another movie, I get a repeat of the error message. Did I drop off the charts completely because I called too many of them wrong? I was doing so well, I'm just sick at this turn of events. Any hope for me? Evelyn

Oh my goodness, Evelyn! It seems that as soon as we caught the old gremlin, there is a new gremlin in the system out to get you!

We are looking into this issue (with high priority), based on your feedback and Sean's similar feedback. I will report back as soon as we have any insight into the issue. Your observations about having missed several in a row is actually very helpful to identifying the issue - so thank you for providing such informative feedback.

We have recently pushed several new updates to system to improve user experience (the sensitivity bar is now more responsive to correct and incorrect answers). In the process, ironically, we may have created this new issue. We will keep you posted!

Best wishes,
Pietro

I'm back "catching" again! I wasn't sure it was a gremlin---I feared I had lost it completely. The blue sensitivity bar was completely bereft of color. But, with your latest "fix" I quickly climbed to a higher level (21) and blue has returned to the bar. However, I remain at Rank 14. I'll work on that. Gosh, you guys are geniuses! Evelyn

I had all the problems described above during the weekend, but today not. Looks like you fixed them.

Sometimes StallCatcher's expert sees a stall when the white line is solid, and sometimes stationary small black gaps in the line aren't counted as stalls, which costs me sensitivity and points. Are the experts infallible? But it's OK, I'll keep on calling 'em as I see 'em.

This project is addictive.

Dear @jfrancis,

Thanks for letting us know the bug fixes seem to be working!

I'm sure the experts aren't completely infallible That said, for the calibration movies, multiple experts examine each movie, and then when there is disagreement, it goes to committee and they discuss the movie until they reach a consensus. If there is not consensus, then the movie isn't used.

You observations are apt. One formal definition of a stall employed by the experts is the absence of apparent movement in the vessel. By this definition, a vessel that appears solid white, would be considered a stall. However, this presentation of a stall accounts for only about 5% of stalls. We are working on a more advanced tutorial that includes lots of examples and considers some of these more nuanced cases. So if you are now saying, "how could anyone possibly get those ones right if you don't tell us that a stall can look like that", then you are absolutely right! At this stage, we are not expecting folks to recognize that particular type of stall, and in those cases the expert answer will be confusing. Again, this will change soon

Regarding the case where you have observed stationary black gaps and the experts called it flowing, I can only speculate. Sometimes, more than one vessel appears within in outline - and in those cases, occasionally a stall occurs in the vessel that is not the "target" vessel, and can be a red herring. The target vessel is the one that most closely follows the shape of the green outline. However, because we realize that even experts can make mistakes I would like to tell you about a feature that we will be adding soon, which will be a checkbox that allows you to flag bad feedback on the feedback screen. If you check that flag, an expert will look at the vessel and comments to see if there is an issue, and either provide clarification in the expert comments or, if justified, change the expert answer.

We are glad you find the game addictive and, just as a teaser, I will let you know that we have some new features planned to make it even more so. So please stay tuned - we are just getting started!

Thanks again for this useful feedback. Perhaps one of our annotation experts will also chime in.

All the best,
Pietro