Please give us your feedback about Stall Catchers!

seplute 2016-10-19 12:05:19 UTC #41

Hi @jfrancis! Just a quick note...

The "Flag" option serves that function just check it in cases where you "can't tell" and we will look into it!

The reason we are not just making that a button is that in some cases it may still be possible to give an answer, even if the image isn't so clear.

Although the fact that the green outlines seem to be invisible in some movies is worrying! Did you manage to flag them any of them and, perhaps, insert a comment with "green outline" keywords? We could then check those specific movies. Or just send us the page source the next time that happens, if you can!

Thanks for all you other suggestions about interface improvement - we will definitely try to do something like this! Satisfying participant curiosity is one of the main goals anyway

jfrancis 2016-10-19 16:17:34 UTC #42

I've now followed your advice - this morning I flagged a movie and added a comment explaining why. Then I clicked Flowing to complete the action. Not sure whether the words green outline were in it; green probably was. If you like, I can copy and send you a page source by e-mail the next time.

The problem may not be with the movie but with me - I've said here before that some of the outlines are extremely faint (others are clearly visible) and hard to find in a screen full of vessels. But other catchers' responses suggest they don't have this problem.

P.S. Another UI suggestion. The red dot where the expert places one or more stalls blocks me from seeing the stall(s) for myself, which would be useful if I've chosen Flowing incorrectly. I'd like to be able to switch it off and back on.

jfrancis 2016-10-21 15:55:40 UTC #43

I'm now seeing the same calibration movies again (and making the same mistakes). Are the "real" movies also repeating, and if so, should I stop until new ones are added?

pietro 2016-10-21 16:39:26 UTC #44

Great question (as usual)! The quick answer is "No - please don't stop!" We have many more real movies than expert (calibration) movies, so you even if you see some calibration movies repeat, you are likely still annotating new real movies. Also, if we happen to collect more than one annotation for a movie from the same person, we can still extract value from that response. (As it turns out people do not always give the same answer, and when we have multiple answers from the same person, whether they are the same or different, that can be informative about the movie.)

Thanks again,
Pietro

jfrancis 2016-10-23 16:19:04 UTC #45

I just got to level 34, to find that the next level is a million points higher. The next move, I clicked on what I thought was a stall, the expert disagreed, and my Maybe score dropped from 84 to 65. Have a heart! Seriously, if the point system is intended to encourage us to keep at it, even after analyzing thousands of movies, the goals and rewards need to be more attainable than this. I note that the next highest scorers behind me haven't played in weeks.

pietro 2016-10-24 19:23:06 UTC #46

WOW - level 34 is Mt. Everest territory! Each subsequent level is indeed harder to achieve than the previous one, which makes it an even more impressive achievement.

I certainly feel your pain regarding the drop in points on "maybes". The challenge for us is that if we make this change more slowly, then it takes much longer for newcomers to get points, and they might get discouraged. If we make it change faster, then someone like you, with a very high sensitivity can get frustrated. So it is kind of a balancing act - and we will do our best to improve user experience in this regard.

Toward that end, we have a new feature planned that is going to completely change how the "Maybes" are handled. We are very excited about it, and we think you will be too. Among other things, I think it will help alleviate this kind of frustration. That's all I am going to say about it for now In the meantime, I hope your Maybes climb back up quickly (if it makes you feel any better, I am going through the same thing at the moment, with my Maybes...)

Thanks for the great feedback, as usual

Best,
Pietro

jfrancis 2016-10-26 14:26:00 UTC #47

And thank you. You and your colleagues have been very patient with me.

About the Levels - would you consider capping the gap between them at something less forbidding than a million points, say 500,000? To recognize progress more often at the upper levels. Since there are no winners and losers in this "game," it's just about encouragement.

I'm very interested in what you have up your sleeve with Maybe points. Presently, it takes me many movies to recover from a 20-point drop, as the calibration movies only increase the Maybe score by 1 and don't come very quickly. Capping the drop at, say, 10 would be a significant penalty that encourages us to be more careful, which is the point, but wouldn't feel unfair.

After a few days, I'm quite satisfied using the browser's full screen mode - I'm now seeing tiny stalls that I was missing before. So I don't need a zoom feature.

P.S. I see that the option to Flag Feedback has been added, so if we have a serious challenge to the expert's Flowing/Stalled rating, we can draw your attention to it. Excellent!

jfrancis 2016-10-30 17:28:52 UTC #48

EMERGENCY! Just now, the virtual microscope failed to save an annotation and kicked me out of stallcatchers.com. I was presented with a login screen, but my e-mail address and password were not recognized and I was told I have to register. I've registered, using the same e-mail address and password as before, but now when I log into the virtual microscope, I see:

Score 0
Level 0
Movies 0
Stalls 0

Can you reconnect me with my "account"? Maybe it's just vanity, I'm sure my work isn't lost completely, but I'd rather resume with my 4.4 million points than have to start over from scratch.

pietro 2016-10-30 18:22:27 UTC #49

EMERGENCY resolved

Today we had our first unplanned server downtime! The system did not crash, but the downtime and data loss (5 hours from today) are my fault. In the course of data management, I accidentally overwrote the SC database.

Therefore, any annotations made approximately between the hours of 8:10am and 1:10pm U.S. Eastern Time, Sunday, October 30 may have been lost. I am very sorry for this and we have made corrections to ensure this doesn't happen again.

If you were annotating during this time period, please send an email to info@eyesonalz.com indicating approximately how many vessels you annotated or points you earned during that period and we will reimburse you with points.

If you registered with Stall Catchers during this time period, you will likely need to re-register. And again, please let us know and we will provide points and any additional help to get you up and running so you don't have to start again from scratch.

Fortunately, we make frequent backups of all our servers, and this was a perfect opportunity to test Stall Catchers' resilience to both human and server failures. The system failed at 1:15pm and was back up and running by 1:42pm (28 minutes later). Most of that downtime involved damage assessment and remediation analysis. Total remediation time was 5 minutes. If a catastrophic server event were to occur in the future, we expect to reduce the total down time to less than 15 minutes.

Thank you for your patience and my sincerest apologies to you, @jfrancis, and anyone else who may have been annotating during that time period.

Best, as always,
Pietro

jfrancis 2016-10-30 18:54:58 UTC #50

I did re-register with Stall Catchers so I could post the EMERGENCY message here, just before the server went offline. As I said above, my points, level, movies, and stalls statistics now show as 0. Presumably the backup you mention includes the missing stats somewhere; my points total is still listed in the top ten catchers as 4352580 with my username jfrancis. Can you reconnect them with my current registration? It uses the same user ID, e-mail address, and password as I had before the outage.

pietro 2016-10-30 19:32:02 UTC #51

Try logging in again. Everything should be (mostly) as it was. If not, please let us know!

Regards,
Pietro

jfrancis 2016-10-30 19:54:05 UTC #52

Yes! I did do some work this morning but can't tell you how much; this is fine. Thank you.

caprarom 2016-11-05 16:15:22 UTC #53

I've been enjoying the variety of new movies of late, but can't believe I'm the first one to view them all. (Surely Donna got to some of them first!) Whenever I check to see how others have rated them, it shows 0 stalls and 0 flowing. Apparently the counters are not incrementing.

pietro 2016-11-06 00:08:58 UTC #54

Hi @caprarom! Thanks for the heads up. There are quite a lot of new movies, so it is entirely conceivable that you are the first to view them. Either way, we will look into it to make sure.

Best,
Pietro

jfrancis 2016-11-06 04:02:27 UTC #55

Same with me - lots of 0 flowing and 0 stalls. Also a higher frequency of calibration movies, though that may just be my impression. This started immediately after the changeover to the orange outline color, when I'd viewed ca. 13,000 movies. Maybe just coincidence.

caprarom 2016-11-07 00:14:19 UTC #56

Well, I'm starting to see some incrementation on the counters now, albeit just 1's. You must have loaded "quite a lot" of new movies indeed!

jfrancis 2016-11-18 09:13:23 UTC #57

I'm curious. What is the "challenge bar" and what does the percentage mean?

seplute 2016-11-18 22:53:18 UTC #58

Hi @jfrancis!

If you click on the hyperlink "Challenge Extended!" it will take you straight to the full answer, which is here:
http://blog.eyesonalz.com/challenge-extended/

The challenge bar is counting the vessels we need to annotate to get through the validation study - we are trying to determine the best way to calculate the Crowd answer for each vessel, and to officially demonstrate how good the Crowd (YOU!) is Here's more about it: http://blog.eyesonalz.com/stall-catchers-30-day-challenge/

The original plan was to run the challenge for 30 days, but, as explained in the recent blog post, the challenge bar kind of ran away from us, and a few days ago we realized we are not as done as it was suggesting! (I guess you might have noticed the bar was 95% full yesterday)

So we had to turn it back a little, and to compensate we extended the challenge to run for another 30 days, leading up to Xmas With Xmas bonuses coming later on! Stay tuned for that one!

Thanks for asking
Egle

jfrancis 2016-11-19 12:00:47 UTC #59

And thank you! The challenge bar suddenly appeared without explanation, and even now the Challenge Extended link does not really explain what it is and means, but now you have.

While I'm writing, what's all this about pancakes? The two graphics in the Challenge Extended page do not display in Firefox, and when right-click and View Image, I get error 403:

"Your client does not have permission to get URL /vaVydtZ0-7x-galfPPyMxnLbacDwtMaZ--9gOk74Xwe6Twzs7Em2Ngasn17vRV2-8ngc5lVB7wAAEQU=w1366-h638 from this server. (Client IP address: 104.162.8.226)

Forbidden That’s all we know."

seplute 2016-11-19 23:17:40 UTC #60

Hi @jfrancis!

Sorry about the pics - I had trouble uploading them, but then they appeared on my browser, so I left them & went on a road trip for two days! And now turns out they're all been broken!

Argh, sorry about that.

The pancakes refer to "first pancake always burnt" but it may (or may not) have made more sense along with the pic