Judging the plausibility of a sentence


Experimenter: Neal Snider

Contact: JaegerLab@bcs.rochester.edu

By accepting this HIT, you confirm that you have read and understood the consent form, that you are willing to participate in this experiment, and that you agree that the data you provide by participating can be used in scientific publications (no identifying information will be used).  I consent to this experiment:


You will see two sentences below.  The first one, a reference sentence, has been given a rating of 100.  Your task is to judge how much more acceptable or natural the second one sounds relative to the reference, and score the second one accordingly. For example, if you think the second sentence is 5 times more acceptable, you would give it a 500.  If you think the second sentence is 3 times less acceptable, or 1/3 as acceptable as the reference, you would give it a 33.  You may also use numbers less than 1, so if you though the sentence was 1000 times less acceptable, you would give it a 0.1

Make your judgement based on how likely you think the sentence would be written or spoken by a native speaker of English, not whether it obeys any rules of grammar you may have learned in school.

To further illustrate, here's an example of the task using lines.  The following is a reference line, which has a score of 100:



The line below seems to be approximately twice as long, so you would give it a score of 200:



On the other hand, this line is about 1/4 as long as the reference, so you would give it a score of 25:



You just need to perform this kind of judgement for the acceptability of sentences.  Here we go!



Reference:  The office workers gossiped.

Score: 100

Test:  It was the office workers that gossiped.




Feedback on this HIT: