A text-to-image retrieval model finds people in surveillance images from a written description. You will see descriptions and images in 2 short task(s), 20 items in total (about 7 minutes). Answer every item, in order; your answers are saved in this browser as you go. When finished, press Submit my ratings; the download button is only a backup. You can change an answer at any time before you send your ratings. There are no right or wrong answers for you; we are measuring the model, not you.
For each description the model ranked image A above image B. In every item the model's decision is correct: image A is the described person. The green button under the images shows the model's decision and the attribute it used; you may answer before or after pressing it. Several candidate phrases are listed. Pick the one that best explains why A fits the description better than B, or choose the last option if none does.
Description: He has short black hair and is wearing a white shirt . He is also wearing grey short pants and sandals .
Correct attribute (model decision): “white shirt”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: A woman with brown hair is walking away from the viewer. She is wearing light blue jeans and a red t-shirt. Her hair is in a messy bun or pony tail.
Correct attribute (model decision): “blue jeans”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: She has long straight black hair and is wearing a all black outfit with red writing,She is also carrying a small pink bag .
Correct attribute (model decision): “pink bag”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: A woman wearing a black jacket with a pocket, a pair of blue jeans and a pair of gray shoes.
Correct attribute (model decision): “blue jeans”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: The woman is walking on the sidewalk and is wearing gray pants and a light gray shirt that is sleeveless.
Correct attribute (model decision): “gray shirt”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: The young woman is carrying a large dark colored bag and a pink folder. She is wearing dark rimmed glasses and a long pink shirt with a white, long-sleeve, duster-style shirt on top of it. She is also wearing green shoes.
Correct attribute (model decision): “pink shirt”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: A woman wearing a yellow shirt, a pair of white and black floral print pants and a pair of white and black shoes.
Correct attribute (model decision): “pants”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: A man wearing a pink shirt, a pair of brown shorts and a pair of black shoes.
Correct attribute (model decision): “pink shirt”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: The woman is wearing a pink flower printed dress shirt with black pants. She also has a blue purse and white shoes on.
Correct attribute (model decision): “white shoes”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Description: The woman has long dark hair and is wearing a bright purple short-sleeved blouse with a light tiered skirt and flat shoes.
Correct attribute (model decision): “blouse”. Model's decision: image A, ranked above B. Correct: A is the described person; B is not.
Which phrase best explains why A fits the description better than B?
Image A is the model's answer for the description. In every item the model's decision is correct: image A is the described person. The green button under the images shows the model's decision and the attribute it used; you may answer before or after pressing it. Image B is a different person who looks like A. Pick the phrase of the description that names a difference you can see between A and B. If several phrases differ, or none visibly does, choose the last option.
Description: He has short hair . He is wearing a grey shirt and is pulling a cart behind him .
Correct attribute (model decision): “grey shirt”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: A man wearing a black shirt with white writing on the front, a pair of black pants and a pair of headphones.
Correct attribute (model decision): “black shirt”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: The man is wearing shorts and a yellow shirt. he is looking at his phone.
Correct attribute (model decision): “yellow shirt”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: A man walking in the opposite direction wearing a black short sleeved shirt, red shorts, and black flip flops.
Correct attribute (model decision): “red shorts”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: An older boy who is wearing a blue jacket and dark pants is walking and looking to his right.
Correct attribute (model decision): “blue jacket”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: He is wearing blue tennis shoes, glasses, black ankle socks, black shorts, and a short sleeved black shirt.
Correct attribute (model decision): “black shirt”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: This man wears long black pants and a red short sleeved shirt with a white collar and white on the end of each sleeve. His sneakers are light green.
Correct attribute (model decision): “black pants”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: guy wearing a long black trench coat with white pants and black shoes talking into a microphone
Correct attribute (model decision): “white pants”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: He has dark slacks on and a dark jacket, and he is wearing glasses and also carrying a case.
Correct attribute (model decision): “jacket”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?
Description: man is wearing dark grey slacks and black dress shoes . a red striped shirt with black yellow and white stripes and is also carrying a briefcase
Correct attribute (model decision): “shoes”. Model's decision: image A. Correct: A is the described person; B is not.
Image B shows a different person who looks like A. Which phrase of the description names a difference you can see between A and B?