Category: uncategorised

Watch the video about Gino Francesca and Dan Ariely presented in above in Canvas, then answer the questions below.

What exactly did Francesca and Ariely do that created the scandal?

Why was it so shocking that these two researchers in particular did this?

What was the ultimate outcome?

Be sure to post your opinions and respond to at least two other students by February 15. You do not have to provide peer-reviewed sources for your post, but if you do reference outside material, then you should cite it in APA 7 format.

I have several other videos on this case if anyone would like me to rip them into a file and post them. Or you can easily look it up for yourself. Judo has several videos on this particular topic.

here is a transcript of the Video:

0:00

Academia is broken universities are

0:03

broken the way that academic research is

0:05

published is broken that’s the message

0:07

that’s come through loud and clear over

0:09

the last few weeks thanks to three

0:11

articles concerning the research of

0:13

Francesca Geno if you don’t know what

0:15

I’m talking about Let Me Explain

0:16

Francesca Geno is a professor of

0:18

Behavioral Science at Harvard University

0:20

she is extremely well known in the field

0:23

I’ve talked about her research to

0:25

clients before I’ve recommended books on

0:27

this channel to you guys that use her

0:28

work as a key reference I’ve used her

0:30

research before as references in my own

0:32

essays and work that I did at University

0:34

when it comes to academic Fame Francesca

0:37

Gino is up there as you would expect

0:38

from someone who is a professor at

0:41

Harvard however the reason why she’s so

0:43

well known is because her research tends

0:44

to bring out a lot of very surprising

0:47

findings now some people just think this

0:49

research is cool and don’t think much

0:50

more about it but a lot of people in the

0:52

industry have been quite skeptical of

0:54

Francesco Gino and her work because her

0:56

results just seem a little bit too good

0:58

her hypotheses are really wacky but yeah

1:00

they always seem to be proved correct

1:02

the effect sizes from her studies seem

1:04

to be really large and her statistical

1:06

significance just seem a little bit too

1:08

significant so while some of us have

1:10

been skeptical of her work for a while

1:11

nobody has taken the time to actually

1:13

investigate her research and go into her

1:15

data to see if they can find anything

1:16

fishy

1:18

until now these three guys Yuri Joe and

1:20

laif are also professors of Behavioral

1:22

Science and other related subjects from

1:24

different universities across the world

1:26

and they took it upon themselves to

1:28

investigate Francesca Gino and her data

1:30

to see if there was anything fishy going

1:32

on and spoiler alert they found a lot of

1:34

fishy stuff in the data and that’s what

1:36

the three articles that they released

1:37

are talking about each article relates

1:39

to a different study by Francesco Gino

1:41

and in this video I’m going to be taking

1:43

you through each one the results of

1:45

their investigation are shocking damning

1:47

for Francesca Gino but I think they

1:49

speak even louder volumes about the

1:50

state of Academia in general and that’s

1:52

what I’m going to be concluding on at

1:53

the end of this video so without further

1:55

Ado let’s jump into the first study so

1:57

this first article is called cluster

1:59

fake and it’s referring to a paper

2:00

written by Gino in 2012 along with her

2:02

collaborators Shu Nina Mazar Dan arieli

2:05

and Max baseman given the fact that I

2:07

know the first names of all of those

2:09

researchers with the exception of Shu

2:10

should tell you that all of these

2:12

researchers are very well-known people

2:13

in the field of Behavioral Science so in

2:15

this study they were trying to get

2:17

participants to be more honest and the

2:19

hypothesis was that if you put an

2:21

honesty pledge at the top of a form

2:22

that’ll make people more honest when

2:24

they then fill out the rest of the form

2:26

so all of the studies in this paper by

2:28

these authors were looking at this idea

2:30

that if you put an honesty pledge at the

2:32

top of a form people will be more honest

2:34

than if you put the honesty pledge at

2:35

the bottom of a form now the first study

2:37

in this paper was led by Francesca Geno

2:39

our protagonist so in this study

2:41

students were brought into a lab to

2:43

complete 20 math puzzles in five minutes

2:45

the students were told that they would

2:46

be paid one dollar for each math puzzle

2:49

they solved correctly and the way that

2:50

this worked is that when students walked

2:52

into the room there were two pieces of

2:53

paper they had their work paper and

2:56

their report paper so on the work paper

2:57

they write down their workings for the

2:59

math questions and of course their

3:00

answers and then on the report paper

3:02

they would then have to report how many

3:04

answers they got correctly and therefore

3:05

how much they should get paid the

3:07

students were then told that before

3:08

handing in their report paper to the

3:10

researchers and getting paid that they

3:12

should shred their original work paper

3:13

the idea behind this is that by

3:15

shredding their work paper there’s then

3:17

a stronger incentive for them to cheat

3:19

on the report paper and lie about how

3:21

many answers they got correct since the

3:23

researchers in theory should never know

3:24

how many answers they got right on the

3:26

work paper but what the students didn’t

3:28

know was that the shredder at the back

3:30

of the room was not a normal Shredder

3:31

what the people in the experiment don’t

3:33

know is that the shredder has been fixed

3:36

so the shredder only showed the sides of

3:38

the page but the main body of the page

3:40

remains intact now in order to test the

3:43

hypothesis of the researchers on the

3:45

reporting paper the participants were

3:47

split into two groups half of them had

3:49

an honesty pledge at the top of the

3:50

paper and half the planet honesty

3:52

pledged at the bottom of the paper with

3:54

the idea being of course that those who

3:56

sign the honesty pledge at the top would

3:57

then cheat less going forward so what

4:00

was the result well the result showed a

4:02

massive effect from this simple

4:04

intervention according to what was

4:05

published in the study originally for

4:07

the students who silently honestly

4:08

pledge at the top of the form only 37

4:11

percent of them lied but when students

4:12

signed at the bottom of the form 79 of

4:15

students lied this is a massive effect

4:18

size that the researchers are reporting

4:19

and as a result of that this study

4:21

gained a lot of public attention and I

4:24

have talked about it with many people in

4:25

the past before because it is so

4:27

surprising but that’s why these

4:29

Vigilantes were suspicious the results

4:31

just seem a bit too good can it really

4:33

be the case that simply moving an

4:35

honestly pledge from the bottom to the

4:36

top of a form can have such a dramatic

4:39

effect on the amount of cheating that

4:40

happens it seems pretty unlikely so our

4:43

Vigilantes managed to Source the

4:45

original data set that was published by

4:47

the authors of the study and when they

4:49

looked into the data it just seemed a

4:51

little bit fishy if you look at this

4:53

table and specifically look at the left

4:54

hand column the P hash column this is

4:57

referring to participant ID this is the

5:00

unique ID given to each participant in a

5:02

study and as is highlighted in yellow

5:04

there are some weird anomalies in the

5:06

way that this data has been sorted

5:07

because when you look at this data it

5:09

seems obvious that this has been sorted

5:10

by first the condition so all of

5:12

condition 1 are together then all of

5:14

condition two are together and then in

5:16

ascending order of the participant ID

5:18

which means that the numbers should

5:20

consistently get bigger as you go down

5:22

the line and there should be no

5:23

duplicates remember each participant has

5:25

a unique ID so when you look at this

5:27

data it’s a bit weird we’ve got 249s

5:29

here that’s a duplicate that should

5:31

never happen and then at the end of the

5:33

condition one set of participants you

5:35

have participant 51 coming after 95 then

5:38

12 then 101 like that sequence doesn’t

5:40

make any sense and similarly when you

5:42

get to condition two we start with 7

5:44

then 91 then 52 then all the way back

5:46

down to 5 again these entries in the

5:48

data set look suspicious they look like

5:50

they’re out of sequence which suggests

5:52

that somebody maybe has tampered with

5:54

them so our Vigilantes are suspicious of

5:56

these rows so then you have to ask the

5:58

question why would the researchers want

6:00

to tamper with the data well it’s

6:02

because they would want to show a bigger

6:04

effect than those actually seen in the

6:06

real data the more dramatic the effect

6:09

of the intervention is the more

6:10

surprising the result of the study is

6:12

and therefore the more likely it is to

6:14

get published in a top journal the more

6:16

likely it is that this will make a lot

6:17

of press headlines that they will get

6:18

lots of interviews and work off the back

6:20

of it and so there’s a strong incentive

6:22

for the researchers to fudge the data a

6:24

little bit make the effect seem larger

6:26

than it really is and so that’s what our

6:28

Vigilantes were looking for they wanted

6:30

to see if these suspicious rows in the

6:32

data set showed a bigger effect than the

6:35

normal data that wasn’t suspicious and

6:37

sure enough that’s exactly what they

6:39

found if you look at this graph the red

6:41

circles with the cross show the

6:42

suspicious data and the blue dots show

6:44

the unsuspicious data and as you can see

6:46

the circles with the red crosses are the

6:48

most extreme ones meaning that these few

6:50

data points are inflating the effect

6:52

size now the article goes on to show how

6:55

our Vigilantes did some very clever work

6:57

to unpack the Excel file that this data

6:59

was stored in and they were able to show

7:01

quite clearly that these suspicious rows

7:02

were manually resorted in the data set I

7:05

won’t go into it on this video because

7:07

it’s quite technical but I’ll have a

7:08

link to all of these articles in the

7:10

description if you want to read them in

7:11

full but as you’ll soon see this theme

7:13

of suspicious data and then there’s data

7:16

showing extremely strong effect sizes

7:18

will be a recurring pattern so let’s

7:19

move on to study two now this second

7:22

article is called my class year is

7:24

Harvard and you’ll see why in a second

7:26

they’re looking at a study from 2015

7:27

written by Francesca Gino as well as

7:30

kuchaki and golinski again two fairly

7:32

well-known researchers in the field now

7:34

the hypothesis for this study in my

7:36

opinion pretty stupid the hypothesis is

7:39

that if you argue against something that

7:41

you really believe in that makes you

7:42

feel dirty which then increases your

7:45

desire for cleansing products which is

7:48

kind of silly in my opinion but

7:50

nevertheless this is what they were

7:51

researching so this study was done at

7:53

Harvard University with almost 500

7:56

students and what they asked the

7:57

participants to do was the following so

7:59

students of Harvard University were

8:01

brought into the lab and then asked how

8:02

they felt about this thing called the

8:03

queue guide I don’t really know what the

8:05

cue guide is but apparently it’s a Hot

8:06

Topic at Harvard and it’s very

8:08

controversial some people are for it

8:09

some people are against it so when they

8:11

were brought to the lab they were asked

8:12

how do you feel about the queue guide

8:13

and they either said they were for or

8:15

against it and then the participants

8:16

were split into two groups half the

8:18

participants were asked to write an

8:20

essay supporting the view that they just

8:22

gave so if they said I’m for the queue

8:24

guide they had to then write an essay

8:25

explaining why they were for the queue

8:27

guide but then half the participants

8:28

were asked to write an essay arguing

8:30

opposite to the point that they just

8:31

gave so if they said I’m for the queue

8:33

guide they would then have to write an

8:35

essay explaining why they should be

8:36

against the queue guide again the idea

8:39

being that those who are writing an

8:40

essay against what they actually believe

8:42

in would make them feel dirty because

8:44

after they’d written this essay they

8:45

were then shown five different cleansing

8:47

products and the participants in the

8:49

study had to rate how desirable they

8:51

felt these cleansing products were on a

8:53

scale of one to seven with one being

8:55

completely undesirable and seven being

8:58

completely desirable and again the

9:00

authors found a strong effect you can

9:02

see here that the p-value is less than

9:05

0.01. and for those of you who haven’t

9:07

had any academic training and statistics

9:09

basically when you’re doing a study like

9:11

this you’re looking for a p-value that’s

9:12

less than 0.05 that’s the industry

9:14

standard if it’s less than 0.05 you say

9:17

yes I’m confident that the effect that

9:18

I’m seeing is caused by the manipulation

9:20

that I just did so less than 0.1 is an

9:24

extremely strong effect you’re basically

9:26

100 confident that what you’re seeing in

9:29

the data is caused by the manipulation

9:31

that you did so once again our

9:33

Vigilantes are suspicious of this very

9:35

strong effect size so the managed to

9:37

Source the data online and do a little

9:39

bit of investigating and what they find

9:41

are some weird anomalies in the kind of

9:43

demographic data that the participants

9:44

have to give when they enter the study

9:46

and this is very common in psychological

9:47

studies that participants have to give a

9:49

little bit of demographic data about

9:50

themselves which gives the researchers a

9:52

little bit more flexibility about how

9:53

they cut up the data later on so in this

9:55

particular study the participants were

9:57

asked a number of demographic questions

9:58

including their age their gender and

10:00

then number six was what year in school

10:02

they were now the way this question is

10:04

structured isn’t very good in my opinion

10:05

in terms of research design but

10:07

nevertheless there are a number of

10:08

acceptable answers that you can give to

10:10

you in school because Harvard is an

10:12

American School you might say I’m a

10:14

senior right which is a common thing or

10:15

a sophomore you might write the year

10:17

that you’re supposed to graduate 2015

10:19

2016 Etc or you might indicate a one a

10:22

two a three or four or a five to

10:23

indicate how many years of school that

10:25

you’ve been in there these are all

10:26

different answers but they’re all

10:27

acceptable and make sense in the context

10:29

of being asked what year in school are

10:31

you and so when our Vigilantes go into

10:33

the data that’s exactly what they saw in

10:34

this column a range of different answers

10:36

that were all acceptable all except for

10:38

one there were 20 entries in this data

10:40

set where the answer to the question

10:41

what year in school are you was Harvard

10:45

that doesn’t make any sense what year in

10:47

school are you Harvard

10:49

what right that doesn’t make any sense

10:51

and the other thing that was suspicious

10:52

about these Harvard entries is that they

10:54

were all grouped together within 35 rows

10:56

again this was a data set of nearly 500

10:58

different participants and yet all of

11:00

these weird Harvard answers were within

11:02

35 rows so once again our Vigilantes

11:05

treat these Harvard answers as

11:07

suspicious data entries they mark them

11:09

in red circles with crosses and as you

11:11

can see the ones that are suspicious are

11:14

again the most extreme answers

11:16

supporting the hypothesis of the

11:18

researchers with the exception of this

11:20

one but come on it’s most suspicious

11:22

when you look at the ones on argued

11:24

other side so these are the people who

11:26

wrote an essay arguing against what they

11:28

didn’t believe in and therefore were

11:30

supposed to feel more dirty and find

11:31

cleansing products more appealing all of

11:33

these suspicious entries on that side of

11:35

the manipulation went for seven that

11:37

they found all of the cleaning products

11:39

completely desirable and so what are

11:41

Vigilantes go on to say is that these

11:43

were just the 20 entries in the data set

11:45

that looked suspicious because of this

11:46

Harvard answer to the demographic

11:48

question but who’s say that the other

11:50

data in the data set was not also

11:51

tampered with but just they were more

11:53

careful when they filled in this column

11:54

and didn’t put Harvard since it seems

11:56

pretty clear that at least these 20

11:58

entries were manipulated and tampered

12:00

with some way it probably means that

12:01

there are other entries within this data

12:03

set that were also tampered with are you

12:04

shocked yet I hope you are but it’s

12:06

about to get worse because there’s a

12:07

third article to do with Francesca Gino

12:09

so this third article was released

12:11

literally yesterday the day before I’m

12:12

filming this video and it’s called the

12:14

cheaters are out of order this is

12:16

written by Francesca Gino and a guy

12:18

called wiltermuth I don’t know

12:19

wiltermuth but again I find it

12:21

incredibly ironic that all of this

12:23

cheating and fake data is being

12:25

conducted by researchers who are

12:27

studying the science of honesty it is

12:29

incredibly ironic so in this third study

12:32

Gino and her co-author are investigating

12:34

the idea that people who cheat people

12:37

that lie who are dishonest are actually

12:40

more creative and they call the paper

12:42

Evil Genius how dishonesty can lead to

12:45

Greater creativity

12:48

really so let’s quickly go through how

12:51

the study worked participants were

12:52

brought into a lab where they were sat

12:53

at a machine with a virtual coin

12:56

flipping mechanism what the participants

12:58

are asked to do is to predict whether

13:00

the coin will flip heads or tails and

13:02

then they would push a button to

13:04

actually flip the coin and if they had

13:06

predicted correctly about whether it

13:07

would go heads or tails then they would

13:09

get a dollar so again there’s a strong

13:10

incentive to cheat so the participants

13:12

were right down on a piece of paper how

13:14

many predictions they got correct and

13:15

then they would hand that to the

13:16

researcher in order to get paid but then

13:18

of course the researchers would then go

13:19

back and look at the machine that they

13:21

were flipping the coin on to see how

13:23

many they actually got correct and then

13:24

they were able to tell how many times

13:26

that participant had cheated so after

13:28

they had completed the coin flipping

13:30

task they were then given a creativity

13:32

task and the creativity task was how

13:34

many different uses can you think of for

13:36

a piece of newspaper so in Psychology

13:38

this is a pretty common technique for

13:40

testing creativity you give somebody an

13:42

inanimate object and then you say how

13:43

many uses can you think of for this

13:46

inanimate object and again with this

13:48

study we see a very strong effect size

13:50

remember the magic number that academics

13:52

look for is p less than 0.05 and here we

13:56

have P less than 0.01 so basically what

13:59

that means is that there’s an extremely

14:00

high likelihood that the effect that the

14:02

academics are seeing is caused by the

14:04

manipulation that they did so again our

14:06

Vigilantes are suspicious but this one

14:09

is interesting because our Vigilantes

14:10

were able to actually get the data set

14:12

from Geno several years ago so they got

14:15

this data set directly from Geno so

14:17

again when our Vigilantes look into the

14:19

data they find some weird things going

14:21

on as you can see it seems to be sorted

14:23

by two things firstly by the number of

14:26

times the participant cheated so all the

14:28

people who didn’t cheat at all are zeros

14:30

and then the number of responses is the

14:32

number of different uses for a newspaper

14:34

that that participant could come up with

14:35

and those are clearly ranked in

14:37

ascending order but as you can see from

14:39

this next screenshot some of the

14:40

cheaters are out of order so these are

14:42

the people who cheated once who

14:44

basically over reported one time and the

14:46

number of uses that they could come up

14:47

with for the newspaper a route of

14:49

sequence here we have 3 4 13 then 9 and

14:53

then back down to five again then back

14:55

up to nine then five then nine and eight

14:56

the nine is just a total mess right so

14:59

these ones that are highlighted in

15:00

yellow are the… [Content truncated to 3000 words]