Selective reporting: The abuse of statistics

Posted by Andre M. Bellve (MSc Candidate)

As a quick preface – what I am talking about in this post isn’t new or revolutionary. My intent with this blog is to package some of the facts in a more digestible form and hopefully steer my fellow biologists to better practice.

When we carry out statistical significance tests (I.e. ANOVAs, t-tests, etc) there are two types of errors associated with them: False positives: falsely rejecting the null hypothesis, H0, and; False negatives: falsely accepting H0. Every test has these two errors in them and they are inversely proportionate to each other. A significance level of 0.05 means that 5% of the time we will incorrectly reject the null hypothesis. This is the basis of selective reporting, A.K.A p-hacking. Collectively, we have (for the most part) deemed a 5% cut-off as acceptable, at least when we aren’t betting on human lives, and I tend to agree.

What if the chance was higher though? If there was a 10% or 25% chance of false positive, would you still trust your results? You might be thinking: “Well that’s silly Andre – who cuts off their p-values at anything higher than 0.05? Social scientists?” Well here’s the catch: if you carry out two significance tests, and both have a cut-off for significance at 5%, then the chance that there is a false positive is no longer 5% – it is much higher. This inflating of the overall error rate is the basis of p-hacking. The chance of a false positive is also increased when making pairwise/multiple comparisons!

type1error

Credit: Statistical Statistic Memes via Facebook

Now this isn’t the end of the world – you can correct for it. The read significance level that you set as your cut-off defines the overall error rate of your experiment – both the type I and II error rates. If you are doing several tests, then you have to correct for the increased chance of a false positive. Fortunately, it’s relatively easy to do! There is the extremely conservative Bonferroni’s correction where you divide your significance level by the number of tests you are doing, which is appropriate in some cases (i.e. when it’s a case of life or death). This isn’t popular among biologists as we often deal with very noisy data and it can be hard enough to pick up a signal as it is. For this reason, the less conservative False Discovery Rate (FDR) correction works well when lives aren’t on the line, as it typically leaves at least one significant result without sacrificing the integrity of your analysis. This is not an exhaustive list and you can read more about different methods for correction here.

Selective reporting happens a lot. One recent example came out of the sensory science food labs from Cornell University: Brian Wansink, director of the Food and Brand lab at Cornell, boasted online that a volunteer research assistant of his was able to take a “failed study which had null results” and produce ‘significant’ results from it (Science of Us, 2016). However, careful investigation of the published papers yielded multiple errors and inconsistencies suggesting that the research assistant had ‘p-hacked’ the data to produce these results, although it seemed she had done so unwittingly. It is this kind of behaviour that brought about an analysis by Ioannidis (2005) which found that most published research findings are actually false. The incorrect use of these methods can put at risk the objectivity of the traditional scientific method and create issues of credibility for the scientific community. With more “fake news” and anti-science rhetoric being thrown around than ever before the last thing we need are blows to our credibility or any more fuel being added to the tangerine tire fire that is my president.

tangerinetyrefire

Image Credit: politicususa.com

References

A Big Diet-Science Lab Has Been Publishing Shoddy Research — Science of Us. (n.d.). Retrieved April 8, 2017, from http://nymag.com/scienceofus/2017/02/cornells-food-and-brand-lab-has-a-major-problem.html

Ioannidis, J. P. (2005). Why most published research findings are false. PLos Med, 2(8), e124.

 

 

Advertisements

Don’t mention the P word

Posted by Tom Bodey

Scientists are always trying to communicate their research and ideas across a wide spectrum of media to varying degrees of success (and I can already feel the hoisting of my own petard here). Some of the difficulties arise because a researcher can apply caution and caveating to their results, whereas recipients may prefer to see a clear-cut outcome that makes for more straightforward decision-taking for example. However, scientists do not always help themselves, either by using jargon, or through the avoidance of terms because of the connotations they may imply.

For my latest research I have decided to enter one of these minefields by looking at individual variation in behavioural responses – you see, I’ve done it myself. This variation, particularly if examined across contexts, has been given a range of names within the scientific literature – behavioural syndromes, coping styles, behavioural tendencies – all of which shy away from the dreaded p word – ‘personality’. Now, of course, there are obvious reasons why a researcher may choose to avoid this word, and certainly to use it within air quotes. After all, without becoming Dr Doolittle, it’s pretty hard to get into a non-human head to actually assess their personality, and what you have instead is the examination of a handful of behaviours where individuals differ in their responses.

image1

Welcome to the office. Rakitu Island study site – you don’t get many laboratories with views like that.

However, even on these highly simplified ‘personality’ scales, it is becoming increasingly obvious that individual animals often differ substantially, and also consistently, in their behaviour. For example, some individuals from species as different as prawns, hyenas and albatross can be bolder, more active, or more social than others, and this variation can then affect other aspects of their lives such as how they find food. These differences are especially intriguing as the maintenance of variation stands in contrast to the movement of species towards a fitness peak. In order to maintain variation, there must be situations in which one set of behaviours are beneficial, and contrasting situations where different individuals prosper – any peak must shift through time or space.

Not content with dipping my toe into an area with such linguistic juggling, I thought I would combine this with invasive species – another area where terminology can take on unfortunately loaded connotations – by looking at individual variation across a range of niche dimensions in everyone’s favourite critter, the humble rat. The Chinese, of course, recognise the rat as the first animal within their zodiac cycle, embodying alertness, spirit and intelligence (as well as being timid, devious and gossipy to balance it out). New Zealand is ‘blessed’ with three species – all invasive and globally widespread – the Brown/Norway Rat Rattus norvegicus, Black/Ship Rat R rattus and the Pacific Rat/Kiore R exulans. I’ve been catching wild rats on offshore islands in the beautiful Hauraki Gulf, running them through behavioural trials in the field before releasing them in order to do it all again the next day, and the next.

image2

Taking pictures of rats in livetraps is tricky. This is a kiore.

Right now I just have a lot of home video, but ultimately this work should provide insights into ecological theory – for example, the ways in which interference competition affects niche space occupancy at individual and species levels. It should also have applied applications for invasive species management. New Zealand, like many oceanic islands, is in the position of having no native terrestrial mammals, and thus can manage invasive mammals, should they choose to do so, in relatively straightforward and robust ways. However, elsewhere management must avoid harming native mammal species. Triaging of the system, making adjustments to maximise the chances of catching the individuals with the greatest potential to cause harm, would be the most cost-effective way forward if eradication is not possible or desirable. And if eradication is the goal, then you need to know there aren’t individuals that will avoid all control mechanisms. Either way, identifying those individuals with a ‘troublemaking personality’ (as any press release might say but any scientific paper would not) is key.

The Challenges of Invertebrate Conservation

Posted by Simon Connolly

Invertebrates are (ironically) the backbone of every ecosystem in New Zealand, providing multiple essential services. It is troubling to note then that in NZ over 1000 invertebrate species are classified as Threatened or At Risk (full lists can be found here).  However, despite evidence they can benefit from conservation action, they are often ignored in conservation policy.

There are exceptions to this rule. Notably, the Cromwell Chafer Beetle was saved from extinction by the world’s first nature reserve dedicated to the protection of an invertebrate species (read more about this here).

The simplest reason for the lack of invertebrates in conservation work is the fact that they are very difficult to study, because of their staggering species diversity. The bane of entomologists is the ‘taxonomic impediment’, meaning that there are so many invertebrate species that we do not have names for all of them yet. I’m sure you can imagine the impossibility of trying to come up with recovery plans for species that aren’t even known to science. This diversity also increases the need for specialist work in terms of distinguishing threatened species from common close relatives.

Wetapunga

A large and iconic Weta being large and iconic.

To classify a species as threatened requires a lot of knowledge about its populations, range, and other related factors. Such data are not as easy to gather as they are for vertebrate species, given that invertebrates are often hard to see, making it difficult to come to any conclusions at all. It is no coincidence that the Weta, easily the most iconic group in NZ invertebrate conservation, are also the largest and most noticeable.

A more uncomfortable truth is that we as human-beings like to play favourites. Charismatic and cute vertebrates tug at our heartstrings, and, for most, invertebrates will never hold the same lustre despite their importance to the world. Inevitably, the species we care about more receive more conservation resources.

What can be done? Well, research in the scientific space is looking at whether we can more effectively use the data we have. This includes using known locations of invertebrates to build up a picture of their climatic preferences or looking at a species’ physical traits to speculate on the threats it may face in the wild. Hopefully, you’ll hear more about this at the end of my Master’s project (fingers crossed). Is this a magic bullet? Certainly not, but hopefully it’s a step in the right direction.

Ngaio_Weevil

The Nationally Critical Ngaio Weevil

For now, I hope when you think of threatened species you will remember that there are hundreds of species that need our attention and some of them are much less cute, but no less important than others.

Simon is a Masters Student at the School of Biological Sciences, University of Auckland. His research is focused on threatened insects and he is supervised by Darren Ward.

Taking the graveyard shift for science

Posted by Tynan Burkhardt @TynanBurkhardt

Bias is a thorn in the side of any researcher, whose goal it is to discern general effects and phenomena in the environment. The most commonly quoted of these biases is the hemispheric bias, in that there is a lot more research occurring in the northern hemisphere, where 88% of the worldly population resides. Another bias, which I have encountered in my research, is the diurnal to nocturnal bias. My research considers the patterns of nocturnal transpiration (night-time water loss) between seasons and between drought and non-drought years. By choosing to study nocturnal transpiration I have effectively taken the graveyard shift, taking leaf-scale measurements all through the night on multiple occasions and depriving myself (and others… for which I thank my volunteers greatly!) of sleep. Apparently, and I have no idea why, other researchers choose to study processes which can be measured at normal working hours, meaning day-time ecology is more frequently documented.

sleepy

Me sleeping on the job, while volunteer Helen does all the hard work.

Studying a topic at night-time and in the southern hemisphere got me thinking… how will biases such as these change in future with technological and socio-economic advancements? For example, remote regions are very much understudied due to inaccessibility. Perhaps in time, when flying cars are common place or sampling robots like the curiosity rover are more affordable, this bias could be completely rectified… at least on a global scale. Likewise, a large contributor to the hemispheric research bias is that the largest populations of the southern hemisphere are within Southern Africa, Indonesia and South America. These regions contain a disproportionate amount of the world’s poverty-stricken people who do not have the resources to contribute to scientific research as readily as the populations of North America and Europe. Fortunately, living conditions worldwide are improving and young people throughout the aforementioned regions are increasingly becoming the first of their lineage to acquire a university degree. So long as this trend continues, which we should all hope it will for reasons other than scientific utility, the southern to northern hemisphere literature gap will surely be reduced.

Biases will always be a part of science as they arise from circumstance, but an important role for the scientific community is to identify the drivers of these biases, whether they be socio-economic, technological or geographic. Scientists already account for bias when making models and statements about the natural world, but nothing can replace an increased sample size!

img_7437.jpg

Tynan is a Masters student at the University of Auckland’s Ecology Ngatahi lab group. He is studying Nocturnal Transpiration in kauri trees and is supervised by Cate Macinnis-Ng.
email: 
tbur187@aucklanduni.ac.nz

You think writing a thesis is hard? Try writing one in another language!

Posted by Carolina Lara Mendoza @carislaris

Doing a PhD is hard. Writing a thesis in English when English is not your first language is harder. According to the Organisation for Economic Co-operation and Development (OECD) in 2015, more than four million students were enrolled in higher education programs outside their home countries. Since 2005, international PhD students at the University of Auckland, New Zealand, have contributed to 45% of all PhD students. This means that from the period 2014-2016, approximately 2,552 students have enrolled in the doctoral program at Auckland Uni. That’s great! And it illustrates the good work that the University and the government have done to reduce fees and make more scholarships available. At the end of the day, we live in an era of globalisation and English is its language.

Before enrolling in a doctoral program overseas, every PhD student must prove they are proficient in English. This is measured through either a TOEFL test, an IELTS test or by an internal English examination. But does passing those tests mean we are ready to write a dissertation in English? From my experience, I must say that is not enough. Furthermore, PhD students who are non-native English speakers are expected to complete the degree with all the struggles while still learning advanced English (academic English). I’m not discounting the efforts of the University of Auckland, because we’re provided with workshops aiming to improve PhD student’s academic English, but think it shouldn’t just be up to the institution and the student. Supervisors of students for whom English is not their first language also play an important role in providing adequate and constructive written feedback, and need to be aware this will likely take time and patience. Raul Pacheco-Vega recently tweeted something of great importance “For those of you who supervise doctoral dissertations in English to be written by non-native English speakers; particularly those of you whose first language IS English: please remember and check your privilege: you get to write in your own language. They may lack the background”.

Write-consistently

In the meantime, here are some tips that have helped me to write my thesis in English:

  • Be proud of yourself and celebrate small achievements. Sounds hard while doing a PhD but it is important. Remember you’re doing something great by writing a thesis in a second language! You wrote a whole section? Celebrate. You wrote two sentences? Celebrate. Everything is a success!
  • Embrace feedback from supervisors. After 3.5 years of doing a PhD I still find this challenging. Getting feedback on a manuscript makes me feel anxious and quite often I feel like I’ve failed. It cannot be further from the truth. Feedback is the only way we can improve our written academic English!
  • Practice writing as much as you can. And don’t leave it until you start writing your thesis. For me writing grant applications at the early stages of my PhD (some of them have been successful!) was very useful, as was writing blog entries throughout, which was good for learning how to target my writing to different audiences.
  • Have a friend in your field proof-read your writing. Getting feedback from someone other than your supervisors is a good idea and helps you gain practice (and confidence).
  • Use the available tools. Either at your institution, books or even online. I’ve found that thesaurus.com is incredibly useful when I can’t think of a synonym or when looking for a word’s meaning.
  • Don’t give up. If you’re consistent and patient, you’ll get there. Rome wasn’t built in a day.

 

CalisCarolina Lara M. is a PhD Candidate within the Centre for Biodiversity and Biosecurity, School of Biological Sciences, University of Auckland. Her research interests focus on seed dispersal networks within fragmented landscapes. She is supervised by Margaret StanleyJason Tylianakis, Karine David, and Anna Santure.

The ants are coming

Posted by Darren Ward @nzhymenoptera

Ants are among the most invasive animals on the planet. Over 240 species have been recorded as being transported by humans to new geographic locations, and 19 of those are considered invasive.

L humile -antweb

Caption. The Argentine ant, a globally invasive species. Photo by Philip Herbst. Image available from Ant Web.

But not all invasions occur as the result of direct transport. Some species have managed to invade one place, survive, and then migrate to another—a process known as the “bridgehead effect”. In a new paper, just published in Proceedings of the National Academy of Sciences, we investigated these secondary invasions, and show bridgehead effects are a major driver of new invasions.

We looked at interception data, that is, records of what ant species have been intercepted at the border. Two large and long-term datasets were examined, one from the USA covering the years 1914 to 1984 and containing 51 ant species, and the second from New Zealand covering the years 1955 to 2013 with 45 ant species.

The most surprising result was that most of the interceptions did not originate from species’ native ranges but instead came from already invaded areas. In the United States, 75.7% of the interceptions came from a country where the intercepted ant species had been previously introduced. In New Zealand, this value was even higher, at 87.8%.

Interceptions also increased when they came from countries that were physically closer (Latin America for species intercepted in the United States and Oceania for species intercepted in New Zealand). Additionally, ant species that travelled the most tended to be more successful in invading a secondary location. This created a positive feedback loop between the introduction and establishment stages of the invasion process, in which initial establishments promote secondary introductions.

Overall, these results reveal that secondary introductions act as a critical driver of increasing global rates of invasions. Consequently, it is not enough simply to account for the original location of an invasive species. To better understand pathways of invasive species, we also need to follow the dynamics of spread throughout their entire range.

 

Cleo Bertelsmeier, Sébastien Ollier, Andrew M. Liebhold, Eckehard G. Brockerhoff, Darren Ward, and Laurent Keller. Recurrent bridgehead effects accelerate global alien ant spread. PNAS (2018). www.pnas.org/cgi/doi/10.1073/pnas.1801990115

 

and if you really like ants, then another really great related paper is

Cleo Bertelsmeier, Sébastien Ollier, Andrew Liebhold & Laurent Keller. Recent human history governs global ant invasion dynamics. Nature Ecology & Evolution (2017). doi:10.1038/s41559-017-0184

 

Darren Ward is an entomologist, Head Curator at the New Zealand Arthropod Collection at Landcare Research, and a senior lecturer at the School of Biological Sciences, University of Auckland.

Social Wasp Invasion on New Zealand’s Offshore Islands

     Watch my vlog on social wasp invasion on New Zealand’s offshore islands
– some of the last refuges for endangered species – below:

 

Julia Schmack is a PhD student at the Centre for Biodiversity & Biosecurity, School of Biological Scinyences, University of Auckland. She is researching the ecology and control of social wasps, supervised by Jacqueline Beggs, Darren Ward and Mandy Barron (Landcare Research). Her PhD is funded by the Biological Heritage National Science Challenge. Download the Highlights 2017 report by the Biological Heritage National Science Challenge here.

twitter_pixabay.com @julia_schmack

email_commons.wikipedia.org j.schmack@auckland.ac.nz

 

The other side of the world

Posted by Noor Rooding @noorrooding 

I left the Netherlands as winter was finally coming to an end.  Arriving in Auckland I have been able to experience the final glimpse of summer, before it heads into the depths of winter.  Coming from the Netherlands I thought I understood rain… Sadly, Auckland has taught me that there is more to rain than I had realised.

In the Netherlands I study Applied Biology at the HAS University of Applied Science in Venlo. This is a four-year bachelor’s programme and I am currently in my third year. This year I had the opportunity to go on an internship abroad. I contacted Cate Macinnis-Ng and she was happy to get some help with some ongoing projects. This is how I ended up on the other side of the world

I have done some traveling in the past but only in Europe, so going to the other side of the world was a big step. I have been in New Zealand for two months now.  One of the first things that I noticed is the average size of the New Zealander is a little bit smaller than in the Netherlands. But after all Dutch people are just tall, so it shouldn’t be a surprise. Of course, there are more obvious differences like the time difference, climate and nature. One of the major differences that has struck me is how much native bush I see, even when I am in the middle of the city.

I have also enjoyed seeing other parts of New Zealand.  The Netherlands is very flat and does not have amazing things like volcanoes. I had a great experience doing the Tongariro alpine crossing and met some awesome people there. Everyone is so friendly and willing to help you. Someone even took me on a road trip the day after the walk.

As well as having lots of fun checking out your beautiful country I have also been doing some work.  I am working on the litterfall project, which is an ongoing project involving many people. For this I am collecting litterfall material and sorting them into different categories such as species, branches and reproductive material. My main focus is looking at the reproductive cycle of the kauri.  This involves looking at the data to date and seeing if drought has any effect on the reproductive cycle.  I am expecting to see that drought conditions lead to more loss of reproductive material, such as seeds.

Coming to New Zealand has been an amazing experience that I will never forget.  One of my favourite parts of my internship has been getting involved in several other projects. I was lucky to be help with the 24-hour project climbing kauri trees. It gave me the chance to climb these amazing trees and see them up close and personal.

noortrees

Climbing the trees for the 24-hour project

I am here to learn and enjoy my stay in the country and experience as much as possible. If anyone needs help with anything, please feel free to contact me.

noor

Noor is an intern visiting from The Netherlands, working with Cate Macinnis-Ng on ‘The Litterfall Project’.  Contact details: noorrooding@hotmail.com

 

Teaching the teachers

Posted by Helen Armstrong

During my career as a primary school teacher I have helped many students plant in excess of a thousand native trees and shrubs in and around our school. And never in that time have I ever wondered about tree physiology-until now.

At present I am part of a group of teachers participating in a Science Teaching Leadership Programme [STLP] administered by the Royal Society Te Apārangi. The programme has two specific phases. The first phase involves teachers taking paid leave from their school for two terms to engage in a programme of learning that involves them working in a host organisation for approximately 15 weeks, alongside scientific staff, to get an appreciation of the Nature of Science. The Nature of Science is the over-arching theme of the New Zealand Curriculum which all schools teach science through. The second phase of the programme involves the teacher working alongside school management and their teaching colleagues to improve the quality of science learning across the school or science department.

I am being hosted by Dr Cate Macinnis-Ng and the School of Biological Sciences. She and her team of PhD and Masters students are working on projects which are looking at how environmental changes, such as drought, affects kauri and other native trees such as tanekaha.

Helen

I am enjoying discovering how and when a tree uses such things as sugar, water and carbon and how these are transported around a tree. Before being part of this research I thought I had a fairly good grasp on how a tree or plant breaths. Turns out there is so much more I didn’t know, but thanks to Cate and her team who have given me the opportunity to be involved in their research, I am learning so much more than I could ever learn by reading articles or watching YouTube clips.

Being part of an ongoing research project has really opened my eyes as to what scientific research involves. In the media you see and read about all the amazing work and breakthroughs scientists have made all around the world. But what you don’t see is the hours of work, determination and set- backs that go into discovering something new- be it a cure for the common cold or how trees react under changing conditions.

I am excited to use what I have learned in my placement back at my school and channel the natural curiosity of our students into as many different curriculum areas as possible, enabling them to have a deeper understanding of science. Some of the projects I have been involved in whilst on my placement can be modified to be used in a classroom so the children will be even more able to benefit from the experience I have had.

I am willing, able and very keen to learn as much as I can whilst on my placement which will end in late June 2018, so if you need an extra pair of hands for fieldwork, experiments, presentations or are involved in anything that you think I might find interesting, please feel free to contact me.

helena@easttamaki.school.nz

A great day (and night) of tree-climbing

Posted by Ben Cranston

On the morning of 20 March 2018, a crew of University of Auckland researchers, professional arborists, and volunteers set out for Huapai Scientific Reserve in the northern Waitākere range with a few objectives in mind: most notably, to collect 24-hr transpiration and leaf water potential values for kauri (Agathis australis). We were equipped for an overnight stay in the forest and spirits were high. Climbing, gear-and-sample running, eating, and sleeping shifts were divided up between participants and the plan was laid out…IMG_1822

The first day was mostly meant to familiarize new climbers with the protocol as well as replace some older, ailing equipment in the canopy. In effect, we ended up using day 1 to ease into an appropriate mindset for the overnight campaign which was to begin early on the second day. In addition, we were able to snag a bit of press and drone footage of the science going on at Huapai which was very cool!

IMG_1840

I also was able to obtain a great vantage point of the throughfall exclusion shelters which were recently installed at the site. In this frame, it is slightly discernible that soil under the tarps is staying dry relative to outside. Preliminary results from sapflow measurements are showing no clear distinction between droughted and non-droughted trees, but that should hopefully change over the course of the year(s) owing to these tarps.. stay tuned.

IMG_1857

Our resident arborist, Freddie Hjelm, had his enthusiasm for New Zealand forests on full display throughout the whole trip. On behalf of everyone in the Macinnis-Ng lab, I offer our sincerest appreciation to him and the rest of The Living Tree Company crew for helping us stay safe and problem-solve when the going got tough!

IMG_1893

Apart from a trove of data (currently being processed), over the course of the collection period we were all treated to views like these. Climbing up the stems of my study trees  revealed a cathedral among the canopy not to soon be forgotten.

IMG_1900

It was a lot of work from a lot of people but we made it to the next day. The volunteers did an amazing job hanging in there well into the night and I can only hope that the experience was worth the toil.

 

Ben Cranston is a PhD student at The University of Auckland. His project is part of the Kauri Drought Experiment under the supervision of Cate Macinnis-Ng.