Statistical analysis of the reports archive

[Tingle]

I thought it would be interesting to analyse the reports from the archive section to see what hidden gems can be found from the data... Some of it's surprising, some of it is nearly on par with my predictions before conducting the analysis and the rest may just be coincidental!

2963 reports were analysed from 3rd June 2016 to the 3rd October 2011, though in some of the below graphs June 2016 and/or the entirety of 2016 and/or the entirety of 2011 were left out so as to not distort the overall picture (as only partial data was available for 2011 and we're only half way through 2016). I've indicated where this has had an effect.

You will find some ex-TC names in the data, I chose not to remove them for simplicity.

Spoilers have been used so you can choose what you want to see.

Relationship between the probability of an admin taking action and the number of posts in the thread

Spoiler (Click to View)

Something I found particularly interesting when analysing the data is the relationship between the number of posts that occur in a thread and the chances of the admin dealing with the report taking action.

It appears that as the number of posts increase the chances of action being taken decrease almost linearly until a tipping point of 10 posts is reached, at which point the trend fails. Fitting a second order polynomial regression model to the data yields an R^2 value of 67.17% which essentially means 67.17% of the data can be explained by the model.

Though this trend is to be taken with a pinch of salt - many factors are at play here, not all of which have been captured by the data. The analysis also does not take account of the severity of action taken.

Admins most/least likely to take action when dealing with a report

Spoiler (Click to View)

I thought it would be fun to see what admins are more likely to exercise their power and which aren't. On the Y axis you will find the probability of action being taken by that admin... I forgot to label the axis. I believe I used a cut off point of either 10 or 20 reports handled at a minimum to qualify for this metric.

Starting with those who are most trigger happy...





...and the friendlier ones




Again, it's important to view what is being presented with some scepticism as the severity of the punishment hasn't been accounted for and neither has the context of the report... Do some admins have a bias towards which reports they take on? Or is it simply that some admins are more/less lenient than others? It cannot be said.

But the data still provides a somewhat useful insight as to how you might fare when a particular admin takes on your report or a report against you.

Proportion of reports resulting in action being taken (overall)

Spoiler (Click to View)

This speaks for itself. The unconditional probability of some form of action being taken in a report is 80.98%. This didn't come as a surprise to me, did it you?

Top 10 reporters by volume

Spoiler (Click to View)

Self-explanatory.

Most and least successful reporters (top/bottom 10) of other users

Spoiler (Click to View)

(Please excuse the axis)

Note: Abnormality refers to the absolute difference between their probability of success (defined by action being taken) in reporting a user and the overall probability of action being taken (80.98% if you looked above). In this analysis I excluded users who had made less than 10 reports in total, otherwise you end up with a list of 10 users who've made 1 report each with a 100% success rate; this wouldn't provide any meaningful result. Hence, I've included the number of reports the user has made to give a better picture of how 'successful' they really are.

Most successful...



So there's a substantial pool of users whose reports have had a 100% success rate despite having made a large amount of reports. I found this surprising because I expected very few to have a 100% success rate after their 10th report.

...and the least successful users




If you are having trouble interpreting these graphs and tables, see this example: Given Thomas's 22 reports against other users, he's only successful 77.27% of the time which is 3.70% below the normal probability of success (where success means that action has been taken against the user he has reported).

Top 10 admins to respond to reports by volume

Spoiler (Click to View)

As expected, BP is right at the top.

Top 10 admins to respond to reports by speed

Spoiler (Click to View)

This was calculated by taking the average time it takes a report from the time it is created to the time a decision has been made by the admin who took the report on.

As can be seen, the majority of the admins who topped this metric are no longer in TC.



So for example, the average time it takes Byran to take a report on and close it is 8.16 hours (0.34 * 24).

The major downfall with this metric is it doesn't necessarily measure how quickly an admin deals with a report once they know about it, but instead measures how quickly they deal with and close a report as soon as it is created. A further weakness of this measurement is it doesn't necessarily reflect how well an admin handles a report; for example, some reports will take much longer than others if the reporter doesn't speak English and is taking days to understand that reports must be posted in a particular format.

I have a table and graph for the 'slowest 10' for anybody who's curious, but it's important to bear in mind the above comment.

Monthly/yearly average of report handling times (e.g. the time it takes for a report to be archived from the day it's created)

Spoiler (Click to View)

Apologies for not labeling the y axis again... In case it wasn't clear, it is in days.

Monthly;


It's quite erratic and I'm struggling to work out why there are large peaks and troughs, though 2011's high average is probably because of the small data set used for that year. Perhaps when I get more time, I will check this analysis again but I am fairly sure it is correct though.

Yearly;


The yearly average (excluding 2011 this time) gives a smooth parabolic like curve and smooths out a lot of the erratic jumps seen in the previous graph.

It would also appear that 2014 was the golden era for report handling times despite it being the year that had the most reports (see below). I find this quite peculiar.

Yearly Report Figures

Spoiler (Click to View)

It would appear the number of reports had been increasing steadily up until 2015 where a small drop of 60 was observed. If 2016 continues as its current rate, a further drop will be observed but this is before accounting for what seems to be a 'summer peak' (see below). A large 'increase' is seen between 2011 and 2012 but this is to be taken lightly; it is because only 2 months of data was available for 2011.

Testing monthly report numbers for normality

Spoiler (Click to View)

The purpose of this is to see whether the number of reports received monthly follows a normal distribution. Not only does the QQ plot imply the normal distribution is a fairly good fit for the data, the theoretical density overlaying the empirical density also suggests it is a good (but not perfect) fit. I suspected this would be the case, partly due to the central limit theorem.

Using this, we can obtain a basic 95% confidence interval for the number of reports received in any given month of [15, 92] which is a fairly large range. This range is mainly so wide because the standard deviation of the number of reports received per month is 19.8 whereas the mean is 53.4.

So to summarise, this essentially says there's a 95% chance the number of reports for any given month will fall between 15 and 92.

This thread wouldn't be complete without a couple of word clouds and so I've made word clouds for the reporting/original posts of the ~2900 report threads as well as a word cloud for the final post by the admin. Word clouds are a 'visual representation of text data' with the frequency of words represented by their size. For some reason the R package I used to produce the word cloud has trimmed a few of the letters from the ending of certain words, but it's still understandable.

Word cloud of the 'reporting' posts (ie the first post of a report thread outlining the reasons for the report, their username and so on):

Spoiler (Click to View)

Word cloud of the 'admin decision' posts (ie the last post of a report thread where a decision on the report is made by an admin):

Spoiler (Click to View)

That's it! I hope at least some parts of this post appealed to you. I scraped the data together using the lite mode of the forum for efficiency (if you can use the words 'efficiency' and 'scrape' in the same sentence), so if you'd like a copy of the dataset I've created do let me know.