Statistical analysis of the reports archive

[Tingle]

I thought it would be interesting to analyse the reports from the archive section to see what hidden gems can be found from the data... Some of it's surprising, some of it is nearly on par with my predictions before conducting the analysis and the rest may just be coincidental!

2963 reports were analysed from 3rd June 2016 to the 3rd October 2011, though in some of the below graphs June 2016 and/or the entirety of 2016 and/or the entirety of 2011 were left out so as to not distort the overall picture (as only partial data was available for 2011 and we're only half way through 2016). I've indicated where this has had an effect.

You will find some ex-TC names in the data, I chose not to remove them for simplicity.

Spoilers have been used so you can choose what you want to see.

Relationship between the probability of an admin taking action and the number of posts in the thread

Spoiler (Click to View)

Something I found particularly interesting when analysing the data is the relationship between the number of posts that occur in a thread and the chances of the admin dealing with the report taking action.

It appears that as the number of posts increase the chances of action being taken decrease almost linearly until a tipping point of 10 posts is reached, at which point the trend fails. Fitting a second order polynomial regression model to the data yields an R^2 value of 67.17% which essentially means 67.17% of the data can be explained by the model.

Though this trend is to be taken with a pinch of salt - many factors are at play here, not all of which have been captured by the data. The analysis also does not take account of the severity of action taken.

Admins most/least likely to take action when dealing with a report

Spoiler (Click to View)

I thought it would be fun to see what admins are more likely to exercise their power and which aren't. On the Y axis you will find the probability of action being taken by that admin... I forgot to label the axis. I believe I used a cut off point of either 10 or 20 reports handled at a minimum to qualify for this metric.

Starting with those who are most trigger happy...





...and the friendlier ones




Again, it's important to view what is being presented with some scepticism as the severity of the punishment hasn't been accounted for and neither has the context of the report... Do some admins have a bias towards which reports they take on? Or is it simply that some admins are more/less lenient than others? It cannot be said.

But the data still provides a somewhat useful insight as to how you might fare when a particular admin takes on your report or a report against you.

Proportion of reports resulting in action being taken (overall)

Spoiler (Click to View)

This speaks for itself. The unconditional probability of some form of action being taken in a report is 80.98%. This didn't come as a surprise to me, did it you?

Top 10 reporters by volume

Spoiler (Click to View)

Self-explanatory.

Most and least successful reporters (top/bottom 10) of other users

Spoiler (Click to View)

(Please excuse the axis)

Note: Abnormality refers to the absolute difference between their probability of success (defined by action being taken) in reporting a user and the overall probability of action being taken (80.98% if you looked above). In this analysis I excluded users who had made less than 10 reports in total, otherwise you end up with a list of 10 users who've made 1 report each with a 100% success rate; this wouldn't provide any meaningful result. Hence, I've included the number of reports the user has made to give a better picture of how 'successful' they really are.

Most successful...



So there's a substantial pool of users whose reports have had a 100% success rate despite having made a large amount of reports. I found this surprising because I expected very few to have a 100% success rate after their 10th report.

...and the least successful users




If you are having trouble interpreting these graphs and tables, see this example: Given Thomas's 22 reports against other users, he's only successful 77.27% of the time which is 3.70% below the normal probability of success (where success means that action has been taken against the user he has reported).

Top 10 admins to respond to reports by volume

Spoiler (Click to View)

As expected, BP is right at the top.

Top 10 admins to respond to reports by speed

Spoiler (Click to View)

This was calculated by taking the average time it takes a report from the time it is created to the time a decision has been made by the admin who took the report on.

As can be seen, the majority of the admins who topped this metric are no longer in TC.



So for example, the average time it takes Byran to take a report on and close it is 8.16 hours (0.34 * 24).

The major downfall with this metric is it doesn't necessarily measure how quickly an admin deals with a report once they know about it, but instead measures how quickly they deal with and close a report as soon as it is created. A further weakness of this measurement is it doesn't necessarily reflect how well an admin handles a report; for example, some reports will take much longer than others if the reporter doesn't speak English and is taking days to understand that reports must be posted in a particular format.

I have a table and graph for the 'slowest 10' for anybody who's curious, but it's important to bear in mind the above comment.

Monthly/yearly average of report handling times (e.g. the time it takes for a report to be archived from the day it's created)

Spoiler (Click to View)

Apologies for not labeling the y axis again... In case it wasn't clear, it is in days.

Monthly;


It's quite erratic and I'm struggling to work out why there are large peaks and troughs, though 2011's high average is probably because of the small data set used for that year. Perhaps when I get more time, I will check this analysis again but I am fairly sure it is correct though.

Yearly;


The yearly average (excluding 2011 this time) gives a smooth parabolic like curve and smooths out a lot of the erratic jumps seen in the previous graph.

It would also appear that 2014 was the golden era for report handling times despite it being the year that had the most reports (see below). I find this quite peculiar.

Yearly Report Figures

Spoiler (Click to View)

It would appear the number of reports had been increasing steadily up until 2015 where a small drop of 60 was observed. If 2016 continues as its current rate, a further drop will be observed but this is before accounting for what seems to be a 'summer peak' (see below). A large 'increase' is seen between 2011 and 2012 but this is to be taken lightly; it is because only 2 months of data was available for 2011.

Testing monthly report numbers for normality

Spoiler (Click to View)

The purpose of this is to see whether the number of reports received monthly follows a normal distribution. Not only does the QQ plot imply the normal distribution is a fairly good fit for the data, the theoretical density overlaying the empirical density also suggests it is a good (but not perfect) fit. I suspected this would be the case, partly due to the central limit theorem.

Using this, we can obtain a basic 95% confidence interval for the number of reports received in any given month of [15, 92] which is a fairly large range. This range is mainly so wide because the standard deviation of the number of reports received per month is 19.8 whereas the mean is 53.4.

So to summarise, this essentially says there's a 95% chance the number of reports for any given month will fall between 15 and 92.

This thread wouldn't be complete without a couple of word clouds and so I've made word clouds for the reporting/original posts of the ~2900 report threads as well as a word cloud for the final post by the admin. Word clouds are a 'visual representation of text data' with the frequency of words represented by their size. For some reason the R package I used to produce the word cloud has trimmed a few of the letters from the ending of certain words, but it's still understandable.

Word cloud of the 'reporting' posts (ie the first post of a report thread outlining the reasons for the report, their username and so on):

Spoiler (Click to View)

Word cloud of the 'admin decision' posts (ie the last post of a report thread where a decision on the report is made by an admin):

Spoiler (Click to View)

That's it! I hope at least some parts of this post appealed to you. I scraped the data together using the lite mode of the forum for efficiency (if you can use the words 'efficiency' and 'scrape' in the same sentence), so if you'd like a copy of the dataset I've created do let me know.

[Adorable]

Interesting stuff. But how long did it take you to write down all of that and scan data from the user reporting section? ^_^

[[TC]►Frozen◄]

Looks nice, very interesting to read

[Barney [CA]]

"[You smart .I; appreciate that.]™" - [GLOW]Antonio, 2016.

You'll apply for TC? Just curious. I like statistics, great stuff. Interesting would have also been the amount of reports depending on the track. KY and SO(followed by BW) might have the most would be my assumption, but guess thats hard to figure with the old rotationshedules not availiable. But might be an idea for 2016 only.

(2016-06-06 03:26)Tingle Wrote:  Do some admins have a bias towards which reports they take on? Or is it simply that some admins are more/less lenient than others?

That has been on my mind for quite some time. I guess some admins prefer handling "more simple" reports. Sometimes you can witness longer reports(or by/towards certain users) being untouched for several days, while newer reports get handled quickly. Also it MIGHT be that certain topics are thought to be handled by certain people. E.g. cop-related reports used to be handled often by Saph, now by Connor(and Luke). Can only guess if this is a personal preference of the admins, or something like an internal guideline. Maybe just coincidence or im just wrong.

Although no major surprises(some exceptions) very interesting imho. gj

[[TC]►Carl◄]

Mr.AD hasn't even been in the community that long and he's not far off the top reporter.. gg.

Some nice data though was interesting to read. Also, I'm sure sure how exactly it works but I remember being told that certain admins deal with certain reports based on who's involved, the age of the report etc.

[Dan]

As somebody who studied A-level Maths (Core & Stats) as well as completed a full Geographical field work which involved going in to similar detail about a real-life topic, this was actually really interesting to read. It was also quite satisfying seeing something from education being applied to a gaming background.

The last word cloud particularly amused me, with the words "ban" and "warning" coming up big lol.

Well done, and thank you.

[[TC]»Pete«]

I see you're putting what you're learning in your degree to good use!

Mr. AD, stop reporting people

I'll take a copy of the dataset if it's available, Tingle.

[[TC] Connor]

Very in-depth Analysis, Tingle; I'm impressed!

By any chance, did you do Psychology for A Level? Just asking because I had to do Statistical Analysis in my Psychology spec (i.e. Statistical/Social Definitions of Abnormality - which you've outlined here). I also noticed your pretty well laid out weaknesses of the results, another method we undertook when evaluating our data in our practicals.

Very interesting read, and good to see I'm on the up for reports handled.

[Some1]

Very interesting, thanks for putting your time into this!
Pity that the data from 2008-2011 is not available, but 5 years is already a lot

[Dino]

Interesting stuff, thanks for sharing.

[Razz]

Great work Tingle! Very interesting data. Just refreshed some things I just learnt in my first year

[SO•Yukine]

Nice work, interesting to read, nice analysis!

Spoiler (Click to View)

[[TC]►Carl◄]

Also, for those in the CSGO scene, Tingle is probably reminding you of this person.

Spoiler (Click to View)

[[TC]►BP◄]

Good effort, interesting stuff.

[[TC]►Dizzle◄]

This is cool....


I dont maths at all.

[Tingle]

(2016-06-06 06:46)Mr.AD Wrote:  Interesting stuff. But how long did it take you to write down all of that and scan data from the user reporting section? ^_^

Overall (including writing the original post) I probably spent 8 or 9 hours on this spread over 3 days.

(2016-06-06 07:56)Barney Wrote:  "[You smart .I; appreciate that.]™" - [GLOW]Antonio, 2016.
You'll apply for TC?

Antonio or DJ Khaled? And no, not this year - perhaps some other time.

(2016-06-06 11:05)Connor Wrote:  By any chance, did you do Psychology for A Level? Just asking because I had to do Statistical Analysis in my Psychology spec (i.e. Statistical/Social Definitions of Abnormality - which you've outlined here). I also noticed your pretty well laid out weaknesses of the results, another method we undertook when evaluating our data in our practicals.

Unfortunately I didn't do it as one of my A Levels, but I come from a statistics background in that my degree is heavily centered around stats. But from what I remember from friends you do a fair bit of statistics in psychology at AS/A2.

Outlining the weaknesses again came from my degree because we spend half the time learning to criticise what we've learnt because there are inherent weaknesses in the methods and techniques used and it's important to understand them. From what I remember from GCSE psychology though, there was a lot of evaluation marks to be had where you had to outline the flaws of a study.

(2016-06-06 11:04)Pete Wrote:  I see you're putting what you're learning in your degree to good use!

Mr. AD, stop reporting people

I'll take a copy of the dataset if it's available, Tingle.

PM'd.

[Dino]

Top 10 of players being reported would have been quite interesting too together with their outcoming, how often actions were taken and how often not.

[Titan]

n1 tingle

[[TC]►Ras◄]

This is quite interesting information. Nice work!

[SO•jones]

Impressive!

Really amazed for the effort you have been done to do this!