This forum is for discussion related to the game.
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #6 (isolation #0 ) » Tue Nov 23, 2021 3:30 pm
Post
by Cabd » Tue Nov 23, 2021 3:30 pm
We've managed to track down all of the quick topics mentioned in public-facing areas of the site. I'm exporting the contents of each now.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #8 (isolation #1 ) » Tue Nov 23, 2021 4:13 pm
Post
by Cabd » Tue Nov 23, 2021 4:13 pm
Yes, although it's simple enough to filter that stuff out since they're going to be in table format.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #9 (isolation #2 ) » Wed Nov 24, 2021 3:53 am
Post
by Cabd » Wed Nov 24, 2021 3:53 am
Sample data from the first 5 QTs in the list:
The question at this point is what format to store it in. Right now, it's a big table that is shoved into Excel/PowerBI, but we could place it just about anywhere. Hosting is somewhere safe and accessible to all is important though.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #11 (isolation #3 ) » Wed Nov 24, 2021 5:47 am
Post
by Cabd » Wed Nov 24, 2021 5:47 am
If anyone has a PT that was NOT linked anywhere on site other than in private message, please let us know and we will manually preserve it if you provide the link.
Right now the data set we are preserving is any quick topic that was posted about in a publicly indexed portion of the site ONLY.
You can post it here, or PM myself and Bork.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #13 (isolation #4 ) » Wed Nov 24, 2021 11:27 am
Post
by Cabd » Wed Nov 24, 2021 11:27 am
Done..... Over 360k posts exported...
Now just gotta get them into a happy readable format.
User Egosearch is also working:
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #15 (isolation #5 ) » Wed Nov 24, 2021 11:57 am
Post
by Cabd » Wed Nov 24, 2021 11:57 am
The raw data for all publicly posted Quick topic links is now safely backed up in several places.
I've got it in CSV format, as well as in a PowerBI data set, and a copy of it in OneDrive, as well as my personal machine's hard drive and workplace hard drive.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #16 (isolation #6 ) » Wed Nov 24, 2021 12:00 pm
Post
by Cabd » Wed Nov 24, 2021 12:00 pm
I intend to do some auto-tagging to allow filtering of potential triggers.... Please PM or reply here if there are specific words/phrases that you'd like added to the list.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #17 (isolation #7 ) » Wed Nov 24, 2021 2:38 pm
Post
by Cabd » Wed Nov 24, 2021 2:38 pm
Getting some AMAZING statistical data out of this too by the by.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #19 (isolation #8 ) » Wed Nov 24, 2021 5:19 pm
Post
by Cabd » Wed Nov 24, 2021 5:19 pm
Still needs a little bit of work on catching some additional words/phrases but here's just my first stab at trigger-filtering:
And here's the basic filtering option, showing opt-out content filters:
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #25 (isolation #9 ) » Thu Nov 25, 2021 7:34 am
Post
by Cabd » Thu Nov 25, 2021 7:34 am
My query has a deduplication step so even if you submit one that we already have it won't hurt.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #28 (isolation #10 ) » Sat Nov 27, 2021 10:41 am
Post
by Cabd » Sat Nov 27, 2021 10:41 am
I will be pushing this data set live this evening.
I have been recommended to add a self-harm/suicide flag, which I have done so.
If anyone else has recommendations or suggestions for additional words of flags, please let me know.
Side note: I am extremely firm on including the L word under the racial tag, this debate was already had when we banned the word in the first place onsite. If that is something you don't like, you're welcome to pull your own copy of the data set and host it under your ideals instead.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #31 (isolation #11 ) » Sat Nov 27, 2021 12:58 pm
Post
by Cabd » Sat Nov 27, 2021 12:58 pm
Yes.
You literally have to *gasp* click the single opt-in checkbox at the top for each type of content that is hidden by default that you wish to see.
The actual underlying database is a raw scrape of every single QT webpage, one comment per row.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #33 (isolation #12 ) » Sat Nov 27, 2021 1:00 pm
Post
by Cabd » Sat Nov 27, 2021 1:00 pm
In post 19 , Cabd wrote: Still needs a little bit of work on catching some additional words/phrases but here's just my first stab at trigger-filtering:
And here's the basic filtering option, showing opt-out content filters:
Take a look at the second image in this post. See those check boxes up top? That simple.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #35 (isolation #13 ) » Sat Nov 27, 2021 1:55 pm
Post
by Cabd » Sat Nov 27, 2021 1:55 pm
My filter software limitations can go ~20 per tag.
Would you be willing to condense to the ones that would be most vital to include?
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #37 (isolation #14 ) » Mon Nov 29, 2021 6:38 am
Post
by Cabd » Mon Nov 29, 2021 6:38 am
Thank you.
With additional filters added, our new list of content is at:
Note that if a comment contains more one triggering aspect, it will lump it in with the largest existing group it can.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #39 (isolation #15 ) » Mon Nov 29, 2021 6:57 am
Post
by Cabd » Mon Nov 29, 2021 6:57 am
I can easily fix that, yeah, oops.
Thanks.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #41 (isolation #16 ) » Mon Nov 29, 2021 7:25 am
Post
by Cabd » Mon Nov 29, 2021 7:25 am
Data is now uploaded and being tested by a select few users. PM me if you wish to get access before we publish it to the entire site, to help check for typos and bugs.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Cabd
Cabd
QT Sniper
Cabd
QT Sniper QT Sniper
Posts: 15543 Joined: February 3, 2013
Post
Post #45 (isolation #17 ) » Mon Nov 29, 2021 4:32 pm
Post
by Cabd » Mon Nov 29, 2021 4:32 pm
viewtopic.php?f=1&t=88317
Version 1.0 is out, and further discussion should be placed there.
If other communities would like a copy of my code, they are welcome to it to pull their own data sets.
Show Have retired for good; Life is too busy to have time or energy for mafia. It was fun~
And then, a Miracle, a Dance Game and a flight of fancy struck, one more game into the abyss
Copyright © MafiaScum. All rights reserved.