Quicktopic shutting down

[Cabd]

We've managed to track down all of the quick topics mentioned in public-facing areas of the site. I'm exporting the contents of each now.

[Cabd]

Yes, although it's simple enough to filter that stuff out since they're going to be in table format.

[Cabd]

Sample data from the first 5 QTs in the list:





The question at this point is what format to store it in. Right now, it's a big table that is shoved into Excel/PowerBI, but we could place it just about anywhere. Hosting is somewhere safe and accessible to all is important though.

[Cabd]

If anyone has a PT that was NOT linked anywhere on site other than in private message, please let us know and we will manually preserve it if you provide the link.

Right now the data set we are preserving is any quick topic that was posted about in a publicly indexed portion of the site ONLY.

You can post it here, or PM myself and Bork.

[Cabd]

Done..... Over 360k posts exported...



Now just gotta get them into a happy readable format.

User Egosearch is also working:

[Cabd]

The raw data for all publicly posted Quick topic links is now safely backed up in several places.

I've got it in CSV format, as well as in a PowerBI data set, and a copy of it in OneDrive, as well as my personal machine's hard drive and workplace hard drive.

[Cabd]

I intend to do some auto-tagging to allow filtering of potential triggers.... Please PM or reply here if there are specific words/phrases that you'd like added to the list.

Spoiler: Example for Filtering the L Word

[Cabd]

Getting some AMAZING statistical data out of this too by the by.

[Cabd]

Still needs a little bit of work on catching some additional words/phrases but here's just my first stab at trigger-filtering:





And here's the basic filtering option, showing opt-out content filters:

[Cabd]

My query has a deduplication step so even if you submit one that we already have it won't hurt.

[Cabd]

I will be pushing this data set live this evening.

I have been recommended to add a self-harm/suicide flag, which I have done so.

If anyone else has recommendations or suggestions for additional words of flags, please let me know.

Side note: I am extremely firm on including the L word under the racial tag, this debate was already had when we banned the word in the first place onsite. If that is something you don't like, you're welcome to pull your own copy of the data set and host it under your ideals instead.

[Cabd]

Are filtered posts still in this archive at all?

Yes.


You literally have to *gasp* click the single opt-in checkbox at the top for each type of content that is hidden by default that you wish to see.

The actual underlying database is a raw scrape of every single QT webpage, one comment per row.

[Cabd]

Still needs a little bit of work on catching some additional words/phrases but here's just my first stab at trigger-filtering:





And here's the basic filtering option, showing opt-out content filters:

Take a look at the second image in this post. See those check boxes up top? That simple.

[Cabd]

My filter software limitations can go ~20 per tag.

Would you be willing to condense to the ones that would be most vital to include?

[Cabd]

Thank you.

With additional filters added, our new list of content is at:



Note that if a comment contains more one triggering aspect, it will lump it in with the largest existing group it can.

[Cabd]

I can easily fix that, yeah, oops.

Thanks.

[Cabd]

Data is now uploaded and being tested by a select few users. PM me if you wish to get access before we publish it to the entire site, to help check for typos and bugs.

[Cabd]

viewtopic.php?f=1&t=88317

Version 1.0 is out, and further discussion should be placed there.

If other communities would like a copy of my code, they are welcome to it to pull their own data sets.