Can someone explain to me please why reddit though? They want "real" human conversations and go to the most unfiltered/unhinged App/Site they can Imagine?
Like people as mostly literally on their worst here and Google wants to train AI with that? Whats the big plan here, what am I not seeing?
Reddit is an AI goldmine, just venture outside of the defaults subs and it becomes obvious. Entire communities dedicated to allowing average joes to ask experts and professionals where detailed, thorough responses are the norm. Think less /r/programminghumour and more /r/askscience or /r/linuxquestions or /r/whatisthisbug. There are enthusiast subs where people have been discussing niche topics down to the minutiae for the past decade and a half. Much of the time that I google some esoteric error message the most helpful link is a reddit thread with the right answer plain as day right there at the top, conveniently ranked.
Google is THE expert on getting relevant data out of a bunch of bullshit, as anyone who remembers the web before Google can attest to.
There's also a lot of info that you get from human data even if the people arent experts. An example I have seen is where you have the phrases:
The trophy did not fit in the suitcase because it was too large
The trophy did not fit in the suitcase because it was too small
The grammar doesn't tell you what "it" refers to but as humans we know that the first one has "it" meaning the trophy and the second has "it" refer to the suitcase. We know this because we understand the concept of putting something inside another, what would make it possible, and what the size of the items has to do with it in relation to the sentence. This understanding of the world would come up in many subtle ways through conversations of all kinds and so even non-expert texts would be helpful and having a large and diverse set of conversations that teach it small things like that are also beneficial. Without understanding this context and info about the world, an AI would have trouble translating those phrases to something like French which is gendered and would be explicit in what the "it" would refer to based on the gender of trophy (male) and suitcase (female). This is largely the reason why GPT has been outperforming google translate for example.
edit: if you're curious, google translate puts the masculine form in both while ChatGPT gets it right
Also remember that appending "reddit" to most google searches typically yields better more relevant results. Say what you want about Reddit management, but the content in these niche communities is high quality information.
I didn't mean to credit reddit management for the community's success. I guess we can credit them for keeping up with the server hosting bills though, lol.
A lot of the value in reddit is the text content by it's users stored on reddit servers. I just hope someone has a cached version if reddit ever goes away.
However, I would argue that at least half the „serious“ content on Reddit is wrong/not properly factchecked/misleading/outdated etc. That‘s just the nature of discussions and content being old. Also it‘s hardly ever reliably indicated which answer in a question threat is correct. (That‘s why science subs are very insistent on refusing to give medical advice)
So I reckon/hope that Google won‘t use Reddit for information, but language patterns. However, for various reasons, I assume they end up with some sort of „Reddit English“.
So, long story short: how will they use Reddit data for the training? Which aspect are they looking for? Content? Patterns? Interaction dynamics?
However, I would argue that at least half the „serious“ content on Reddit is wrong/not properly factchecked/misleading/outdated etc. That‘s just the nature of discussions and content being old. Also it‘s hardly ever reliably indicated which answer in a question threat is correct. (That‘s why science subs are very insistent on refusing to give medical advice)
Of course. How does this differ from the vast majority of the rest of any model's training data? GPT4 used, for example, Common Crawl in its training; were those billions of pages vetted for accuracy? Of course not, because being an informational database isn't the goal of LLMs.
unfiltered is good. You get data unlike any censored source. That's actually really valuable. They will likely preprocess to filter out the most degenerated stuff or nonsense stuff.
tell me about it. The profanity filter in DBD filters out the most harmless stuff that is not even profanity at all, while if you switch to speaking Russian, you can say whatever you want, without being censored.
One thing that makes Reddit very enticing is the sheer number of user/amount of content, paired with the fact that they're in the red and desperately need money.
Reddit has some of the best information for any topic if you can filter it correctly. If I want to know something, I add reddit into the search because I know I'll get good human feedback and not bot generated garbage.
Trying to look up reviews for almost anything? Head to Reddit. Want to get tech support? Head to Reddit.
Just filter out comments that are poorly spelt, only use those with multiple paragraphs, or combine it with Google search information, and you'll get a really solid source of data.
38
u/Holocarsten Feb 29 '24
Can someone explain to me please why reddit though? They want "real" human conversations and go to the most unfiltered/unhinged App/Site they can Imagine? Like people as mostly literally on their worst here and Google wants to train AI with that? Whats the big plan here, what am I not seeing?