Bad pelicans are in the training set because it's read his blog post. Including a good pelican in midtraining wouldn't help the problem because you'd just produce that every time.
It is not a sponsored article and he writes one of these every time a new model releases. Why would a professor at Wharton need to write sponsored Substack articles.
> Super catchy names like Opus, Mythos and Fable trying to get you to think that these software products are actually super-human life changing experiences.
They're originally named after the blends at a nearby coffee shop.
I've noticed nobody at HN knows what "marketing" is or how to do it. It's not just naming things and being evil and cynical is not the most successful method.
…also frontier models are a superhuman life changing experience. If they aren't, what possibly could be?
Scraping the internet isn't a copyright violation. Using it for LLM training is much more transformative than Google and Internet Archive, which are legal.
Except it ignores the entire premise of copyright which is to protect incentives to create original work, which Google does not destroy and which LLMs (very loudly and proudly) try to do.
There are several components of the Fair Use test, "transformation" is just one of them. The most important dimension is the effect on the market, i.e. the effect on incentives.
You probably shouldn't base your legal analysis on pithy internet comments regardless of how succinct or agreeable they are to you.
Your right, scraping is legally protected. It's reproducing verbatim text that's a violation, which is why LLMs still clumsily refuse to produce song lyrics. They are capable of copyright violations and have to be 'aligned' not to get their providers sued.
Verbatim reproduction is neither necessary nor sufficient to create a copyright violation.
"Copyright violation" is what we call the set of things that destroy the incentive for people to create original work by unduly benefitting from someone else's original work.
I've seen a few studies about it. If i remeber correctly, basically making them talk like a caveman increases the information density of every token and decreases the chance that they would allucinate.
As sources this is the one i found but i'm sure there are others:
> Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models
> Text Compression as a Proxy for LLM Reasoning Efficiency
reply