Posted by karpathy 1 day ago
https://hackertube.org/download/hackertube-data.zip
"YouTube's own metadata (descriptions, view counts, durations, publish dates) can't be redistributed under YouTube's API terms. Every record has the video id, so you can fetch it with a free YouTube Data API key in about 130 requests. DATA.md has step-by-step instructions for you (5 minutes in Google Cloud) and a ready-to-run script for your agent."
The early internet (content) was often comprised of textfiles containing URLs and descriptions (metadata) about what could be found at those locations
Would be interesting to know what sort of damages YouTube would allege if someone violated the terms of service that attempt to restrict disseminating such information, e.g., metadata about other peoples' videos
The success of a YouTube copyright infringement claim seems dubious
I have been shouting that YT, the "video museum" is unmissable for cultivation in the current days ever since this obvious fact met the will of pitiful entities to bar it to minors (unbelievable) and impede it to adults (through privacy or practically dubious constraints). Karpathy has proven it once more.
Questions and suggestions:
-- Does it also contain submissions? What we believed were the most important ones are at https://news.ycombinator.com/from?site=youtube.com , more than in the comments
-- Are you also processing 'youtu.be' links? We use them for brevity
-- The videos in HackerTube do not seem to be all we linked to at HN. I get only 6 months old videos as the closest in time, and I verified many are missing. The filter applied to select the links on HN is too restrictive.
-- Put the link on the "channel" anchor - if a video is hosted by that channel the channel is probably interesting. Even much, much better if you create "Channel" pages on HackerTube, to make them better browsable (for example, on YT it is extremely difficult to browse them - "see the oldest", "see all" - given the lack of pagination). (To get a good YT channel browser would be of immense value.)
-- Another paramount feature on YT that can use much better usability are transcripts - they are available but they must be better presented.
-- Maybe you could use the "three frames" at https://i.ytimg.com/vi/<videoid>/hq<1|2|3>.jpg
-- How often will the DB on hackertube be synced with the HN content (if one posts a link to YT on HN, how long before it will appear on HackerTube)? A clean way to add new material to HackerTube is sought.
-- What could happen to embedded YT links when access restrictions will be mandated by legislations?
-- There should be RSS on HackerTube. The correct formula should be, I believe, serve "a minimum of 10 and a maximum as the amount of new records of the past 48 hours".
-- Given the selected amount of videos, it could be possible to also index the videos by embedding. (I have just finished in the past few hours my own engine for embeddings operation - search, similarity, recommendation - over my daily updated ~million RSS records from YT: it can be done efficiently with minimal space allocation.)
-- Some posts here just gave me this later wilder suggestion: what if you could have an LLM in the internals of HackerTube that recognized and fixed (through descriptions and transcripts) the vague clickbait - e.g. translating titles like "This will be the solution to all the problems" into an additional "[that actually was: adding sugar to the soil may help the growth of basil]"...
https://www.trustmebro.reviews/r/hackernews
"47,699 products, places and tools ranked from 13,576,547 opinions."
Opinions extracted using local models running on 3090s in my basement. If someone wants to take it over, you can mail me.
The travel opinions on a map are pretty neat: https://www.trustmebro.reviews/c/travel?view=map