Posted by arnemunthekaas 20 hours ago
Off-topic, but what is up with the increased use of the phrase "in a minute", presumably to mean "in a long time", lately?
I've only started encountering it in the past year.
Did it get popularized by some celebrity, tv show, influencers, etc?
https://trends.google.com/explore?q=I%27ve%20seen%20in%20a%2...
They took a well-defined unit of time, which is relatively short, and made it mean “some unknown but very long period of time”.
So frustrating. /oldmanyellsatcloud
It always seems kind of silly to me to throw everything at an LLM. I know they’re huge and can automatically handle a huge number of tasks but something in me finds it wasteful when we could be creating easily trainable, cheap to run bespoke models for a lot of stuff
From my testing of open weights LLMs with audio support, they basically are only trained to recognize audio as an alternative to text input, they treat audio as basically equivalent to a transcript, and can't recognize or distinguish things like music, accents, background sounds, etc.
So they're only really good for transcribing or summarizing or using audio input in place of text input for prompts, but not anything that requires distinguishing any information about the audio that would not be present in a transcript.
It can be tempting to try to use an LLM for a variety of tasks; kind of the whole thing about an LLM is that you don't have to do a separate complex training run for every task, but can just provide instructions in natural language. But it only works as far as what the training data covers, if the training basically always treated audio and a text transcript as equivalent, the model has nothing causing it to learn other relevant features of the audio. If there's enough bird call identification in the training data of an LLM, it might be able to do that, but I think multimodal training data tends to be much more limited than the text training corpus
My current approach, not yet validated, is trying to generate training data from masterclass recordings on Youtube, and then fine tuning MOSS-audio on a bunch of those. But I'm interested if there are better models, or large training sets I don't know about.
When I first heard of it (through a community member in discord), I wanted to see if I could replicate it on FrameOS (my software for running anything on an e-ink panel), and sure enough, I got quite close: https://scenes.frameos.net/s/bird-field-journal
This is obviously nowhere as complicated as OP's project, but I thought I'd just share that your project inspired me to build something similar too.
I just got and setup a BTLE e-ink driver the other day and according to my calculations it will last _years_ on a single charge (2000mAh) even with multiple refreshes per day (crazy compared to the Wifi ones). Year+ lifetime completely changes the calculus for where to put these IMHO.
I'm considering designing a multiple-screen "art" piece to hang on my wall and playing around with various ideas (on both content and design).
LLMs + 3D Printer + e-ink is my new favorite hobby.
Screen: 7.5" Monochrome eInk / ePaper Display with 800x480 Pixels (https://www.seeedstudio.com/7-5-Monochrome-ePaper-Display-wi...)
BTLE board: XIAO ePaper Display Board(nRF52840) - EN05 (https://www.seeedstudio.com/XIAO-ePaper-Display-Board-nRF528...)
Battery: 3.7V 2000mAh - (https://www.amazon.com/dp/B0FR9GH966)
Picture of it all assembled (no case yet): https://cs.joshstrange.com/zJFvGGPB
The image rendered to the screen is coming from a little service I wrote that renders a book quote it pulls from BookOrbit which syncs with KOReader (annotations, progress, etc). My plan is to set up one of these one each bookshelf/series rotating quotes I highlighted from the series.
Here are 2 ESP32-based screens I have, the one on the left is a an all-in-one [1] and the one on the right I printed the case from the TRMNL DIY kit from SeeedStudio [2]: https://cs.joshstrange.com/knk4Ns2h
Right now only the BTLE one is running OpenDisplay (the other two are semi-managed by HomeAssistant ESPHome) and all the devices are in HA for controlling them, though the ESP32's all deep-sleep for saving battery so I can can't "push" stuff to them like I can the BTLE one.
[0] https://opendisplay.org/index.html#battery
[1] https://www.seeedstudio.com/reTerminal-E1001-p-6534.html
[2] https://www.seeedstudio.com/TRMNL-7-5-Inch-OG-DIY-Kit-p-6481...
The BTLE boards have just crazy higher battery time with no need for deep sleep which is why I'm moving in that direction.
~7.5" is the size I've been buying so far, the larger e-ink screens get pricey but I can't wait for a calendar-sized one to come down in price such that I can frame it and hang it on the wall. Lastly I'd avoid the color e-ink screens unless you know you need color, won't need to refresh it often, and it's not in your eye line. Color refreshes take way longer (~30s in my experience for a 5-6 color screen) and the flashing it has to do is obnoxious. Monocolor displays take <2s normally and a single flash. I got a color screen first because "why not?" and I got color, but the refresh was too annoying so now it refreshes once a day in a place that I'm not when it refreshes so I don't have to see it.
I also recommend you check out OpenDisplay for what you will run on the device: https://opendisplay.org/index.html
[0] https://www.seeedstudio.com/reTerminal-E1001-p-6534.html
So reiterate, with a 2000mAh lipo battery I should be able to drive the board _and_ the screen for over 3 years if I only refresh it at most once every 4 hours (with "push" capabilities, I don't have to deep-sleep the board to get that battery life). At least, that's what the calculator says, I _just_ got my hardware to play with and so I can't speak from experience. Unfortunately my board is not charging the battery so I need to get a replacement which will probably take a week or two to get here.
And then you're stuck with a device that can only be communicated with when it chooses to wake up, whereas with BLE it seems you can listen for a remote OTA wake much more cheaply.
It seems IP over Avian Carriers is finally within reach!
The only thing I wish to be added is a short video showing the actual frame in action to express motion and sound so I can better feel it. While the provided image (hero.jpg) gives me a sense of the product, I just can't smell it yet. I visited https://fugleramme.arnegiacomo.dev/ hoping for a live feed but it seems just a static image.
Kudos for the author and I hope we can see the next iteration!
_______________
1. I made an open-source laptop from scratch (https://news.ycombinator.com/item?id=42797260)
2. I replaced a $120k bowling center system with $1,600 in ESP32s (https://news.ycombinator.com/item?id=48968606)
3. A retro video game console I've been working on in my free time (https://news.ycombinator.com/item?id=19393279)
Then have the screen look like a biologists notebook where he sketches the bird and writes a few notes about the time and its behavior.