Omnom: Self-hosted bookmarking with searchable, wysiwyg snapshots

Posted by mstef 5 days ago

Omnom: Self-hosted bookmarking with searchable, wysiwyg snapshots(omnom.zone)

a bit of background. this is a rewrite of omnom by asciimoo - the guy who also made searx. originally i wrote omnom, back in the days when del.icio.us was enshittified and gone down the drain. instead of complaining i was taking the opportunity to write a replacement that i would want, it became omnom, and for many years me and a bunch of users were very happy with it.

but the fast moving world of browser extensions made maintenance (for a non-commercial free software side project) too expensive, so the extension support became erratic, until it died completely. for a year or two i didn't use it.

but then asked asciimo if he could have a look a the extensions (since he was recently working on another extension, so had some experience), he looked at it, and rewrote the whole thing in go and fixed the extension. he even got support from nlnet/ngi0.

if you want to have your own self-hosted libre software bookmarking service for you and your community, give it a go, it's very simple, privacy respecting and most robust when it comes to snapshotting. asciimoo did a great job.

211 points | 59 commentspage 2

marcod 5 days ago|

> if you want to have your own self-hosted libre software bookmarking service for you and your community, give it a go

I haven't worked with go - where could I host that myself?

Could I do this with the free tier of https://cloud.google.com/run/docs/quickstarts/build-and-depl...

dewey 5 days ago||

I wish ArchiveBox had such a feature where you can archive “private” and logged in pages. I often had the case where I then realized I only archived some login page.

5evOX5hTZ9mYa9E 5 days ago||

I use Linkding for my bookmarking. It has the ability to snapshot the links by using SingleFile, which is super handy cause it produces a single .html file that you can send or save separately if you want.

To save logged in content, you can either pass it cookies.txt with the login cookies it needs, and it works pretty well. Alternatively, you can also use the SingleFile extension to snapshot what you want and upload it in place of the automated snapshot. This is also handy because the extension allows you to remove private data prior to screenshot, such as your name or username.

I personally have cookies in place for most common social media sites that need login (twitter, reddit), and if I need to snapshot something else occasionally, I do it manually and upload it to Linkding.

https://linkding.link/

https://linkding.link/archiving/

https://www.getsinglefile.com/

dewey 5 days ago||

ArchiveBox also supports SingleFile from what I remember, but I wish there was a extension to just pass everything (Browser context) to ArchiveBox. Having to keep cookies in sync with a second service is a bit of a hassle.

5evOX5hTZ9mYa9E 5 days ago||

If you read the Linkding archiving page, you can see that it is possible to set up SingleFile extension to auto-snapshot the page when the page is added, from the browser you're using at that moment. The advantage is that loing is handled implicitly, but a disadvantage is that this only works in the browser, and not for example when you use the Android app. But for me that's fine,

Also, it is possible to configure SingleFile extension to automatically push a snapshot to a REST Form API. You can even make it do so when you bookmarks something in your browser, to achive a flow where bookmarking something in the browser saves the page via SingleFile extension and gives it to ArchiveFile.

https://docs.archivebox.io/dev/Contents.html#api-reference

Also, look in the SingleFile extension options, under Destination.

dewey 5 days ago||

That sounds like that does the job, thanks! I’ll try to set this up.

kapnobatairza 5 days ago||

You can pass cookies or Chrome user data into ArchiveBox, but you do risk account suspensions if your account gets flagged for scraping.

bosse 5 days ago||

I've been using Wallabag for articles I want to deal with later, with the Frigoligo app on iOS for offline reading. For bookmarks, I've been using the Floccus extension with the Nextcloud Bookmarks app.

I'm missing the social aspect of bookmarks on del.icio.us though, so I'll check out Omnom and the other mentions in this thread of what people are using.

dpcx 4 days ago||

Is there anything like omnom or karakeep that allows for storing things without having to manually bookmark or add them to the system? I want any URL I hit to have its content sent to my service so that I can search it later.

fullsort 4 days ago||

Take a look at our creation which is free. Also, you may try Full Sort's demo without sign up to see if you like the interface and features. https://fullsort.com

rcarmo 4 days ago||

I use archivebox for this. No requirement for an extension, just a simple bookmarklet.

bravura 4 days ago||

Question: Does this use single file or something similar? I've basically been dying for an easy to use local bookmarking service that uses singlefile?

Second question: How easy is it to crawl the local store and apply LLMs or RAG?

vandyswa 5 days ago||

I have a static web page with a table of A href's. Source controlled under git. Problem solved.

mstef 5 days ago|

sure, for bookmarking that's fine. but it does not help against linkrot, or censorship, or pages changing with time.

quantadev 5 days ago||

I agree. Saving web-links is a solved problem. Saving the CONTENT afaik is not solved and I've wanted a solution for YEARS, as well as the rest of humanity has needed this (including those who don't even know they need it, haha!).

Closest thing would be to "print" to PDF but that always creates a trainwreck in the PDF and is never good enough for me. Looking forward to trying Omnom if it's open source.

asciimoo 5 days ago||

It's not only open source, it is free software. Take a look at https://github.com/asciimoo/omnom - suggestions/contributions are appreciated =)

quantadev 5 days ago||

That looks like a pretty heavy-weight solution, with a lot of complexity, and I don't mean that as a criticism at all. I'm not a 'go' developer myself. I've always wanted a pure JS solution (as a browser extension, maximum of 200 lines of code) that can capture the content of a web page (doing a virtual scroll to the bottom, to capture the whole page). Since there's no perfect way to translate HTML to PDF, my idea had always been to capture the IMAGE of the page (aside from capturing keywords for DB indexing which can be done separately just for 'search' support later on).

The fly in the ointment is of course the scrolling too, because some apps have "infinite" scrolling, and so in many SPAs there's literally no such thing as "The whole page". Anway, I haven't tried your app yet, because of not-JS and not-Small, reasons, but I'm just sharing my perspective on this topic. Thanks for sharing your project!

Ametrin 3 days ago||

I recently released a Chrome extension that converts webpages to PDF. It's free, but you need to register to get a key. Unfortunately, this solution isn't client-side JavaScript; I'm using an API underneath. To be honest, I mainly created it to promote the API, but if it's useful for people, I might develop it further. Perhaps it could be useful to you in some way. I don't know your requirements, but maybe with this base in the form of this extension, it wouldn't be difficult to add something that meets your expectations, let me know. However, if you want to export a PDF from Ahrefs, for example, I'm afraid that might not be possible; currently, only basic authentication is supported. Unless maybe I could add an option like in my API to pass JavaScript code, but I also doubt that would work because Ahrefs probably has some bot protection.

edit: i forgot the link https://chromewebstore.google.com/detail/pdfbolt-web-to-pdf/...

quantadev 3 days ago||

Thanks for sharing that. Looks pretty nice!

submeta 5 days ago|

I created something like that for my spouse and myself. An app that creates an archive.ph url, extracts full text and generates summaries via an llm. I‘d open source it, but as I also extract paid articles via archive.ph, I think it wouldn’t be ethical to publish it.

mstef 5 days ago|

the difference is, that archive.ph snapshots something in headless. omnom snapshots the exact same state that your browser is displaying you. so if there is js interactions that change the dom, those will be snapshotted, unlike with archive.ph.

also lets not forget that archive.ph wraps everything in their own frame and has their own way of mangling the result. not in a bad way, it's just not the original as it would have been rendered in your browser.

viraptor 5 days ago||

Archive.ph is often used, because the "js interactions that change the dom" is typically a paywall. And you won't want it.

mstef 5 days ago|||

omnom is for snapshotting, not for circumventing paywalls. i'm merely comparing the snapshot feature of the two projects. circumventing paywalls is out of scope.

your bookmarks will never linkrot away.

mstef 5 days ago|||

btw it is perfectly fine to circumvent a paywall with archive.ph and then to snapshot it with omnon so your bookmark never linkrots away. also when i say "js manipulation" i also mean stuff like captchas, or dynamic documents that you change by interacting with it, or even private services like e.g. rocket chat hidden behind some barrier like http auth, or private vpn. archive.ph will never have access to what your browser might have access to.

More comments...