Top
Best
New

Posted by twapi 17 hours ago

Darkbloom – Private inference on idle Macs(darkbloom.dev)
452 points | 220 commentspage 2
frankfrank13 6 hours ago|
This is one of those ideas I think makes perfect sense, but requires so much operational change for the entire stack, that it would be very difficult to scale:

- Convincing labs to run distributed, burst-y inference

- Convincing people to run their Mac all day, hoping to make a little profit

- Convincing users to trust a distributed network of un-trusted devices

I had a similar idea, pre-AI, just for compute in general. But solving even 1 of those 3 (swap AI lab for managed-compute-type-company, eg Supabase, Vercel) is nearly impossible.

jzig 4 hours ago||
[fix: remove hardcoded API_KEYS ](https://github.com/Layr-Labs/d-inference/pull/39/changes)
creamyhorror 4 hours ago|
oh boooy, it's a benchmarking script, but still...
poorman 3 hours ago||
As one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.
icedrift 46 minutes ago||
Are you actually seeing demand?
qurren 3 hours ago||
> You can get a 400 watt solar panel on Amazon for $300.

Too expensive. It's probably producing 200 watts average for 8 hours a day. That's 1600 watt hours, which is about $1.60 at PG&E prices. That would take 187 days to recoup the cost of just the panel.

If you include installation costs and "what PG&E steals if you wire it to the same grid" it's probably more like 4x that, which is too long.

Tell me when we can have 400 watt solar panels for $50. Stupid capitalism literally forces solar panel prices to make it unprofitable.

People should never have to take out loans for solar. Solar should be subsidized and forced by the government to be so cheap that it repays for its cost within a month. Then we're talking. Most things I buy to save money, I expect them to repay within a month. Maybe 2 months max.

dgacmu 2 hours ago|||
You can get them used for that price, and new for $107 if you buy qty 10+. See signature solar as one example.

Installation costs and inverters not included, however.

poorman 3 hours ago|||
If you are worried about a $300 solar panel you are not going to like the cost of a Mac Studio M3 Ultra 512 GB! haha
qurren 3 hours ago||
I'm not "worried" about that cost, I would rather just pay PG&E electricity if the solar panel cost $300.

Just pointing out why capitalism + solar is a failure. Capitalism reprices the good thing to be equally expensive to the bad thing, so that nobody buys the good thing anymore.

autodidacticon 2 hours ago||
bittensor has something to say about this
TuringNYC 17 hours ago||
I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute. Any suggestions from anyone? We currently run ollama but manually manage the pools
damezumari 17 hours ago||
https://github.com/exo-explore/exo
zozbot234 13 hours ago|||
If you set CPUSchedulingPolicy=idle Nice=19 IOSchedulingClass=idle in the ollama server configuration it should run in the background with lowest priority.
utopiah 14 hours ago||
Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.
TuringNYC 6 hours ago||
It is much more work because for many workloads you have geographic ringfencing and cannot send it out to the cloud
haspok 11 hours ago||
Having strong SETI@Home vibes from 25 years ago, except of course, this is not for the greater good of humanity, but a for-profit project.

Problem is, from a technical point of view, what kind of made sense back then (most people running desktops, fans always on, energy saving minimal) is kind of stupid today (even if your laptop has no fan, would you want it to be always generating heat?)...

I definitely want my laptops to be cool, quiet and idle most of the time.

kamranjon 9 hours ago||
My m4 max mbp with 128gb of memory is constantly training 24/7 on weekends- it’s why I bought the thing.
vorticalbox 11 hours ago||
I some times play about with local models via ollama/comfyui and more recently ace-step to generate music.

This is short bursts of heat 5-10 m during the render I would not be happy with that for multiple hours a day. I am sure that would have a negative effect on battery health.

pants2 16 hours ago||
You might not even know it as a user but the payment/distribution here is all built on crypto+stablecoins. This is a great use case for it.
rvz 16 hours ago|
Good. Another great non-speculative use-case for crypto and stablecoins.
kennywinker 16 hours ago||
Amazing! Let me see, doing the math r/n… carry the one, yup that makes the total number of non-speculative uses for crypto and stablecoin: 1

;P

stuxnet79 16 hours ago||
So basically ... Pied Piper.
JaggerJo 15 hours ago|
finally!
auslegung 5 hours ago||
How can one do this safely? If I create a new, non-sudo user, can I install the MDM profile only for that user? I don't understand how this all works obviously so maybe this is a very dumb question
NiloCK 13 hours ago|
Interesting to see an offering with this heritage [1] proposing flat earnings rates for inference operators here, rather than trying to sell a dynamic marketplace where operators compete on price in real-time.

Right now the dashboards show 78 providers online, but someone in-thread here said that they spun one up and got no requests. Surely someone would be willing to beat the posted rate and swallow up the demand?

I expect this is a migration target, but a tactical omission from V1 comms both for legitimate legibility reasons (I can sell x for y is easier to parse than 'I can participate in a marketplace') and slightly illegitimate legibility reasons (obscuring likely future price collapse).

Still - neat project that I hope does well.

[1] Layer Labs, formerly EigenLayer, is company built around a protocol to abstract and recycle economic security guarantees from Ethereum proof of stake.

Inferlane 12 hours ago|
[dead]
More comments...