Top
Best
New

Posted by tanelpoder 19 hours ago

Use your Nvidia GPU's VRAM as swap space on Linux(github.com)
418 points | 107 commentspage 2
ProllyInfamous 5 hours ago|
Why does my Apple Sillicon Mac with 32gb of RAM use (or even create?!) a swapfile, when 20gb is still unused/"free"?

Why can I not just enter a simple command to entirely-disable swapfile, like with Linux's:

>>>>swapoff -a

Seems kind of silly, unless the point is intentionally to wear-down the SSD's lifespan.

Having a GUI swapfile-disable system preference would be awesome. It would also be awesome if Apple finally abandoned this system settings/layout "phase" – it's still word-salad (compared to decades of preference panes).

#Apple #Feedback #swapfile

netbsdusers 3 hours ago|
The principle of a paging system is that main memory is just a cache for secondary memory, and the concept of "free memory" ideally rather means something like "memory that can be quickly reclaimed for another purpose". Sometimes anonymous memory will be of less us occupying main memory at some given time than would be letting cached file contents take their place.
tlb 3 hours ago||
Building a swap device at user level used to be one of those classic unsolvable problems, because what if your daemon needs to swap in a page in order to swap in a page? Or at least it was discussed at a reason why microkernels will never work. I’m not sure what the solution is here.
kccqzy 3 hours ago||
Your daemon can be smart enough to know which are its own pages and prevent them from being swapped out. The Linux kernel also prevents its own text pages from being swapped out, so the solution exists and I don’t see why it doesn’t apply to microkernel designs.
netbsdusers 3 hours ago||
The general principle is that what is involved in paging should not be paged itself. Wiring the memory of that whole daemon is then a trivial solution to the problem.
NortySpock 4 hours ago||
Nice, I might try using this as I'm currently on 16 GB of RAM / 11 GB VRAM and feel like the VRAM is usually idle except for when I game or try a local LLM.

It would be nice to have dynamic scaling or even just auto-shutoff on VRAM pressure if I forget I have this enabled and then fire up a game or LLM.

willis936 18 hours ago||
I'm more interested in the opposite. Nvidia linux drivers crash when you try to address more VRAM than you have. It'd be nice if they didn't.
SV_BubbleTime 17 hours ago|
They already do that on windows and it kinda sucks. If you are targeting something like LMStudio or ComfyUI, both of those have superior methods to do exactly this.
dlt713705 17 hours ago||
Does anyone these days really use swap for anything than S4 suspend ?
kccqzy 17 hours ago||
https://news.ycombinator.com/item?id=40697318

This HN comment and the linked post brought up a lot of good points. The main takeaway is that swap should primarily be considered a mechanism for equality of reclamation, not for emergency extra memory, where equality of reclamation means file-backed pages and anonymous pages are subject to similar criteria for being evicted from physical memory.

I used to have zero swap on my Linux desktop and this convinced me to add at least a small swap partition.

dlt713705 13 hours ago|||
My point is not to say that swap should not be configured on a Linux system. On bare-metal machines, I personally always set a swap partition equal in size to the amount of RAM because I usually want to be able to put the machine into S4 (suspend to disk).

I don't consider swap to be emergency RAM storage. I know that the kernel will decide by itself to use swap even if it has plenty of available RAM and the swappiness threshold is not reached.

Nevertheless, my two decent laptops (one with 16 GB RAM, the other with 64 GB RAM) never swap, even with Docker Swarm and multiple stacks, multiple VMs, desktop activities, and gaming.

It's been a while since I last saw a physical machine actively swapping.

I understand that some limited hardware may need swap, but I can't see such hardware having a GPU with plenty of VRAM.

That said, hacking things is always fun :)

sidewndr46 16 hours ago|||
I just set swappiness to zero years ago and never looked back.
kccqzy 15 hours ago||
That’s like the complete opposite advice. Chris said the lowest recommended swappiness is 1. I have it set to 100.
sidewndr46 6 hours ago||
Agreed, but it means I still don't run into an OOM condition abruptly since the swap is there. It turns out modern disks are pretty fast, so the system freezes for a bit as physical memory is exhausted. Then I usually stop some of what I am doing and it goes back to normal.
Saris 16 hours ago|||
It's useful on lower RAM systems as the least frequently used memory can be moved to swap, freeing up more RAM for stuff that needs it. Even when using zram it works out pretty well on my laptop with 8GB of RAM, it'll often have 4GB+ in zram swap space compressed down to only 1GB or so of physical RAM usage.
yjftsjthsd-h 17 hours ago|||
It really depends on what you run and how much RAM you have to do it in. I run some machines into swap just by running a couple browsers and some containers in the background on a 16GB laptop. I've also run a single light browser and essentially nothing else on 4GB and been fine:)
dlt713705 14 hours ago||
I'm very surprised, I run a docker swarm with multiple docker stacks on my 16GB RAM laptop which is also my main machine. i have multiple browsers each with multiple tabs. I also run multiple VM (Qemu/KVM) and even by gaming on top of all of that, I can not make it swap.

Edit: Typo

xxs 6 hours ago|||
>S4 suspend

Is not popular in general, so yes. But also no - I don't use swap ever, if I have to go over the RAM (32GB being low, with 64GB the norm), might as well consider the system dead.

the8472 9 hours ago||
For me opening huge datasets, e.g. many gigabytes worth of profiling data, combined with other stuff running on the system, can end up pushing things to swap.
sgjohnson 16 hours ago||
>Sequential throughput: ~1.3 GB/s

sounds VERY low, also, wouldn't random read/write speed be MUCH more relevant here?

mmastrac 16 hours ago||
I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.

In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...

This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?

Teknoman117 16 hours ago|
Memory on an expansion card isn't gated on PCIe 5, it's gated on CXL support. CXL and PCIe use the same electrical/physical layer but the protocol is very different.

The problem with putting (system) RAM on a PCIe card is that PCIe is not a cache-coherent interconnect. If you have a cache line that resides on your GPU sitting inside your processor's cache a remote modification to that memory by either the GPU, another CPU core or some other PCIe device with NOT invalidate the CPU cache line. You also have the fun situation that if it's modified on both ends simultaneously the resulting state will be non-deterministic.

Device drivers have to be very careful about synchronization when accessing memory-like areas on PCIe. CXL adds a cache coherency protocol among other things, so that invalidations and snoops can be exchanged over the interconnect.

wang_li 15 hours ago||
It’s deterministic. But as the user you don’t know enough to know what was determined.
londons_explore 10 hours ago||
It seems obvious to me that this should be a built in functionality of the kernel.

The kernels job is to manage resources - and GPU ram is one such resource, and it can be used for many of the same uses as regular ram.

theblazehen 10 hours ago||
I've implemented the same idea with OpenCL: https://github.com/theblazehen/vramblk

There is originally https://github.com/Overv/vramfs however that has the overhead of a FUSE filesystem + loop device when using as a swap device.

The performance is rather lackluster however, it's far from a miracle "now you effectively have more ram for a 90% performance drop" - it definitely feels like traditional swapping

LouisvilleGeek 17 hours ago|
Finally a use for the expensive ram when it's not needed in workloads!

Now if it could be dynamically used and vacated on other GPU workloads?

More comments...