Fast Data Processing with a Ramdisk

When we process data with a computer CPU, we need to place it somewhere. Large data may not fit in your computer memory, so it is necessary to store it on disks that may be slow. Using SSD and especially NVMe has increased the access speed significantly in contrast to cheaper HDD. One way to accelerate data access is to use disks in parallel on multiple storage devices. But wait, what if you only have moderate data and it fits into your computer’s RAM. Can we not just create a volatile disk that is placed in the RAM entirely?

So let us start looking at how we can achieve this in Ubuntu. We will be using ubuntu 20.04, but it should also work on other versions of Ubuntu. We discuss the two different parts separately.

1. Setting up the ramdisk
2. Saving and Restoring the ramdisk

Setting Up the ramdisk

For our purpose, we will set up the ramdisk at `/mt/ramdisk’ with the commands while assuming you have plenty of RAM for it. On my system, I have 128GB, and I use 28GB for the ramdisk. Please adjust your ramdisk size according to your needs and availability. If you need a smaller ramdisk, you can also do this while using the size in megabyte such as 512M.

sudo mkdir /mnt/ramdisk
sudo mount -t tmpfs -o rw,size=28G tmpfs /mnt/ramdisk
df -h | fgrep ramdisk

Next, we need to register the ramdisk in the file system table. We will see an entry for the ramdisk. Edit it with your favorite editor. Mine is emacs, but you can use pico, nano, vim, vi, or others.

sudo emacs /etc/fstab

Go to the end of the file and add the line

tmpfs  /mnt/ramdisk  tmpfs  rw,size=28G  0   0

Chose the same size that you have used earlier. Next, reboot your machine with

sudo reboot

After the reboot, you can make sure your system is still there, just as we did previously with

df -h | fgrep ramdisk

Unmounting

In case you do not need the ramdisk, you can unmount it, and the RAM can be used otherwise. To remount it, reboot the system or use

sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk

Saving and restoring content with rsync

Please be aware that in case something goes wrong on your system and it shuts down or reboots, all data in the ramdisk gets lost. Thus it is important that you employ a backup strategy.

Let us create a backup directory on the regular file system at /mnt/ramdisk.bak. Let us create the directory with

sudo mkdir /mnt/ramdisk.bak

As we expect to not change always all data, we use rsync to copy only the modified files.

rsync -av /mnt/ramdisk/. ~/Desktop/ramdisk.bak

To refresh the ramdisk from your backed up data, use the inverse with

rsync -av ~/Desktop/ramdisk.bak/. /mnt/ramdisk/

Conclusion

Try this all out on some test example to see if it works for you before you engage in employing it. Always keep in mind the volatility related to power outages and unexpected system shutdown; you will lose your data in the rammdisk.

In many cases it may be sufficient to place your data sets in ramdisk, while you do your other program development on the regular file system.