I don’t think it’ll surprise anyone to know that over here at Snaplet, we’re huge fans of Postgres, and when we saw Crunchydata’s post on HackerNews about Postgres running in a browser, we knew we wanted to use that to offer an easier way to evaluate Snaplet without having to install any software.
We were inspired by Crunchydata to replicate the feat, but also, to open-source the work and make it available to everyone. We put out the call in the comments of the HN post for collaborators and were thrilled to have Mark Burggraf from Supabase join us!
💚 Supabase is an amazing open source Firebase alternative. Get your own project started with a Postgres database, Authentication, instant APIs, Edge Functions, Realtime subscriptions, and Storage with Supabase. 💚
We first tried to compile PostgreSQL for WebAssembly, but that approach quickly fizzled out, because frankly, we’re web developers and didn’t have enough exposure to C, make, and the other tools to be really effective. The HN post provided some additional details about the approach they took, which was to virtualize a machine in the browser. We settled on v86, which emulates an x86-compatible CPU and hardware in the browser.
Our goal was to get it working, so we hacked something together using a Debian image to prove the concept worked in principle, but the image size was huge! More challenging was building out a robust system that included a stable (and small) Linux distro. We found Buildroot, which has the goal of “making embedded Linux easy”, and we were able to create an 12MB snapshot image capable of running a PostgreSQL server!
So now that we have a VM running Linux running Postgres in the browser, where does it suck? Mostly in the I/O parts, which is to be expected, since one of the goals of a VM (and a browser) is to create an isolated environment and we’re attempting to break out of that!
Where this can be improved
Networking
Getting data in and out of the virtual machine is not as easy as one might expect it to be, since browsers do not support raw TCP/ UDP sockets, so we defaulted to the lowest common denominator: WebSockets.
There are elegant hacks using a virtual networking device in the VM that communicates via WebSocket to a proxy. We’ll work towards making that an easy-to-launch and integrated experience.
Performance
The initial machine state file is a fairly large 12MB. We cache those after the first run, but the first launch is brutal. We’ll continue to drive the performance train.
We reduced the size of the CPU and memory snapshot by enabling page_poisioning in the Linux kernel. This flag writes an arbitrary sequence of bytes, instead of random bytes, to freed memory, which compresses really well.
Additionally, we use the 9p remote filesystem protocol, which allows us to lazily load elements on demand: As an example, if you ran gzip this would reach over the network, download, and cache that binary and its dependencies, and then execute it.
File system access
Getting files into the VM is easy, our basic UI allows you to upload files into the VM. The uploaded files are stored in /mnt folder - and getting files out is not currently implemented.
We want to use the File System Access API to automatically sync files between your local filesystem and the filesystem running in the VM.
Conclusion
We’re pretty happy with our results. Supabase and Snaplet glued together the work of other brilliant engineers to create something we’re excited to share with the community. If the idea of contributing towards open-source Postgres in a browser excites you, come join us! See the demo here.
Read more:
https://news.ycombinator.com/threads?id=kiwicopple
https://postgres-wasm.netlify.app/
