Jian Reis
October 24, 2022

Snaplet and Netlify - bringing databases and data into your preview environments

We love Netlify - and not just because we’re part of the Netlify Jamstack Innovation Fund family. Many of our team have lived firsthand the real pain of building front end sites and apps the old-fashioned way: manually managing servers, trying to optimize sites for speed, configuring and maintaining CDNs, revision control and collaboration with a team using a hodge-podge of different third party applications 🤯. It was a nightmare, until preview environments came along and ushered in a new way of doing things.

As good as Netlify is at giving us that thoroughly modern web experience by stitching CI/CD, git, version control, hosting, scaling, and a global CDN together - there’s always been one piece of the puzzle missing: a stateful database with quality development data that you can use seamlessly across preview environments. “Ah,” we hear you say, “but you can just connect your staging database to Netlify and use that!” True, you can, and it’s what many people do.

The problem with this approach is that it fundamentally breaks the stateless model that makes Netlify so great. Preview environments are terrific when working on new features, reproducing bugs, and generally increasing your confidence that what you're building is behaving correctly, because you can see and test your code live as if it was in production already. You know this - this is why you use Netlify.

Preview deployments are difficult to use effectively when your website uses a database though. You can’t use your production database because it’s definitely not safe to have production data in your test environment, and your staging environment’s data probably doesn’t accurately reflect production, which defeats the point of a preview deployment. Neither way really works, and it gets worse if you’re testing a database change, which potentially breaks everyone else’s preview environments linked to that database. Of course, you can branch that database and configure your preview environments to use that new database, but you’re now spending your time managing the kinds of services and servers you specifically didn’t want to when you started using Netlify! Catch 22.

The problem with using a staging database for preview deployments

Here’s where Snaplet can help - our new Netlify Preview Deployment plugin brings a Netlify-like experience to managing stateful databases, with the ability to create unique, instantaneous preview databases (with data) for each of your Netlify preview deployments. The Snaplet plugin lets you have stateless preview environments in Netlify, isolated from each other, each with their own unique and temporary databases that are bootstrapped with production-realistic data, all delivered in your existing Netlify gitops workflow! And because it’s Snaplet, the data you get to use in your Netlify preview environments is based on your production database, but with all the sensitive personally-identifiable information pseudonymized.

If you’ve bought into the idea of using Netlify, you don’t want the hassle of managing databases, and now, thanks to Snaplet, you don’t have to. Simply setup and install the Snaplet Netlify Preview Deployments plugin, and you’ll be able to use Snaplet in your workflow to create unique preview databases for each Netlify deployment. No more worrying about conflicting migrations related to data changes, and no more worrying about databases.

How preview databases can solve conflicts for preview environments

If you’re currently a Snaplet user, you may have already realized that this also allows you to utilize the full power of Snaplet’s data operations to transform, subset, or generate new data as part of your development process. Want to test a new feature on your web app that requires a new table with data? With the Snaplet Netlify plugin, it’s as simple as configuring your Snaplet transformations or running one of our data operations and deploying as usual with Netlify to get a new preview environment.

Check out our How To guide on how to get Snaplet integrated into your Netlify workflow, read more on the plugin's Github, or watch the video of Snaplet co-founder Peter Pistorius demoing the feature on “What’s new in Prisma”. While we’re confident that this solution is working well, there’s a few different moving parts involved as the preview databases themselves are hosted on Fly.io. If you experience any problems getting this working, we’d be happy to help you over in our Discord.

Jian Reis