Deno KV vs. Cloudflare Workers KV, Upstash Redis, AWS DynamoDB, and Google Firestore
(Originally published on deno.com/blog.)
More developers are choosing to host stateful applications at the edge for a variety of reasons — higher availability with more points of presence, lower latencies due to being physically closer to users, controlling data residency for compliance, and more.
Our managed Deno KV, now in open beta, is our key value store with read-committed, sequentially consistent, multi-region replicas, that’s built on FoundationDB and baked right into the runtime. You can add state without configuring a database or juggling API keys in as little as a single line of code:
const kv = await Deno.openKv();We wanted to see how Deno KV stacked up against other leading distributed serverless databases on dimensions such as read/write latencies and pricing, as well as qualitative characteristics, such as developer experience:
If you’d like us to evaluate other serverless databases, please let us know in the issues.
Methodologies for measuring latencies
We built a benchmarking app to measure real-time read and write latencies for the five serverless databases. Each database is seeded with 14,275 key-value pairs. Anytime the app receives a request from a browser, it initiates five serverless functions (each corresponding and colocated, if possible, to the serverless database it’s measuring) that will:
- perform a simple read operation to warm up the instance (not measured)
- read a random 10 records out of 14,275 total key-value pairs from the database (the size of any combination of 10 records will always be 189KB) to avoid caching where possible, and measures it as
readlatency - alter the 10 records, then concurrently remove the old 10 records from and write the altered 10 records to the database, which is measured as
writelatency - store measured latencies in a main database, from which percentile data is calculated and displayed in the frontend
Consistency and transaction isolation levels
Since we selected these serverless database based on popularity, we recognize that not all reads and writes are equal, as not all databases offered the same consistency and transaction isolation levels:
Isolating read and write latencies
To ensure we’re only measuring read and write latencies, we:
- Minimized network latency by having all logic in edge functions colocated with corresponding databases (where possible):
- Excluded latency between browser to server by measuring read and write within the serverless function
- Excluded cold start time by warming up the database connection with an initial read
- Recorded timestamps right before and after the read and write
With all of these considerations, the recorded read and write latencies should reflect the time it takes the serverless database to complete its operation.
Results
We’ll dive into the performance more in detail later in this post, but here are the results at a glance.
Read latencies
Write latencies
The 99th percentile for read and write latencies with sample size of 10,000, except for Cloudflare KV, which had a sample size of 5,712 due to caching. Tail latencies matter because they can negatively affect user experience and confidence.
View the app or check out the source code.
Comparing performance and pricing
Before we dive into analysis for each serverless database, let’s compare them on performance and pricing.
Performance
Below is the data collected from our app with a sample size of 10,000 transactions. Note that Cloudflare KV’s longer cache expiration resulted in a smaller sample size of 5,712 transactions.
Upstash Redis performed the best with Deno KV in close second, and DynamoDB and Firestore lagging behind. Cloudflare KV performed the worst, due to its heavy caching and eventual consistency model, which isn’t well suited for measuring propagated changes with frequent reads and writes. In fact, their documentation explicitly states that changes may take up to 60 seconds to be visible due to cache timeouts.
Deno KV led the batch in write latencies with Upstash Redis as a close second, followed by AWS DynamoDB, Google Firestore, and Cloudflare KV.
Pricing
In terms of reads and writes, AWS DynamoDB and Google Firestore are the most competitive, while Upstash is the most expensive (especially global). Cloudflare reads are relatively competitively priced, while their writes are the most expensive at $5 per million writes. Deno KV is in the middle with $1 per million reads and $2.5 per million writes.
For storage, Google Firestore was cheapest at $0.18 per GiB per month. On the higher end is Cloudflare and Deno at $0.5 GiB per month.
Upstash Redis
Upstash, a serverless data platform, offers a key value store through Upstash Redis.
At a glance:
- Atomic Operations: ✅
- Global replication: ✅
- Locations: 8
- Consistency control via API: ❌
Upstash Redis uses a leader-based replication mechanism and can only provide eventual consistency.
Developer experience
Overall, using Upstash Redis is the “standard” serverless experience — you provision a database and connect to it via a Redis client library with unique host and password variables defined in your environment. There’s neither anything frustrating nor is there anything particularly delightful about the experience of setting up and connecting a serverless database.
AWS DynamoDB Global Tables
AWS DynamoDB Global Tables is a fully managed serverless distributed database from Amazon.
At a glance:
- Atomic Operations: ✅
- Global replication: ✅
- Locations: 17
- Consistency control via API: ✅
As expected from AWS, this serverless database solution is fully featured and offers developers flexibility in configuring it down to the smallest details.
Developer Experience
It should come as no surprise that dealing with anything AWS will be met with frustration and headaches. Setting up the database took a few clicks and some configuration, nothing too out of the ordinary. However, when connecting to DynamoDB, the biggest issue was bumping against stale and obtuse documentation with @aws-sdk/lib-dynamodb and figuring out the right path through a GitHub issue response from one of the maintainers…
When developing locally, if you want to use a local instance of DynamoDB for testing and iterating, there’s another step you have to take, which is to download and install “DynamoDB local”. It’s good to know that it’s possible to develop against a local version of DynamoDB, but unfortunately its a tedious multi-step setup process.
Google Firestore
Google Firestore, part of the Firebase development platform, is a serverless non-relational database.
At a glance:
- Atomic Operations: ✅
- Global replication: ✅
- Locations: 23
- Consistency control via API: ✅
Similar to AWS DynamoDB, Google Firestore is featureful and offers a ton of configuration and flexibility to meet a wide range of developer needs.
Developer Experience
Aside from the typical database provisioning steps that come with all serverless platforms, Google Firestore is a little bit simpler than the previous solutions, in that there’s one Firestore database per project. This makes connecting to the database simpler, since you just need new Firestore() and don’t have to worry about connecting to the “wrong” database.
Aside from that, interfacing with Firestore via a client library was relatively straightforward.
Cloudflare Workers KV
Cloudflare Workers KV is a low latency serverless key value storage.
At a glance:
- Atomic Operations: ❌
- Global replication: ✅
- Locations: 300
- Consistency control via API: ❌
Cloudflare Workers KV is designed to be eventually consistent, and uses a global cache system to offer quick reads on data that doesn’t change often. It does not offer atomicity nor consistency control. For those, Cloudflare recommends Durable Objects.
Developer Experience
Cloudflare Workers KV was pretty standard to provision and configure through the Cloudflare Workers website. Connecting to Cloudflare Workers KV is relatively straightforward, as well, using KV namespaces and the right settings in a config file, wrangler.toml.
Deploying Cloudflare Workers is a little tricky, as we need to bundle the function in order to deploy it to Cloudflare with wrangler. This adds a fold of complexity when testing and troubleshooting between local dev and prod environments, since bundling and sourcemapping happens on deployment.
Global Deno KV
Deno KV on Deno Deploy is built on FoundationDB and is built right into the runtime.
At a glance:
- Atomic Operations: ✅
- Global replication: ✅
- Locations: 3
- Consistency control via API: ✅
Being atomic was a prerequisite for Deno KV so it impacted how we designed the API. There’s currently three regions, and we plan to add more.
Finally, you can choose how consistent you want your reads to be, depending on your performance and data integrity needs.
Developer Experience
Connecting to Deno KV was the fastest and easiest, as it only took a line of code. There were no separate steps to provision or configure a database, or to juggle API keys.
On top of that, the local environment and production environment on Deno Deploy are the same, making testing and deployment a seamless experience.
What’s next?
Our managed version of Deno KV, built on FoundationDB, not only offers a simple zero config, ACID-compliant JavaScript API for persistent data, but also offers some of the best performance among other serverless databases.
In the past months, we’ve continued to add features to Deno KV, such as the ability to remotely connect to a managed Deno KV instance, continuous backups to S3 and GCS, and point-in-time recovery. Deno KV, alongside, Deno Queues, Deno Cron, and access to npm, means you can build for the web faster with minimal boilerplate.
As we continue to improve Deno KV towards a generally available release, we welcome your feedback.
We just released Deno KV Watch! Building real-time apps such as notifications, newsfeeds, multiplayer games, and more just got easier.
