Amazon DocumentDB: how it works
Amazon DocumentDB (with MongoDB compatibility) is a fully managed, MongoDB-compatible database service. With Amazon DocumentDB, you can run the same application code and use the same drivers and tools that you use with MongoDB. Amazon DocumentDB is compatible with MongoDB 3.6, 4.0, 5.0, and 8.0.
Topics
Amazon DocumentDB versions
Note
Minor version numbering applies to Amazon DocumentDB 5.0 and later. Amazon DocumentDB 3.6 and 4.0 use only major versions with engine patches.
Starting with Amazon DocumentDB 5.0, the engine version uses a four-level numbering scheme: major.major.minor.patch.
-
Major version—Identified by the first two numbers of the version (for example, 5.0). The major version represents the MongoDB compatibility level. Major versions introduce new features, performance improvements, and MongoDB compatibility updates. Upgrading between major versions (for example, from 4.0 to 5.0) requires a major version upgrade (MVU). For more information, see Amazon DocumentDB in-place major version upgrade.
-
Minor version—Identified by the third number of the version (for example, the
1in 5.0.1). Minor versions include improvements such as new features, security fixes, and bug fixes within the same major version. You can upgrade to a new minor version manually. For more information, see Amazon DocumentDB minor version upgrade. -
Patch version—Engine patches are applied within a minor version and contain critical security fixes and bug fixes. The patch version is tracked by the engine patch version (for example,
3.0.17983), which you can check by runningdb.runCommand({getEngineVersion: 1}). Patches are applied during your cluster's maintenance window. For more information, see Maintaining Amazon DocumentDB.
When you create a new cluster, you can specify any available engine version. For information about available versions and support dates, see Amazon DocumentDB engine version support dates.
When you use Amazon DocumentDB, you begin by creating a cluster. A cluster consists of zero or more database instances and a cluster volume that manages the data for those instances. An Amazon DocumentDB cluster volume is a virtual database storage volume that spans multiple Availability Zones. Each Availability Zone has a copy of the cluster data.
An Amazon DocumentDB cluster consists of two components:
-
Cluster volume—Uses a cloud-native storage service to replicate data six ways across three Availability Zones, providing highly durable and available storage. An Amazon DocumentDB cluster has exactly one cluster volume, which can store up to 256 TiB of data for Engine Version 8.0 and beyond (128 TiB for earlier engine versions).
-
Instances—Provide the processing power for the database, writing data to, and reading data from, the cluster storage volume. An Amazon DocumentDB cluster can have 0–16 instances.
Instances serve one of two roles:
-
Primary instance—Supports read and write operations, and performs all the data modifications to the cluster volume. Each Amazon DocumentDB cluster has one primary instance.
-
Replica instance—Supports only read operations. An Amazon DocumentDB cluster can have up to 15 replicas in addition to the primary instance. Having multiple replicas enables you to distribute read workloads. In addition, by placing replicas in separate Availability Zones, you also increase your cluster availability.
The following diagram illustrates the relationship between the cluster volume, the primary instance, and replicas in an Amazon DocumentDB cluster:
Cluster instances do not need to be of the same instance class, and they can be provisioned and terminated as desired. This architecture lets you scale your cluster’s compute capacity independently of its storage.
When your application writes data to the primary instance, the primary
executes a durable write to the cluster volume. It then replicates the
state of that write (not the data) to each active replica. Amazon DocumentDB replicas
do not participate in processing writes, and thus Amazon DocumentDB replicas are
advantageous for read scaling. Reads from Amazon DocumentDB replicas are eventually
consistent with minimal replica lag—usually less than 100
milliseconds after the primary instance writes the data. Reads from the
replicas are guaranteed to be read in the order in which they were written
to the primary. Replica lag varies depending on the rate of data change,
and periods of high write activity might increase the replica lag. For more
information, see the ReplicationLag metrics at Amazon DocumentDB metrics.
Amazon DocumentDB endpoints
Amazon DocumentDB provides multiple connection options to serve a wide range of use cases. To connect to an instance in an Amazon DocumentDB cluster, you specify the instance's endpoint. An endpoint is a host address and a port number, separated by a colon.
Connect to your cluster using the cluster endpoint in replica set mode (see Connecting to Amazon DocumentDB as a replica set). Use this approach unless you must connect to the reader endpoint or an instance endpoint. To route requests to your replicas, choose a driver read preference setting that maximizes read scaling while meeting your application's read consistency requirements. The secondaryPreferred read preference enables replica reads and frees up the primary instance to do more work.
The following endpoints are available from an Amazon DocumentDB cluster.
Cluster endpoint
The cluster endpoint connects to your cluster’s current primary instance. The cluster endpoint can be used for read and write operations. An Amazon DocumentDB cluster has exactly one cluster endpoint.
The cluster endpoint provides failover support for read and write connections to the cluster. If your cluster’s current primary instance fails, and your cluster has at least one active read replica, the cluster endpoint automatically redirects connection requests to a new primary instance. When connecting to your Amazon DocumentDB cluster, use the cluster endpoint in replica set mode (see Connecting to Amazon DocumentDB as a replica set).
The following is an example Amazon DocumentDB cluster endpoint:
sample-cluster.cluster-123456789012.us-east-1.docdb.amazonaws.com:27017
The following is an example connection string using this cluster endpoint:
mongodb://username:password@sample-cluster.cluster-123456789012.us-east-1.docdb.amazonaws.com:27017
For information about finding a cluster's endpoints, see Finding a cluster's endpoints.
Reader endpoint
The reader endpoint load balances read-only connections across all available replicas in your cluster.
A cluster reader endpoint will perform as the cluster endpoint if you are connecting through replicaSet mode, meaning in the connection string, the replica set parameter is &replicaSet=rs0.
In this case, you will be able to perform write operations on the primary.
However, if you connect to the cluster specifying directConnection=true, then attempting to perform a write operation over a connection to the reader endpoint results in an error.
An Amazon DocumentDB cluster has exactly one reader endpoint.
If the cluster contains only one (primary) instance, the reader endpoint connects to the primary instance. When you add a replica instance to your Amazon DocumentDB cluster, the reader endpoint opens read-only connections to the new replica after it is active.
The following is an example reader endpoint for an Amazon DocumentDB cluster:
sample-cluster.cluster-ro-123456789012.us-east-1.docdb.amazonaws.com:27017
The following is an example connection string using a reader endpoint:
mongodb://username:password@sample-cluster.cluster-ro-123456789012.us-east-1.docdb.amazonaws.com:27017
The reader endpoint load balances read-only connections, not read requests. If some reader endpoint connections are more heavily used than others, your read requests might not be equally balanced among instances in the cluster. It is recommended to distribute requests by connecting to the cluster endpoint as a replica set and utilizing the secondaryPreferred read preference option.
For information about finding a cluster's endpoints, see Finding a cluster's endpoints.
Instance endpoint
An instance endpoint connects to a specific instance within your cluster. The instance endpoint for the current primary instance can be used for read and write operations. However, attempting to perform write operations to an instance endpoint for a read replica results in an error. An Amazon DocumentDB cluster has one instance endpoint per active instance.
An instance endpoint provides direct control over connections to a specific instance for scenarios in which the cluster endpoint or reader endpoint might not be appropriate. An example use case is provisioning for a periodic read-only analytics workload. You can provision a larger-than-normal replica instance, connect directly to the new larger instance with its instance endpoint, run the analytics queries, and then terminate the instance. Using the instance endpoint keeps the analytics traffic from impacting other cluster instances.
The following is an example instance endpoint for a single instance in an Amazon DocumentDB cluster:
sample-instance.123456789012.us-east-1.docdb.amazonaws.com:27017
The following is an example connection string using this instance endpoint:
mongodb://username:password@sample-instance.123456789012.us-east-1.docdb.amazonaws.com:27017
Note
An instance’s role as primary or replica can change due to a failover event. Your applications should never assume that a particular instance endpoint is the primary instance. Don’t connect to instance endpoints for production applications. Instead, use the cluster endpoint in replica set mode (see Connecting to Amazon DocumentDB as a replica set). For more advanced control of instance failover priority, see Understanding Amazon DocumentDB cluster fault tolerance.
For information about finding a cluster's endpoints, see Finding an instance's endpoint.
Replica set mode
You can connect to your Amazon DocumentDB cluster endpoint in replica set mode by specifying the replica set name rs0. Connecting in replica
set mode provides the ability to specify the Read Concern, Write Concern, and Read Preference options. For more information, see Read consistency.
The following is an example connection string connecting in replica set mode:
mongodb://username:password@sample-cluster.cluster-123456789012.us-east-1.docdb.amazonaws.com:27017/?replicaSet=rs0
When you connect in replica set mode, your Amazon DocumentDB cluster appears to your drivers and clients as a replica set. Instances added and removed from your Amazon DocumentDB cluster are reflected automatically in the replica set configuration.
Each Amazon DocumentDB cluster consists of a single replica set with the default name rs0. The replica set name cannot be modified.
Connecting to the cluster endpoint in replica set mode is the recommended method for general use.
Note
All instances in an Amazon DocumentDB cluster listen on the same TCP port for connections.
TLS support
For more details on connecting to Amazon DocumentDB using Transport Layer Security (TLS), see Encrypting data in transit.
Amazon DocumentDB storage
Amazon DocumentDB data is stored in a cluster volume, which is a single, virtual volume that uses solid state drives (SSDs). A cluster volume consists of six copies of your data, which are replicated automatically across multiple Availability Zones in a single AWS Region. This replication helps ensure that your data is highly durable, with less possibility of data loss. It also helps ensure that your cluster is more available during a failover because copies of your data already exist in other Availability Zones. These copies can continue to serve data requests to the instances in your Amazon DocumentDB cluster.
How data storage is billed
Amazon DocumentDB automatically increases the size of a cluster volume as the amount of data increases. An Amazon DocumentDB cluster volume can grow to a maximum size of 256 TiB for Engine Version 8.0 and beyond (128 TiB for earlier engine versions). However, you are only charged for the space that you use in an Amazon DocumentDB cluster volume. Starting with Amazon DocumentDB 4.0, when data is removed, such as by dropping a collection or index, the overall allocated space decreases by a comparable amount. Thus, you can reduce storage charges by deleting collections, indexes, and databases that you no longer need. In Amazon DocumentDB version 3.6, the cluster volume can reuse space that's freed up when you remove data, but the volume itself never decreases in size. As a result in version 3.6, you may not witness any change in storage when you drop a collection or index, even though the freed up space is reused.
Note
With Amazon DocumentDB 3.6, storage costs are based on the storage "high water mark" (the maximum amount that was
allocated for the Amazon DocumentDB cluster at any point in time). You can manage costs by avoiding ETL practices that
create large volumes of temporary information, or that load large volumes of new data prior to removing unneeded older data.
If removing data from an Amazon DocumentDB cluster results in a substantial amount of allocated but unused space, resetting
the high water mark requires doing a logical data dump and restore to a new cluster, using a tool such as mongodump or
mongorestore. Creating and restoring a snapshot does not reduce the allocated storage because the physical layout
of the underlying storage remains the same in the restored snapshot.
Note
Using utilities like mongodump and mongorestore incur I/O charges based on the sizes of the data
that is being read and written to the storage volume.
For information about Amazon DocumentDB data storage and I/O pricing, see Amazon DocumentDB (with MongoDB compatibility) pricing
Amazon DocumentDB replication
In an Amazon DocumentDB cluster, each replica instance exposes an independent endpoint. These replica endpoints provide read-only access to the data in the cluster volume. They enable you to scale the read workload for your data over multiple replicated instances. They also help improve the performance of data reads and increase the availability of the data in your Amazon DocumentDB cluster. Amazon DocumentDB replicas are also failover targets and are quickly promoted if the primary instance for your Amazon DocumentDB cluster fails.
Amazon DocumentDB reliability
Amazon DocumentDB is designed to be reliable, durable, and fault tolerant. (To improve availability, you should configure your Amazon DocumentDB cluster so that it has multiple replica instances in different Availability Zones.) Amazon DocumentDB includes several automatic features that make it a reliable database solution.
Storage auto-repair
Amazon DocumentDB maintains multiple copies of your data in three Availability Zones, greatly reducing the chance of losing data due to a storage failure. Amazon DocumentDB automatically detects failures in the cluster volume. When a segment of a cluster volume fails, Amazon DocumentDB immediately repairs the segment. It uses the data from the other volumes that make up the cluster volume to help ensure that the data in the repaired segment is current. As a result, Amazon DocumentDB avoids data loss and reduces the need to perform a point-in-time restore to recover from an instance failure.
Survivable cache warming
Amazon DocumentDB manages its page cache in a separate process from the database so that the page cache can survive independently of the database. In the unlikely event of a database failure, the page cache remains in memory. This ensures that the buffer pool is warmed with the most current state when the database restarts.
Crash recovery
Amazon DocumentDB is designed to recover from a crash almost instantaneously, and to continue serving your application data. Amazon DocumentDB performs crash recovery asynchronously on parallel threads so that your database is open and available almost immediately after a crash.
Resource governance
Amazon DocumentDB safeguards resources for critical processes such as health checks. When an instance experiences high memory pressure, Amazon DocumentDB throttles requests. Some operations might be queued until the memory pressure subsides. If memory pressure continues, queued operations might time out.
Monitor for low-memory throttling with the following CloudWatch metrics: LowMemThrottleQueueDepth, LowMemThrottleMaxQueueDepth, LowMemNumOperationsThrottled, LowMemNumOperationsTimedOut. For more information, see Monitoring Amazon DocumentDB with CloudWatch.
If you see sustained memory pressure, scale up your instance to provide additional memory for your workload.
Read preference options
Amazon DocumentDB uses a cloud-native shared storage service that replicates data six times across three Availability Zones to provide high levels of durability. Amazon DocumentDB does not rely on replicating data to multiple instances to achieve durability. Your cluster’s data is durable whether it contains a single instance or 15 instances.