This document assumes that you are already familiar with the process of subscribing to a Pub/Sub topic and receiving messages in your subscriber client.
If you're new to Pub/Sub, see one of the Quickstart guides and learn how to run Pub/Sub using the console, Google Cloud CLI, or the client libraries.
Choose the right subscription
Pub/Sub offers standard subscriptions such as push and pull subscriptions. In addition to the standard subscriptions, Pub/Sub also offers export subscriptions that lets you store messages directly to a Google Cloud resource, without requiring Dataflow as an intermediary. For example, BigQuery subscriptions store messages in a BigQuery table.
Push subscriptions are recommended for the following scenarios:
You cannot include any code in your subscriber application that imports the client library as a dependency.
The subscriber client cannot make any outgoing requests.
You want to use the same instance to process messages from different topics and subscriptions where the subscriber client doesn't know the list of subscriptions.
For general cases, we recommend using the
high-level client library. If you're
instead using unary pull, don't set
returnImmediately
to true. Setting it to true adversely impacts pull performance.
The field returnImmediately is now deprecated.
To compare all the subscription types and choose the one that best fits your business needs, see the Pub/Sub subscription comparison table.
To learn about the benefits of an export subscription, see When to use an export subscription.
Process messages before acknowledging them
By default, Pub/Sub discards a message from a subscription after the message is acknowledged. If you don't process a message before sending an acknowledgment and the processing fails, the service does not redeliver the message. The exception is when you have configured retaining acknowledged messages or topic retention and you perform a seek operation.
If you have high-latency subscribers, you might need to set custom values for flow control and lease management.
Configure subscriber flow control for transient traffic spikes
Flow control on the subscriber side lets you prevent subscribers from being overloaded by traffic spikes. It can allow time for autoscaling mechanisms to respond to an increased load or it can spread the processing of the load over a longer period of time. The former method saves latency, while the latter saves costs.
To configure flow control, you must set
appropriate values for maximum outstanding messages and
total outstanding message bytes. The default values for these flow control
variables and the names of the variables might differ across client libraries.
Maximum outstanding messages defines the maximum number of messages delivered to the client for which Pub/Sub has not received acknowledgments or negative acknowledgments.
Total outstanding message bytes defines the maximum total size of messages delivered to the client for which Pub/Sub has not received acknowledgments or negative acknowledgments.
If the limit for one of these options is crossed, the subscriber client does not pull more messages. This behavior continues until the messages that are already pulled get acknowledged or negatively acknowledged. In this way, you can trade off throughput with the cost associated with running more subscribers.
Handle duplicate deliveries
By default, Pub/Sub provides at-least-once delivery of messages to subscribers. That means messages can be delivered multiple times, even if they were acknowledged. The following sections discuss how to address common redelivery scenarios.
Consistent redelivery of many messages
If you are experiencing cases where many messages are always getting redelivered, then your subscribers are overloaded or they are not acknowledging the messages before the deadline expires.
If you are using a pull subscription, you might need to set custom values to flow control values or increase the lease extension periods using lease management.
If you are using push subscriptions, you might need to increase the acknowledgement deadline setting. You can also follow best practices regarding how to maintain a healthy subscription.
Occasional redelivery of messages
When you see messages get redelivered before the acknowledgement deadline has passed or after the messages were acknowledged within a few seconds, Pub/Sub is behaving as expected. You shouldn't see these redelivery spikes frequently, but when redeliveries do occur, they are likely to occur on several messages simultaneously. Your system must be built to tolerate these occasional duplicates.