What is an Apache Pulsar Topic?

Apache Pulsar topics are the categories used to organize messages. Each Pulsar topic has a name that is unique across the entire Pulsar cluster.

Messages are sent to and read from specific topics. Producers write data to topics, and consumers read data from topics. 

Pulsar topics are multi-subscriber.  This means that a topic can have zero, one, or multiple consumers subscribing to that topic and the data written to it. 

Consumers subscribe to topics, choosing from four different subscription types: Exclusive, Failover, Shared, and Key-Shared. It’s often best to use a Key_Shared subscription because it is both scalable and retains message ordering.  Learn more about Pulsar subscription types.

In Pulsar, topics are partitioned and replicated across brokers.  Brokers refer to each of the nodes in a Pulsar cluster. The partitions are important because they enable parallelization of topics for high message throughput. 

For an overview of Pulsar architecture, check out our blog post “What is Apache Pulsar?”.

How to Create Pulsar Topics

Pulsar topics can either be created automatically or manually. And as a default setting, Pulsar automatically creates a topic if it doesn’t exist when a client attempts to produce or consume. 

However, it is best practice to manually create all topics before starting an application, rather than using auto topic. 

Use the following command to create a topic manually. In this example we are creating a topic in the apache/pulsar namespace with 3 partitions: 

> bin/pulsar-admin topics create-partitioned-topic apache/pulsar/test-topic -p 3

If you need to create a namespace, use the following command:

> bin/pulsar-admin namespaces create apache/pulsar

In the above example, we created the namespace apache/pulsar. 

Where are Pulsar topics stored?

BookKeeper bookies act as the persistent storage layer for Pulsar.  Typically there are at least three bookies per topic for redundancy and performance. 

Recall that Pulsar is a two-layer system with separate storage and serving layers. Pulsar brokers act as the serving layer.  A single Pulsar broker owns a single partition topic, and all writes and reads to that partition topic must go through that specific broker. 

Learn more about Pulsar’s two-layer architecture and specifically how it relates to partitioned topics.

How to View a List of Pulsar Topics

Use the following command to view a list of Pulsar topics:

> bin/pulsar-admin topics list-partitioned-topics apache/pulsar

How to Clear a Pulsar Topic

A Pulsar topic can be cleared (also referred to as being cleaned or purged) by reducing the retention time.  

For instance, if the retention time is 168 hours (one week), then reduce retention time down to a second.  After that 1 second, the Pulsar topic will be cleaned / purged / cleared.