Apache Pulsar Schema Registry

The Apache Pulsar schema registry ensures a smooth transfer of information between producers and consumers.

One of the reasons a distributed messaging platform is important is because sometimes a producer writes data in a format that cannot be read by a consumer.

For instance, maybe a producer sends data in CSV format but the consumer only reads data in JSON format. The schema registry in Pulsar solves this issue by providing flexibility in how applications interact or share data.

How the Pulsar Schema Registry Works

Pulsar offers two different approaches to type safety.

Client-side – In the “client-side” approach the producers and consumers are responsible for knowing the data types that are transmitted via each topic.

And additionally, the producers and consumers are responsible for serializing and deserializing messages. Serialization is the conversion of a data structure into a format that can be stored or transmitted and then reconstructed later through deserialization.

This “client-side” approach severely limits the flow of data by putting the responsibility on the producers and consumers to know or anticipate the data types being transmitted.

Server-side – The other approach is referred to as “server side”. The built-in schema registry in Pulsar enables clients to upload data schemas. These schemas instruct which data types are valid for a particular topic.

This can be done at the topic level.

Pulsar supports both approaches, and the approaches can be chosen on a per-topic basis.

What Data Types are Supported?

As of version 2.10.x, the schema registry is only available for Java client, Go client, Python client, and C++ client.

For an up-to-date list of the support schema registries visit: https://pulsar.apache.org/docs/2.10.x/schema-get-started/.

Where Pulsar Schemas are Stored

Schemas are stored on BookKeeper out-of-the-box.

Schemas can also be automatically uploaded when a typed Producer with a Schema is created. Conversely Schemas can also be manually uploaded using Pulsar’s REST API.

24x7 Pulsar Support & Consulting

Visit our Apache Pulsar™ page for more details on our support services.

Apache Pulsar Schema Registry