Published August 2023
The Apache Pulsar schema registry ensures a smooth transfer of information between producers and consumers.
One of the reasons a distributed messaging platform is important is because sometimes a producer writes data in a format that cannot be read by a consumer.
For instance, maybe a producer sends data in CSV format but the consumer only reads data in JSON format. The schema registry in Pulsar solves this issue by providing flexibility in how applications interact or share data.
How the Pulsar Schema Registry Works
- Client-side – In the “client-side” approach the producers and consumers are responsible for knowing the data types that are transmitted via each topic.
And additionally, the producers and consumers are responsible for serializing and deserializing messages. Serialization is the conversion of a data structure into a format that can be stored or transmitted and then reconstructed later through deserialization.
This “client-side” approach severely limits the flow of data by putting the responsibility on the producers and consumers to know or anticipate the data types being transmitted. - Server-side – The other approach is referred to as “server side”. The built-in schema registry in Pulsar enables clients to upload data schemas. These schemas instruct which data types are valid for a particular topic.
This can be done at the topic level.
What Data Types are Supported?
As of version 2.10.x, the schema registry is only available for Java client, Go client, Python client, and C++ client.
For an up-to-date list of the support schema registries visit: https://pulsar.apache.org/docs/2.10.x/schema-get-started/.
Where Pulsar Schemas are Stored
Schemas are stored on BookKeeper out-of-the-box.
Schemas can also be automatically uploaded when a typed Producer with a Schema is created. Conversely Schemas can also be manually uploaded using Pulsar’s REST API.
Apache Pulsar Support Services
If you are interested in 24/7 support, consulting, and/or fully managed Pulsar services, you can find more information on our Apache Pulsar services page.
Schedule a call with a Pulsar solution architect.
Published by