Kafka decouples producers from consumers — they can be deployed, updated, and scaled independently. This independence is a strength, but it also means a producer may write messages using a newer schema while an old consumer is still running, or vice versa. Without a shared schema contract, either side risks receiving bytes it cannot deserialize.

• A producer adds a required field. Old consumers that are unaware of this field crash or silently drop data.
• A consumer is upgraded to expect a field that old producers do not yet emit — null pointer exceptions follow.
• A field is renamed in the schema without coordination — deserialization succeeds but the field is always empty.
• Zero-downtime rolling deploys require new and old instances to coexist and share the same Kafka topics simultaneously.

Confluent Schema Registry solves these problems by storing every schema version, assigning it a unique integer ID, and embedding that ID in every Kafka message (as the first 5 bytes of the payload). Before a producer can write, the serializer checks the Registry to confirm the schema is registered and compatible with the subject's configured mode.

Subject Naming Strategies

A subject is the namespace under which schema versions are grouped in the Registry. The naming strategy determines how subjects are named for a given topic and schema.

Apache Avro schemas are defined in JSON using .avsc files. Avro is schema-based — the schema is always required for serialization and deserialization, which makes it an ideal pairing with Schema Registry. Avro data is compact (no field names in the binary payload) and supports rich evolution rules via default values.

Primitive Types

• null — no value
• boolean — true / false
• int — 32-bit signed integer
• long — 64-bit signed integer
• float / double — IEEE 754 floating point
• bytes — sequence of 8-bit unsigned bytes
• string — Unicode character sequence

Complex Types

Full Example: Order Schema (v1)

Schema Registry enforces a compatibility mode per subject. When a new schema version is registered, the Registry checks it against existing versions according to the configured mode. If the check fails, the registration is rejected with a 409 Conflict response.

The default global compatibility mode in Confluent Schema Registry is BACKWARD. You can override it globally or set a different mode per subject.

BACKWARD compatibility means the new schema can read data written with the old schema. A consumer upgraded to the new schema can still deserialize messages that were produced before the upgrade. This is the most common mode because consumers are usually upgraded before producers in a rolling deploy.

Rules

• Allowed: Add a new field that has a default value — old messages simply use the default.
• Allowed: Remove a field that had no default — old consumers that had the field will no longer receive it.
• Forbidden: Add a new field without a default — old messages do not carry the field and deserialization fails.
• Forbidden: Change a field's type — binary encoding is incompatible.
• Forbidden: Rename a field — the field is treated as removed and re-added, breaking old readers.

Example: v1 → v2 (add optional field)

Schema v1 — original Order schema registered to the subject orders-value:

Schema v2 — adding an optional couponCode field with a default of null (BACKWARD-compatible):

Register v2 via Schema Registry REST API

Verify Compatibility Before Registering

FORWARD compatibility means the old schema can read data written with the new schema. An old consumer running the previous schema version can still deserialize messages produced by a newer producer. Unknown fields in the new message are simply ignored by the old reader. In this mode producers are upgraded first, then consumers catch up.

Rules

• Allowed: Add a new field without a default — old readers simply skip the unknown field.
• Allowed: Remove a field that had a default — old readers use the default when the field is absent.
• Forbidden: Remove a field that had no default — old readers require that field and will fail.
• Forbidden: Change a field's type or rename a field.

Example: v1 → v2 (add required field — FORWARD-compatible)

The new schema adds region without a default. Old consumers using GenericRecord will receive the field but simply ignore it if they do not reference it.

Example: v1 → v2 (remove field with default — FORWARD-compatible)

Removing legacySource which had a default of "WEB". Old readers that expected this field will use the default value when reading the new (shorter) message.

FULL compatibility is the intersection of BACKWARD and FORWARD: the new schema must be readable by old consumers and old messages must be readable by the new consumer. This is the most restrictive mode but also the safest for rolling deploys — you can upgrade producers and consumers in any order without risk.

Rules (must satisfy both BACKWARD and FORWARD simultaneously)

• Allowed: Add a field that has a default value.
• Allowed: Remove a field that has a default value.
• Forbidden: Add a field without a default.
• Forbidden: Remove a field without a default.
• Forbidden: Rename a field, change a field type, reorder enum symbols.

Example: v1 → v2 under FULL (add field with default, remove field with default)

v1 had legacyChannel with "default": "UNKNOWN" and lacked priority.

Because priority has a default, new consumers reading old messages will receive NORMAL. Because legacyChannel had a default, old consumers reading new messages will use UNKNOWN. Both directions are covered — FULL compatibility is satisfied.

Set FULL compatibility at the subject level

Schema Registry exposes a REST API on port 8081 by default. All schema management operations — registration, retrieval, compatibility checks, and deletion — are available over HTTP.

Register a Schema

List All Subjects

List Versions for a Subject

Fetch Latest Schema

Fetch Schema by Version Number

Fetch Schema by Global ID

Check Schema Compatibility

Set Subject-Level Compatibility

Soft Delete a Schema Version

Confluent CLI (kafka-schema-registry-cli)

The Confluent Kafka client library includes KafkaAvroSerializer and KafkaAvroDeserializer that handle schema registration, ID embedding, and schema lookup transparently. You only need to configure the Schema Registry URL.

Maven Dependencies

Producer Configuration

Consumer Configuration — Specific Class

Consumer — GenericRecord (Schema-Agnostic)

The following changes break compatibility regardless of mode and will cause runtime failures in either producers, consumers, or both.

1. Renaming a Field

Avro has no concept of field aliases that survive binary round-trips in a transparent way. Renaming customerId to userId is treated as removing the old field and adding a new one. Old consumers expecting customerId will receive null; they will not find the value under userId.

The aliases field helps Avro's schema resolution during deserialization, but only when both writer and reader schemas are available to the decoder (e.g., in Avro file-based reads). In Kafka with Schema Registry, consumers fetch the writer schema by ID — aliases are not a reliable rename strategy. Introduce a new field, migrate consumers, deprecate and remove the old field across multiple releases.

2. Changing a Field's Type

Changing amount from double to string is a binary-incompatible change. Avro's wire encoding for these types is completely different — a consumer using the old schema will misread the bytes as a double and produce garbage or throw an exception.

Avro does support a limited set of type promotions during schema resolution (e.g., int → long, float → double), but changing double to string is never safe. Always introduce a new field with a new name if you need a different type.

3. Removing a Required Field

Removing a field that has no default value is forbidden under BACKWARD and FULL modes. Even under FORWARD mode, producers that still run the old code will continue to produce the field — but consumers using the new schema that drops it will silently lose that data because it will no longer be mapped to any field in the deserialized object.

4. Skipping Version Registrations

Going from v1 directly to v3 without registering v2 bypasses the transitive checks. Under BACKWARD_TRANSITIVE the Registry checks all registered versions — but if v2 was never registered, there is a gap in compatibility guarantees and consumers running v2 code may encounter unexpected messages.

5. Using NONE Compatibility in Production

Setting NONE disables all Registry enforcement and delegates compatibility entirely to manual coordination. In a team environment or microservices architecture, this is equivalent to having no schema contract at all — it is only appropriate for exploratory development topics that are never replayed.

6. Reordering Enum Symbols

Avro encodes enum values as integers (the index in the symbols list). Reordering ["PENDING","PAID","SHIPPED"] to ["PAID","PENDING","SHIPPED"] silently maps 0 to a different symbol — old messages are misread without any error.

Schema Registry supports three serialization formats: Avro, Protobuf, and JSON Schema. While Avro is the most widely used with Kafka, each format has specific strengths.

Protobuf — Field Numbers for Evolution

Protobuf schema evolution relies on stable field numbers. As long as you never reuse a field number for a different type, old and new messages are always forward and backward compatible.

Register a Protobuf schema with Schema Registry by setting "schemaType": "PROTOBUF" in the registration request body.

JSON Schema — $schema Validation

JSON Schema does not have a compact binary encoding — messages are plain JSON. It is suitable when consumers are HTTP-based services, the payload is already JSON, and you want schema validation without a separate serialization layer.

When to Choose Each Format

• Choose Avro when you have a Java-heavy ecosystem, need compact payloads, and want strict schema enforcement. Avro has the richest Schema Registry tooling and the longest track record with Kafka.
• Choose Protobuf when you have polyglot consumers (Go, Python, Rust, C++) or when the same message type is also used in gRPC service APIs. Field numbers make evolution intuitive and explicit.
• Choose JSON Schema when your consumers are HTTP microservices that already consume JSON, when human readability of the wire format matters for debugging, or when the team is not familiar with binary formats. Accept the tradeoff of larger message sizes.