Tuple Serialization

The tuple is Heron’s core data type. Heron’s native Tuple interface supports a broad range of basic data types, such as strings, integers, and booleans, out of the box, but tuples can contain values of any type. You can use data types beyond the core types by providing a custom serializer using the instructions below.

Kryo

Heron uses Kryo for tuple serialization and deserialization. You can create a custom tuple serializer by extending Kryo’s abstract Serializer class. More information can be found in Kryo’s documentation.

Registering a Serializer

Once you’ve created a custom Kryo serializer for a type:

  1. Make sure that the code for the serializer is on Heron’s classpath.
  2. Register the class with Kryo using the topology.kryo.register parameter for your topology. Here’s an example:
topology.kryo.register: - biz.acme.heron.datatypes.CustomType1 # This type will use the default FieldSerializer - biz.acme.heron.datatypes.CustomType2: com.example.heron.serialization.CustomSerializer

Once your custom serializer is on Heron’s classpath and Heron is aware of its existence, you must re-compile Heron.