30 August,2019 by Rambler
Question:I've been specifying the Avro data format through Kafka - but I'd like to understand the format in more detail and also understand why you would use Avro .
Answer: Avro is an Apache open source project which supplies data serialization. Avro supports exchanging data for Apache Hadoop. Avro is very useful in situations of different programs using different languages, exchanging data. When Avro serializes , Avro stores the data definition and the data in one file.
Avro supports Schema evolution i.e data schemas changing over time is a key component of Avro.
Why would you use Avro ? Once you think about the Avro principles it make sense to consider Avro for certain circumstances.
> Data structures changing over time - aka rich data structures
> Remote procedure calls - e.g Data transfers over a remote system and avoiding the java serialization overhead
> Persist data containers where the data schema and data do not need to be separated
> Avro binary output is smaller than Java , which supports a more efficient transfer ratio across remote procedure calls
This is only a preview. Your comment has not yet been posted.
As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.
Having trouble reading this image? View an alternate.
Posted by: |