Follow dba-ninja.com

Subscribe to RSS feed  Follow @jackvamvas - Twitter

*Use the Comments section for questions

dba-ninja.com Links

Dba_db2_button

Sqlserver_dba_button

What is the Avro data format

30 August,2019 by Rambler

Question:I've been specifying the Avro data format through Kafka - but I'd like to understand the format in more detail and also understand why you would use Avro .

Answer: Avro is an Apache open source project which supplies data serialization. Avro supports exchanging data for Apache Hadoop. Avro is very useful in situations of different programs using different languages, exchanging data. When Avro serializes  , Avro stores the data definition and the data in one file.

Avro supports Schema evolution i.e data schemas changing over time is a key component of Avro. 

Why would you use Avro ? Once you think about the Avro principles it make sense to consider Avro for certain circumstances. 

 > Data structures changing over time - aka rich data structures

>  Remote procedure calls - e.g Data transfers over a remote system and avoiding the java serialization overhead

> Persist data containers where the data schema and data do not need to be separated

> Avro binary output is smaller than Java , which supports a more efficient transfer ratio across remote procedure calls


Author: Rambler (http://www.dba-ninja.com)


Share:

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment on What is the Avro data format


dba-ninja.com