Dive deep into database migration services AWS DMS and AWS SCT

Migrating databases to AWS is a big topic - but there's good tooling provided by AWS , Database Migration Services(DMS) and Schema Conversion Tool (SCT) Read more →

How to review AWS Database Migration Service for successful DB migration

Starting to review AWS Database Migration Service Read more →

How to master KSQL with cheat-sheet

KSQL is the the streaming SQL Engine for the real time data processing of Kafka. These are some of the commands --check KSQL is running ps aux | grep ksql --check KSQL error logs (if using Confluent) confluent log ksql-server --list created streams LIST STREAMS --View the structure of STREAM DESCRIBE mySTREAM_NAME --display all topics SHOW TOPICS --display values in topic PRINT 'topic_name' Read more on Apache Kafka How to Rock Apache Kafka with a cheat-sheet Read more →

Which one to use - Apache Kafka versus RabbitMQ

Best case scenario (use case) for Apache Kafka 1) Requirement for partitioned order at least once write. Requirement for A to B streaming with high throughput. 2) Durable message store . Why would this be important? Consumers can get a replay of the event stream - whereas limited in message brokers - where once message is delivered , removed from queue 3) If a solution requires data stream processing. 4) Event sourcing required. Event sourcing persists the state of a business entity e.g maintaining a chain of event at different states in a sequential order for a Customer 5) More... Read more →

Apache Kafka FAQ

These are some common questions I've received in working with Apache Kafka. I'll keep adding more detail What is Apache Kafka ? Apache Kafka is a distributed , partitioned, replicated commit log service. Apache Kafka can also be described a Publish - Subscribe message system for distributed applications. What was the original purpose of Apache Kafka ? Apache Kafka was developed within LinkedIn. LinkedIn invested in developing a single, unified , distributed pub-sub data pipeline. Prior to Kafka - LinkedIn maintained multiple data pipelines , such as inmail messaging , site events e.g site views and other operational data. Kafka... Read more →