Follow dba-ninja.com

Subscribe to RSS feed  Follow @jackvamvas - Twitter

*Use the Comments section for questions

dba-ninja.com Links

Dba_db2_button

Sqlserver_dba_button

How to use Inline LOB for DMS to improve speed of an AWS DMS task

07 August,2023 by Rambler

The AWS Database Migration Service (DMS) has methods for moving LOB columns from one database engine to another. The  functionality to manage the LOBs is part of the Database Migration Task

There are 3 primary modes for LOB migration . Full LOB , Limited LOB & Inline LOB

 

Full LOB

When using Full LOB mode, AWS DMS migrates LOBs whatever the  size. Because AWS DMS doesn't know the size of the LOB data that needs to be migrated, AWS DMS migrates LOBs one at a time -  row by row to the source table. Depending on the amount of LOB data , there can be a significant impact on performance . Although there can be a performance hit , causing the migration to be slower , there is the benefit of data not being truncated. 

Limited LOB

When using Limited LOB mode, configure the task  the maximum size of LOB column data. DMS will pre set the required resources and apply a bulk process. The disadvantage of this method is if the database LOB column size exceeds the size detailed in the task - AWS DMS truncated the data.  You'll need to review the AWS DMS log files .    Also checkout  AWS DMS Context Logging for more contextutal logging 

Inline LOB

When using Inline LOB mode, you can migrate LOBs without truncating data or slowing your task performance, by replicating both small and large LOBs.The AWS DMS task transfers the small LOBs inline, which is more efficient.

Then, AWS DMS migrates LOBs that are larger than the specified size in Full LOB mode by performing a lookup from the source table. However, Inline LOB mode works only during the full load phase.

 

To view the current settings in the Replication Task you can view either through the interface or use AWS CLI command

aws dms describe-replication-tasks --query "ReplicationTasks[?contains(ReplicationTaskIdentifier,'<replace_with_task_identifier>')].[ReplicationTaskSettings]

For more information on AWS CLI commands refer to AWS CLI Cheatsheet

To make the changes and initiate the Inline LOB

Open your DMS task , & use the JSON editor mode 

First, specify a value for the InlineLobMaxSize parameter, which is available only when Full LOB mode is set to true

to set up - 

Place LOB setting in Full LOB mode 

Set Inline LOB size 

{
"TargetMetadata": {
"TargetSchema": "",
"SupportLobs": true,
"FullLobMode": true,    ==> Set to true
"LobChunkSize": 64,
"LimitedSizeLobMode": false,
"LobMaxSize": 0,
"InlineLobMaxSize": 32,  ==> Set the max size 
"LoadMaxFileSize": 0,
"ParallelLoadThreads": 0,
"ParallelLoadBufferSize": 0,
"BatchApplyEnabled": false,
"TaskRecoveryTableEnabled": false,
"ParallelLoadQueuesPerThread": 0,
"ParallelApplyThreads": 0,
"ParallelApplyBufferSize": 0,
"ParallelApplyQueuesPerThread": 0
}

 

 


Author: Rambler (http://www.dba-ninja.com)


Share:

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment on How to use Inline LOB for DMS to improve speed of an AWS DMS task


dba-ninja.com