I’ve been using OttoDeploy for a while now. It’s an amazing tool which we saw a community are very fortunate to be able to take advantage of for free so thank you for sharing this amazing tool.
I’ve been trying to run an update this week from Dev to Prod. I’ve been starting it at around 2330 and at 0630 it’s still running. Looking at the detailed log it seems to be getting stuck on the same table. However, even deleting the 2k records from this table does not solve the problem.
Via email, Angelo suggested trying a consistency check and compacting the file. I did both of these yesterday before retrying the migration but still seeing the same issue. The file is 26Gb (post compression) and I have the latest version of Otto installed on the server. Screenshot shows where the migration seems to hang for over 4 hours. This seems to be consistent across the 5 migration attempts I’ve made this week.
Wondering if you had any suggestions on what might be causing this?
You don’t say this message if the migration ever completes or not. Does it?
One thing to note. You can see right on your screen shot that one of the tables is forced to go into Record Mode. This means it can only process one record at a time. In that case the migration for the table will be many times slower than the otherwise. The Data Migration Tool itself determines if it has to go into Record Mode. There is nothing you can do to fix, because nothing is broken. It is just doing what it has to do.
Record Mode is not a permanent thing. Each time you do a migration the DMT checks each table and decide what mode it needs to use. Block Mode = Very Fast, Record Mode = Slow
If your migration doesn’t complete or seems like it might take forever, there are two causes that make up most of the cases we see.
An under-specced server. Data Migrations take a lot of resources. If your server is too small it can take a very long time. Related to this is AWS T-Type Burstable Severs. Basically do not use them. They will never work well enough.
Corruption. You either have corrupted data or corrupted schema. When the Data Migration Tool hangs it does not report the table it is on, because well, it is stuck. It can’t do anything. The migration log will tell you the LAST table it was able to get through.
You can look in the otto-log file of the server that is doing the deployment and see if it has been forced to restart by the DMT encountering corruption, during the time of the migration.
Use Recover to check both files, and if you find corruption you’ll want to deal with it.
Hi Todd. Thanks for the feedback and suggestions. I was able to complete a migratino last night using Lesterius’s FMDataMigration tool. It was on a freshly compacted copy of the file which was 25.98Gb and took just under 14 minutes so I have at least got the update out to users.
I’ll try and run another update using OttoDeploy on a copy of both this newly updated file and a copy of the previous version and see if they complete. Unfortunately with the previous attempts I had to abort on each occasion as I needed the file back online for people to use and had run out of time.
I’ll have a dig through the files and see if I can identify where it was getting stuck.
I ran them on the same server. It’s a Mac Mini server (Mac mini G5N - AS/M2/8C/10C/GPU/16G/1T/SSD/10G) so plenty of processing power. I followed the steps below
Created a clone of my new update on my Dev server and copied this onto prod
Made a copy of the source file as a backup
Saved the current source file as a compacted copy
Pointed FMDataMigration at the compacted source and clone files
Ran the migration. In FMDataMigration I had the attached settings
