Are you planning to include the concurrent files option from Otto3? We typically run 5 fmdatamigrations at a time as we have a few larger files and it let the smaller ones integrate while those chugged.
Hello,
We aren’t planning on bringing that forward. In fact later versions of migrator default to 1. This is becuase even with Otto 3 setting the concurrency above 1 could be problematic. If you were running a migration on a small server the DataMigration tool could use up all the available memory, which could lead to instability. Doing one file at a time is the best way to go.
Also OttoFMS fetches all the files at once as a zip. Which gets around one of the things that we did concurrency for. It would interleave the fetches which could save a lot of time. Now there is only one fetch.
That said we could be convinced to add it back. If people have a really compelling case for setting it above one, let us know.
Thanks
Todd
I see your point about smaller servers potentially running out of memory by running multiple at once. That feature is actually what brought me over to Otto. Our production server is 96 cores with 384 GB of ram, which puts us in a little different position. We push code weekly, and being able to push multiple files at the same time really cuts down on our downtime. We generally limit that to 5 running at any given time, more than that and the return on performance doesn’t seem to really be there.
Yes I can see how a server that big could handle it. Thats a big honking server.
Ok we’ll see what we can do about adding back an optional concurrency.
How many files do you push each week? How big are they? Just curious.
Todd
Ha. Yes, big server. Last week was 11 files and 40 GB. A couple times a year I’ll do a full integration back to my dev system which is 36 files and 167 GB as of now.
Wow that is a beast of a system. Glad Otto/OttoFMS helps you so much there. It’s helpful for people to know that even system like this can be tamed some what by using a Dev and Prod software development cycle.
We love to hear it.
Thanks
Todd
I will add my voice to that request. We have a customer that we update 4-5 times a year. Without concurrencies this will take 7-9 hours, with concurrencies it will „only“ take 3-4 hours. Since we can only start the migration after 9pm this is really a crucial factor for how long we have to stay up „just“ to watch the server. (Since sometimes it failed on 1 or 2 files due to some of them being more than 20 GB in size.
So yes, please add it back. 
And just to add to the complexities of some systems: We have our customers solution spread over 3 servers due to the fact that the files are huge and we have 600 concurrent users that need reliable performance.
Thanks for that note. I love to hear about these huge systems and what you amazing intrepid developers have to go through and still create massive value for your companies and customers.
Maybe I’ll make new posting asking people to share these stories. They are amazing.
Or maybe we can get some of you on the Context podcast to talk about this stuff. It’s awesome.
Todd
1 Like
Hey Lee,
Concurrency has been added to OttoFMS in version 4.2.0 and OttoDeploy version 1.1.0. Thanks for the request!
-Kyle
1 Like
Thanks! I’ll fully admit I’ve been trolling on the site watching all the updates. Did the update last night, ran the command via api and can confirm it works as expected. Only ran into one snag, but I’ll make a clean post for it so we don’t tangle up this thread.
1 Like