"Deployment crashed" when trying to refresh DEV

OttoFMS
1

I tried running a deployment to refresh my DEV environment with files from PROD; i.e. the operation is set to ‘replace’ instead of ‘migrate’.

The deployment ended up crashing. I’m not entirely surprised due to the size of the files. There are 7 files totaling 15GB. The largest file is 6GB.

Would you expect the ‘replace’ operation to work with larger files?

Even if it were to work, how long would you expect it to take to transfer 15GB worth of data over a fast connection? The log shows an estimate of close to 24 hours.

Here is the deployment log csv:
message,phase,level,timeElapsed,file deployment queued,queued,info,00:00.0, "Starting deployment: ""Refresh DEV""",starting,info,00:00.0, 7 files,starting,info,00:00.0, Starting to close files,closing,info,00:02.9, Files closed successfully,closing,info,00:03.4, Sending just in time build request to source server,building,info,00:03.4, Just in time build started on source server,building,info,00:03.6, "Build running, completed 1 of 7 files",building,info,03:07.6, "Build running, completed 2 of 7 files",building,info,03:15.7, "Build running, completed 3 of 7 files",building,info,04:53.7, "Build running, completed 4 of 7 files",building,info,06:39.9, "Build running, completed 5 of 7 files",building,info,07:12.6, "Build running, completed 6 of 7 files",building,info,07:37.1, Just in time build completed on source server,building,info,09:06.9, Downloading source files for buildId build_XYZ,fetching,info,09:10.1, Downloaded 0 of 15 GB. 1.9% completed. Estimated time remaining: 23:33:37,fetching,info,36:46.2, Deployment crashed,unknown phase,error,36:51.2,

2

Hello Mislav,

Are you pulling these files from a Windows server? Which versions of OttoFMS are running on your Prod and Dev servers?

Normally we would expect the replace option to work properly, although we are seeing some issues with fetching from Windows servers (see docs for more info). This looks like a problem with fetching from the source server, not with the rest of the process.

Could you DM me the otto-info.log file of the dev server you were running this deployment on? I’d like to take a look at the error that caused the crash. Thank you!

-Kyle

3

Hi Kyle, thanks for responding.

  • Yes, Windows server.
  • OttoFMS version 4.2.3 on both servers.
  • I’ll DM you the log files.
4

Hmm ok it looks like your process got terminated. Usually this happens when the Operating system decides that you’ve had enough (enough memory, enough cpu, etc.) and cuts you off. How big is your dev server?

I suspect that your issue would be solved by a fix to the issues caused by Windows IIS, which is what I sent you information about above. Could you try building ahead of time and either 1: putting the build into the inbox on your dev server manually, or 2: putting the build up on an accessible url to be pulled down using the url source option. For option 1 I would recommend storing the build somewhere else on your server and copying it to your OttoFMS Inbox, as builds are deleted from the inbox as part of the deployment process. It will be faster to copy from on the server again rather than uploading the build again.

We’re continuing to research methods of getting around the issues caused by the IIS limitations, we’ll keep you updated on progress.

-Kyle

5

The DEV server is a t3.large.

Thanks for the troubleshooting suggestions.