"Deployment crashed" even though it worked

OttoFMS
1

I ran a deployment overnight last night. The last entry in the detail log says “Deployment crashed”, but as far as I can tell, everything worked normally. The DMT ran and finished without error. The post-deployment script ran and finished without error.

The deployment status is “crashed” and there is a “resume deployment” button in the OttoFMS portal. Any way to find out what Otto thinks the issue is?

I can send you the log if needed. Lmk.

2

Hi Mislav,

It might be possible that something happened at the very end of the deployment, after everything was done, but before the deployment status was set to “done”.

What version of OttoFMS are you using? If it the latest version 4.5 you can download the debug info for the deployment. It’s in the upper right under the download button.

TG2024-07-31 at 06.39.36

If you are on an earlier version you can download everything separately , and then also include the otto-info.log.

Send all that stuff to us via a DM or message.

We can take a look

Thanks

Todd

3

Todd, thanks for your response.

I’m on v4.3.4. I’ll send you the info via DM.

4

Hi

It looks like OttoFMS might have timed out waiting for the Post Deployment Script to complete. You mentioned that it completed. But do you know how long it took to run.

You can look in the event.log to see if the script completed.

If that is the issue, I’d like to at least make sure we can log that correctly.

Thanks

Todd

5

It looks to me like the script completed.

Info from Event.log:

2024-07-31 02:28:36.732 -0700 Information 10 <<host>> Schedule "OTTO_Deploy to PROD | Pos_gbBBk" created by "<<admin>>". 2024-07-31 02:28:36.747 -0700 Information 146 <<host>> Schedule "OTTO_Deploy to PROD | Pos_gbBBk" run now by "<<admin>>". 2024-07-31 02:28:36.747 -0700 Information 689 <<host>> Schedule "OTTO_Deploy to PROD | Pos_gbBBk" has started FileMaker script "Deploy to PROD | Post-Processing". 2024-07-31 02:31:00.370 -0700 Information 150 <<host>> Schedule "OTTO_Deploy to PROD | Pos_gbBBk" completed.

Also, the post-deployment script finishes by doing these two things:

  • It sends me an email, which I did receive. In the email, the script reported that it ran for 2 minutes and 20 seconds.
  • It tells Otto that everything was successful by creating the otto-pms-result.txt file with “done”. I configured the deployment to revert if this success message isn’t sent, and it did not revert; i.e. the migrated files were hosted in production.
6

Ok , well all I can say at this point is that the OttoFMS timed our waiting for the script to finish. It didn’t handle that as well as it should have. That caused OttoFMS to restart and marked the deployment as crashed.

We can probably improve the handling of that problem so it doesn’t cause the deployment to crash.

Thanks for reporting it. Look for a fix in a future version.

Todd

7

Hey Mislav,

I’ve been looking into this with Todd. It looks like we fixed the issue causing the crash in OttoFMS version 4.5.0. Upgrading to 4.5.0 will stop this issue from crashing OttoFMS, but your script will still timeout, as we do not currently have a way to override the post deployment script timeout time. I will be upping the default timeout to an hour and adding an override env variable which will allow management of this timeout time.

Thank you for the report, the update including the new timeout time will be coming out in the next couple weeks, keep an eye out for OttoFMS 4.6.0 for the fix.

-Kyle

8

Excellent, thanks for letting me know.

9

Hey Mislav,

OttoFMS version 4.6.0 includes the fix to the timeout times for the scripts. you should be able to run this without issue after upgrading!

-Kyle

1 Like