We have created an API process which copies a file from server A to B. The intention here is to isolate the production server A from ‘expensive’ ODBC and OData queries that are creating performance issues on server A.
For more information, you could check out this topic.
The process used to work great until recently. We are now getting a lot of deployment errors:
1. Timeout
building Build server timeout: Timeout exceeded
Is there a timeout? Is there a way to edit that?
2. Build already in progress
building Build server error. http Status: 409. FMS Error 18000: build already in progress
My guess is that this happened due to a non-graceful restart we had to do when a deployment was stuck. The error is incorrect, meaning there was no deployment at the time I initiated this. Is there any way to clean up the ‘stuck’ deployments, so OttoFMS can start operating again?
3. Too many sessions
building Build server error. http Status: 401. FMS Error 18000: Invalid auth token: Too many admin sessions
I am not using Admin API for authentication, I am using an OttoFMS api key, so Otto can handle that session lifetime for me. Is there an issue there?
Lastly, we are on version 4.8.1.241217468, which of course is not ideal. I have asked our IT team to upgrade to OttoFMS latest version, just to eliminate any issues that might have already been addressed in latest releases.
The “Timeout exceeded” error on the build server means that your Just In Time build took longer than the allowed time during a deployment, which is 30 minutes. You should check the otto-info.log on your source server to see what is causing the build to take so long. My suspicion is that a backup is getting hung and causing the build to then hang as well. This will be (hopefully) fixed in the version I’m releasing in the next couple days as we move away from the admin api for backups.
Your second error there is because of your first error. Since your build hung, when it tries to start a new build it cannot, as OttoFMS can only run one build at a time. Restarting OttoFMS on your source server will fix this issue, and there are numerous fixes for this in OttoFMS version 4.10.0 that should prevent this issue.
We added some fixes for session leaks in OttoFMS in some of the last few versions, so hopefully your admin api session limit should not be a problem. However, if you do hit that again, the immediate fix is restarting the FMS Admin Server, which will clear existing sessions (don’t worry, OttoFMS will handle this gracefully).
tldr: Upgrading OttoFMS will probably fix all of this
We have upgraded both servers to version 4.10.2.250318454
We successfully deployed 1 file, but we weren’t able to deploy the main file. Here is the error we get:
queued Deployment queued
starting 1 files
pre-deployment script No pre deployment script to run
closing Files closed successfully
building Just in time build completed on source server
fetching Source files fetched successfully
installing Incoming file not found for install at /opt/FileMaker/FileMaker Server/Data/Backups/OttoFMS/inbox/jit_build_xxxxxxxxxxxxxxx/Filename.fmp12
reverting Error reverting deployment: Backup folder does not exist: /opt/FileMaker/FileMaker Server/Data/Backups/OttoFMS/deployment-backups/deploy-576/sub-deploy-576
post-deployment steps Keeping builds in inbox since deployment failed. They will be deleted when this deployment is rerun successfully or in 48 hours.
opening Files opened successfully
done Deployment process failed. Original files are unmodified.
If you look at that build in the OttoFMS inbox (you can find this in the file browser), does it have the pertinent files there? Are there any .fmp12 files there, is the .build.zip file there? Also, which operating system is this server?
A clean OttoFMS install is not going to hurt (as long as you aren’t making heavy use of api keys or webhooks), but there shouldn’t really be anything that causes this that would be in the database.You could try clearing out the OttoFMS inbox to see if that helps.
In the source server file browser, there are no items in the inbox
In the target server file browser, there is one folder that starts with jit_build_xxxxx
Within that file that is a file called manifest.json
In the manifest.json, I can see the information of a deployment ran 2 days ago that was failed due to backup failed: Filename.fmp12. Error: Local Admin API error: fetch failed; SocketError: other side closed; Path: /fmi/admin/api/v2/schedules/16
I would try deleting that build from the destination server inbox. I would also try updating to 4.10.3, it should fix the error with the Admin api on backups. It doesn’t look like it is using the one from the inbox. How long did the files take to download during the deployment?