It would be great to have a feature to exclude external container files when creating builds or update/replace deployments.
I couldn’t find a place to exclude them (let me know if I’m missing something), and not realizing they were being copied ended up filling up my disk.
We want to be able to copy files from production to staging after a migration is complete and local tweaks have been completed, but we almost never need the container data and including it can be huge amount of overhead.
I am not sure I understand containers are not transferred in migrations?
Hi Ken
Welcome to the community 
Currently, in order to make a copy without closing the file, OttoFMS runs a backup, and because that backup is not part of standard set it moves the containers into the file.
You can change this behavior by moving your data bases to one of the additional Database folders, and setting up an additional container data folder, and finally disabling backups on that folder. See image below.
We are thinking of letting you just get a copy of the file with out using a FileMaker backup. BUT we would have to close the files, make the copy, and then re-open it. It’s more complicated and would kick people out of the files. but it would let us avoid this problem.
Hope that helps
Todd
Thanks Todd. I appreciate your quick response.
I’m actually using additional folders in my backup, but we chose to back up the container files as part of the standard backup. I’m still discussing that option with the client, so this may be a reason to change that option.
I haven’t experimented with this - can a backup created by the Admin API exclude containers when the standard setting includes them?
In this use case, I’d probably be willing to close the file in order to back it up. I’d just have to schedule it for late at night. Of course, a backup with just the db file would be ideal.
I just did some experimenting on the server and it seems like an all-or-nothing proposition. If I want to backup containers, I need that feature enabled. In which case I can’t use your wonderful tool to retrieve files from production. This has become a standard part of our workflow (manually copying files from prod to staging after migrations have been completed, sometimes after tweaks are made on production).
I was very excited about using Otto for this part of our workflow because that step of copying back to staging is always a pain. But (at least for this client) the size of the container folder makes this a non-starter. Your solution of grabbing the file from the live hosted may be the best option. If I could run a scheduled script to move a copy of a backed up fm file (sans containers) to specific location, would OttoDeploy be able to access it? Could I use a build for something like that?
Hi Ken,
I have an idea that we could try. I’ll have to do some experimenting. But currently the only way OttoFMS can do it is the way I described above.
You could back up the alternative containers folder to s3 using OttoFMS
Thanks
Todd
Saw you just had a new release. Thought I’d check in and see if you had any more thoughts about this. A great stopgap would be a what you mentioned earlier - the possibility of closing the file on the server and copying just that.
Thanks
Ken
HI Ken,
I think this is not really an issue. When I went back and looked at this problem, I found that external containers were not ever brought into the file.
So could you go back and confirm that this is really an issue for you.
Todd
The issue is not that the containers are brought into the file. The issue is that a backup is run, including all the containers. If the client has thousands of large containers, they’re all copied. For this client, this takes a huge amount of time and disk space. I don’t know if Otto also tries to copy the containers to the staging server as part of the copy script after the backup is complete. I haven’t gotten that far - the Otto script eventually times out.
Right, ok Got it
This version does not address that issue.
Thanks
Todd
Alright thanks Todd.
I don’t suppose there are any hooks to run a shell script or something, Is there? Then I could copy a file from a previous nightly backup or something.
I was so excited for Otto to address this part of our workflow - the alternative is to manually log into the server using RDC and copy the file back to our production server using a web service or desktop mount or something.
Otherwise, Otto’s working great for our migrations. Thank you for such a great contribution to the community!
Ken
Hi Ken,
Sorry, the only way to avoid making the backup of external containers is the method I already mentioned. Use alternative folders and backup the containers using OttoFMS’s offsite backup feature.
We will look at addressing this in the future.
Thanks
Todd
+1 from me for this feature request. I am perfectly fine with the production databases being closed in order to prep the file without container data as opposed to a backup.
My use case is much the same as @kdoronzio . I am deploying a new release to Production, and when done, I want to be able to pull it back to development (or anywhere for that matter). I really don’t care about external container data for my development copy, so currently I can’t use otto for this process as the transfer of data takes too long.
I’d love to simply be able to copy the files only. When I do deployments there aren’t users connected to production anyway.
I did also note another feature request that woudl be very useful for this particular situation also, and that is to leave the deployed files closed on the destination after a deployment. If that were an option, then once deployment is finished you could easily move that copy (minus containers) back to development, either as a new deploy, or a sub-deployment.
Yes it’s a very similar use case. In our case, we sometimes make additional tweaks or fixes to production after deployment. So the originally migrated copy wouldn’t usually work for us.
+1 from me on this use case - It would be great to copy files back to the dev server from production without the container data.
Hello everyone
This is on our roadmap for an upcoming release. Stay tuned.
Todd
Hi Todd.
Just bumping this request - any movement on this?
Thanks
Ken
Hi Ken,
It was addressed in version 4.3.3
Specifically
- added the closeFiles option to the build input, allowing copies to be closed before running, speeding up the build dramatically.
Since the files are closed we don’t have to run a backup. Since we don’t run a backup external containers are ignored.
Thanks
Todd
Thought I replied.
Thanks so much for adding this. I haven’t tried it yet, but I will likely need to in the next week or so.
Ken
Hi Todd -
I tried this a couple of ways and I must be doing something wrong.
I updated to the latest and greatest on both servers, then I set up a JIT Install/replace and selected “Close files during build”. The entire file with all of its containers are still being backed up to an Otto folder. Then I’m getting timed out on the process. With containers it’s way to big, even if it was a separate build process.
Do I need to a separate build first? If so, can those be scheduled? If I’m closing files on the server, I have to schedule them for overnight.
I can still copy files back manually, and the Otto file manager is still easier then it used to be with multiple tools. But I’d love to automate it if I could work out how to do it without containers.
Thanks
Ken
