Slow transfers still persist

OttoFMS
1

I am doing a 7 file build and deploy about 280mb the build went fine many hours before the migration 1 file was a replace the rest was migrated.

Server A to Server B on OCC still take way too long to transfer the data between them.

I am able to download the build to my computer and upload the build faster then Otto can transfer the files on the OCC infrastructure.

And of course as I am typing this (because I am not doing anything else) the deployment crashed 3 times during the transfer.

Using FileZilla server to server via RDC was the only remedy.

I am just documenting my bemusements.

Humbly Stephen

2

Hi Stephen,

The slow transfers are a windows issue as you know, and has been well documented at this point.

We would like to know more about the crashes. Could give us the usual information about the systems involved and the logs?

Please include the following.

Version of FileMaker Server
Version of OttoFMS
Operating System
Deployment Results and Log
otto-info.logs from both systems.

This information would be excellent documentation to add to your bemusements

Thank you

Todd

3

Hi Todd,

can you please tell me what makes windows so slow? And is it the source or the target server that makes it slow?

Thanks, Tobias

4

Hello Tobias

It’s Windows IIS on the source server.

We cover this here. Handling very large files on Windows servers | OttoFMS

Thanks

Todd

5

Thanks for the clarification -

I have no idea if it’s possible or not could webRTC be option here as a vehicle to transfer data between servers?

I was playing with https://wormhole.app the other to send a file to a client.

6

Hi Stephen,
An IIS upload might be slow, but an IIS download has normal speed. I think this a possible scenario for a fix. I have created an extra web site on the dev site, on some very non-standard port. The web site points to the OttoFMS outbox. So no extra software had to be installed, I’m just doing a build phase, followed by a deployment phase using a url to that web site, (with a UUID in the path so I do not have to use a password, my setup is lan-only anyway).
Just letting you know how I’m going around the issue until it can be resolved. – peter

7

That is a very nice elegant solution. I wish I had thought of that :slight_smile:

We need to take a closer look at that.

Todd

8

Just to clarify this a bit.

This works because you are setting up a second website in IIS to statically serve files out of that Outbox folder. Static file serving works fine. What is slow is downloading through IIS to a node application like OttoFMS is. It’s the reverse proxy that causes the slow down here.

So this gets a round that problem. We are going to see if we can do with this idea.

Todd

9

So the part where I think the upload is slow, is wrong. The target server is downloading, but on the source server end, IIS has a literally lame reverse proxy system. I’m just not using the reverse proxy mechanism.

10

Right exactly.

We have something coming next week, that we think will solve this problem in most cases. But there may still be some cases that hit the issue. For those… this very simple solution will be a good work-a-round

Thanks

11

Hey y’all,

OttoFMS version 4.5.0 includes the new “pushBuild” option which helps with this issue. You’ll need to make sure both your source and destination servers are upgraded to OttoFMS version 4.5.0, and then use the “push Build” option in OttoDeploy (available in version 1.2.6).

The new option pushes the build straight out from OttoFMS rather than needing to go through IIS, so we bypass the issue entirely. This will not fix slow downloads of files from the File Manager, but it should cut down on deployment times significantly. In our tests this took the transfer of a 30 GB file from a very long time (was not even 1/30th complete after an hour) to 10 minutes.

There is documentation of the new workaround on the Large File and Windows page in the docs.

-Kyle

2 Likes
12

Thank you for this!!! So much less tedious then having to RDC and SFTP files - Bravo :clap:

1 Like