Failed webhook tests

on a test server, suddenly getting this error message
Failed Message not published. Error: 20603. . code: 20603

any idea?

Hey John,

Do you see anything in the otto-info or otto-receiver log files at the time the webhooks are being sent?

-Kyle

Nothing untoward, it just started going this responseā€¦

Hereā€™s what I did
This is a test server on MacPro for an in-house I am mentoring so he can play along without doing stuff anywhere evener real, and he is learning about webhooks as the use Jot a lot

I restarted Otto, no change
We restarted server, no change
I downloaded copy of the receiver file
Cloned it and opened it to check the file
Uploaded it with new name and set up an API key with my credentials, sent a test, al worked
Imported all the records into the clone, removed the previous version and uploaded this with original filename
webhooks failed and I was unable to add a new api key, didnā€™t have time to take a screenshot as I was rushing out.
Now I am back in and webhooks are working for the last hour and now I can add an API key, which is really rather frustrating.
Didnā€™t do anything in the meantime so that is a bit of a mystery.

Will obviously keep an eye on it though

I left the building just before 3
The webhooks started again at 3:16, which is suspiciously like 15 mins after as if somehow the token got broken, and once it renewed all was fine after what I had done

And having said that a few minutes ago it stopped again. Once it has failed then testing any other webhook also fails

and this is what I see when trying to add an API key

am going to download and see if a recovery does anything

1 Like

@kduval
in the light of yesterday, is there some way that you can provide that can ā€˜deleteā€™ all dapi tokens?
If they are required by a webhook, then if they do not exist then they will be re-instantiated, its just if for some reason it is broken and another webhoook comes in within the next 15 mins it will try again with the key that is not workingā€¦

Hey John,

There is an API to log an API Key out of a file, that might work for you.

I suspect what is happening is that OttoFMS is not receiving an ā€œUnauthorizedā€ response from the Data API, which means it will simply return the error rather than attempting to refresh the token and try again. Did you find anything when trying the recover on the file?

-Kyle

No clean as a whistle
Thereā€™s only the receiver table and one to store the names and id of the webhoooks, but I donā€™t think thatā€™s even used in that copy of the file

John and I discussed this in DMs so we could exchange some logs. It looks like the error was fixed by exchanging the erroring API Key for a new API Key. If you see this issue as well, please report it so I can collect more info on what is causing this issue!

-Kyle

For additional info, part of the issue is that for the 15 min gap after the error has occurred you are unable to create a new API key, as it thinks the file is corrupted (which is was not, and then later was successful) - and this may be longer if webhooks come in during the 15 mins, although they will all fail

1 Like

@kduval
Hmm just happened again today
Here is the last part of the log

What is the opportunity here for any kind of notification outwards of this error even?
Otto woudl seem to be running, and the server
otto-info.log.zip (1.5 KB)

Hey John,

its weird because the 20603 error seems like a network problem or like the Data API is crashing somehow. Does restarting the Data API fix it?

-Kyle

These seems like it might be related to
https://community.claris.com/en/s/question/0D53w00005KWfAzCAL/web-publishing-engine-error-code-20603

Very similar

Error 20603 is lost communication with the host/guest
which is more odd given that Otto is on the same physical machine as FMS
in this case also WPE is off so am not getting a wpe0.log

I am wondering if there is a way of detecting this error and invalidating the token and starting afresh?