Unfortunately, we have had issues affecting Kalix's performance over the past week, including several downtimes. For the timeline, please see Kalix's Status Page.
We want to offer our sincere apologies for the reoccurring problems. The downtimes were the last thing we wanted to happen.
Subscribe to Kalix's Status Page
To receive automatic updates on incidents affecting Kalix, we recommend subscribing to Kalix's Status Page by following the link and clicking Subscribe.
We completed a significant infrastructure update last Sunday to improve the security of Kalix and support HL7 in the future. Extensive testing over several weeks prior to the update was completed. Unfortunately, it can be near impossible to predict everything that will occur when the program is being run live on a large scale.
Unknown errors were occurring in Kalix, resulting in one or all servers becoming unstable (or sick). To recover, the serves can automatically reset, which can cause slowness or even downtime. Documents having the highest workload were the worst affected.
Please be assured we worked tirelessly to resolve the underlying issue causing the downgrades. We thought that we had solved the problem several times throughout the week, but later finding out this is not the cause. Since Kalix is such an extensive program, the identification of the triggering event can be very trickly.
Yesterday, we believe we made a major breakthrough. We hope we successfully identified and released a fix for the underlying error causing the downtimes or slowness.
The error was happening outside of Kalix, from our faxing provider. It occurred when Kalix customers sent faxes containing images. Whenever the errors occurred, Kalix's servers reset, causing downtimes or slowness.
Two other events happened some weeks ago, one was due to a problem with our cloud provider, and another one was server crashes due to some code that created an infinite loop. These two issues are also entered into our status page if you would like further information.
We are deeply distressed and very apologetic that there have been so many issues, especially since it may be affecting our customers' business. Historically, Kalix is a very stable program, and we usually achieve a 100% uptime each month. The last few two months have been the exception, not the rule.
Again our apologies; please reach out if you have any questions or issues.