Blog Post

There is no final word. We live in a universe that endlessly cycles space-time, so there can be no end word. It can’t even exist for what are the customer’s requests. Nothing will stand between him and progress, not even a simple web app that can work without the web, but which isn’t an app.

Perhaps service workers were born precisely because creation is never satisfied with what it does.

What could ever go wrong if my web app, born based on TCP and optical fiber, now has the ability to work offline too?

Here are some thoughts:

1. Data caching and cache invalidation
   The data can no longer be retrieved from the server, but coexists on the clients. This means that they may differ, as they are in two different places. You will therefore have to take care of keeping them synchronized, both when they change on the client and when they change on the server.
2. Authentication
   Being offline I have no way to validate my JWT, or have no way of knowing until when it might be valid.
3. In different scenarios it may happen that the same resource can be modified by two different devices and in different ways. I may have a document that I edited from my mobile on the train this morning, I got home and edited it from my laptop without realizing that the new changes are missing, what will happen when the mobile tries to sync the changes? A merge conflict for our poor user?
4. In some cases the connection may not be completely missing, it could be the classic faint connection that is in the back left corner of the office. Some requests may go through and some may not. It is not easy to understand if it is possible to retry a failed request or to wait because the connection is almost non-existent.

Just reflecting on all this, I came across an interesting article that described this way of working to support offline in a webApp with what they call modifiersQueue. I wanted to give my interpretation.

Basic Concept

It is assumed that in the application every user interaction can be expressed as a modification, large or small, to the data that make up the system. Whenever the user takes an action it is represented by an instance of the Modifier class. This instance will then be inserted into a queue and will then be used to apply the changes to the data stored in the device (to ensure offline operation) and those centralized on the server.

In fact, there isn’t just one modifier class, but as many as there are operations for which you want to ensure offline operation. The Modifier class can therefore be defined as abstract (like an abstract factory). In the article he therefore speaks of a class composed as follows:

abstract class Modifier{ 

modify(args) 

persist(args) 

}

Persist would be a method that applies the modifier on the server side, modify instead applies it to the data available offline on the client side.

The article says that modify is a pure function, i.e. a function that always behaves in the same way with the same parameters, and that persist is idempotent, which means that this can be called 1 or more times arbitrarily without obtaining side effects.

These two properties make it possible to consider a Modifier independent from the rest and capable of retrying several times in case of failure.

Modifier queue

When I create an instance of a Modifier I have to put it in a queue, which operates in FIFO mode: the modifiers must be executed in order.

Therefore, there must be a mechanism by which this queue is consumed, and the modifier that each device has arrived at must be known.

In practice it is necessary to know the status of a modifier: if the modify has been applied, if the persist has been applied, if the persist has failed and if it has already been retried.

It should be noted, as reported in the article, that the modify is an operation that has nothing to do with the network and that it is unlikely that it will have a negative result, unless of course there are internal failures of the device or errors in the code. In fact, a modifier therefore has no sense of existing if its modify has not been launched, one less state to consider.

Data sync

ModifierQueues are an interesting method to be able to manage offline, it is as if we kept a journal of the actions performed by the user, then used it to make sure that these actions actually reflect both on the UI and on the data in the cloud.

What is missing in the article in my opinion is a method that uses modifiers to synchronize data: invalidating the local cache and updating it, synchronizing other devices to the same data, managing the online back-up of a device.

In practice, the article theorizes the way to synchronize data from the client to the server, not vice versa.

The possibilities are these:

1. The client asks the server for the latest data for a certain resource, deletes the local ones and replaces them with the central ones.
2. The server sends the client a message with new data for a certain resource, the client replaces its own data with the received ones.

In both cases the sync is independent of the modifier queue: it is possible to synchronize a resource regardless of which modifiers have been applied previously or will be applied in the future, provided that in the end the data is consistent with those on the server.

I decided to call the sync described in point 1 hard sync, the one described in point 2 soft sync.

There are some observations about both:

1. In case of hard sync the server load is higher. To be able to sync data, you need to search for it, serialize it and then send it, every time it changes.
2. In case of hard sync it is difficult to work with large amounts of data. Every time the server is asked, the intention is to throw away the existing ones (always remember that we still have the modifiers and that we do not lose the local changes) and recreate them based on those received. In a case where we are talking about GB of data, perhaps the operation, if it becomes frequent, could be too invasive.
3. In case of hard sync the implementation is simpler. There is no logic to apply data locally: I delete and recreate.
4. In the event of hard sync and a call gone wrong, it is possible in any case to decide to try again to request the data, which will therefore always be updated.
5. In cases of soft sync, the data can be transmitted by the server causing a real-time change. When it changes it is sent, so the UI updates.
6. In case of soft sync, the danger of having inconsistent data increases: it can happen that a message is lost, that the client does not correctly receive the data from the server, for some connection problem it can lose some messages, that it can really be offline while the server transmit.
7. Local data change logic must be implemented for every action that the server reports on the data.

The crucial question now is: when is it convenient to do a hard sync, when is it convenient to do a soft sync?

Schedule syncs

When a device launches the application it is always best to hard sync everything that is stored locally. Sometimes the amount of data that should be updated is too much, so you can adopt a lazy sync, for which a resource is updated only when requested.

Since the device goes online then only soft sync can be used, with the data that the server sends. In the event of a sudden disconnection we will not be able to be sure of what has been sent, so once you return online you will need to resort to a hard sync again, as if the application restarted. Since with modifiers we apply changes both locally and then in the cloud, we don’t have to worry about what will happen with them. A hard sync will bring them back to us if persist has already been applied, but we will already have them on our device. It will be rare for the persist not to be applied in the event that the device is connected to receive a result from a hard sync, meaning that there will not be a case of we need to sync the data before applying a persist, we will only do it after.

Conclusion

It would be useful to have a single point where sync behaviors are defined, as happened for modifiers. It would therefore be convenient to have, as a modifier exists, a Syncer. This class will have to be able to handle messages arriving about that entity (soft sync) and to replace local data with those on the server. Furthermore, it would be correct to have an order of execution of the modifiers and syncers, inserting them all in the same queue.