Synchronizing data to the Edges: a PoC with Couchbase -

Here, we present a Proof of Concept (PoC in the following) for the automatic synchronization of data between a central database and Lite databases on Android systems managed with Couchbase

Situation

When an emergency team (for example the crew of an emergency room ambulance) is entrusted with a mission, the team leaves with its own vehicle, goes to the location of the mission and communicates the details of the situation to the operations center; from the operations center, they then receive instructions on what to do and are directed to one of the available emergency centers.

Obviously, for all this to be feasible, the team must receive information from the operations center regarding which emergency center to go to. For example, the team needs to have the address of the center available.

[At a later stage, it may be necessary for a certain central database to receive information from one of the mobile phones in the field. For the moment, in order to simplify the presentation, we will ignore this possible requirement.]

The idea is that emergency vehicles are equipped with an application for mission management that takes data from the central database and presents it.

An example could be that of an ambulance that mounts a tablet with an Android application for managing missions. The ambulance crew communicates with the operations center via the Android application.

The Problem

One of the first problems we find ourselves having to face is that of synchronizing the centralized records of emergency response with the information displayed on the various mobile devices (tablets). The persistence of the entire databases on the mobile side seems less than optimal as a solution: it requires saving all the data (and not just that necessary for the mission) and presents the problem of having to be manually updated (or updated with some specific mechanism) when they change in the centralized database .

We can also assume that the central emergency databases and services are already implemented and functioning well: we certainly don’t want to rewrite them or embark on risky database migrations! For situations in which the emergency services already have a desktop software application that connects to on premises services and refers to a relational database, it is necessary to find an expedient way to make the previously saved information available to mobile clients.

The classic solution would involve writing a mobile application that synchronizes with the centralized database, writing a service that allows and manages synchronization on the backend and the corresponding end point on the frontend.

The problem, in this case, is that it is necessary to establish a communication protocol, implement APIs, manage synchronization as well as the various cases in the mobile application

In short, it certainly doesn’t seem like an easy task…

A possible solution

However, those who know Couchbase know that there is a solution for a Lite (mobile) database with an efficient synchronization service between a server and mobile clients that is automatically managed by a component called Sync Gateway.

In other words, Couchbase provides the following architecture:

a centralized Couchbase Server database
a series of mobile databases, Couchbase Lite instances
a component for synchronization between the two previously mentioned Sync Gateway

Gateway The Sync Gateway is configured to synchronize the databases in two directions: from the Server to the Lites and vice versa. In our case, in particular, only the first direction is relevant.

This solution seems to be the right one for us: by adopting the Couchbase architecture we could obtain “free” synchronization between the central relational database and the various mobile devices.

If we imagine that we already have the relational database component available and cannot modify it, it would also be necessary to write a new, simple service that transfers the data already present in the relational database to the Couchbase server. This, however, is definitely easier to write than designing and writing specific APIs and a synchronization protocol!

Updating changes to the relational database is easily implemented thanks to CUBA’s EntityListeners: these allow you to execute custom code every time a database entity is modified. In particular, we synchronize the entity on the Couchbase Server with the one in the relational database.

Proof of Concept

We developed a Proof of Concept (PoC) to test the feasibility of our solution.

The architecture of the solution is probably not the best possible: the PoC is developed based on a pre-existing CUBA backend connected to a PostgreSQL database. Given this starting point, we chose to test the Couchbase component solely in regards to synchronization with the mobile phone. Perhaps a better solution would be to completely replace the PostgreSQL database with Couchbase. This solution was not implemented to avoid having to manage the migration and update of the architectural components already present.

The PoC foresees the architecture shown in the figure:

There are:

a PostgreSQL database for centralized data persistence
a centralized Couchbase Server database that replicates PostgreSQL data
a CUBA backend that synchronizes the relational database with NoSQL
a Sync Gateway that synchronizes mobile clients with the Couchbase Server
a mobile application with Couchbase Lite that displays centralized master data.

The backend was developed with CUBA Platform and implements a simple entity, called Hospital, which simulates Hospitalthe registry of a hospital, reporting its name, address and telephone number.

The data store and data sync components, i.e. the PostgreSQL and Couchbase Server databases, along with the Sync Gateway, are configured and released for the PoC using a Docker Compose stack.

Finally, the mobile application was implemented with Android Studio.

Testing the solution is done by inserting data from the user interface of the CUBA application and verifying correct synchronization from the mobile application.

Relational database

We simulate the relational database through a PostgreSQL database.

As mentioned, a different possibility involves the use of Couchbase for this functionality as well: the advantage of this solution lies in the cleanliness and simplicity of it. However, in the case we are analyzing, the fact that a PostgreSQL database was already present and connected to a backend application must be considered.

By using database version 12, the data is stored in a database called hospital The PostgreSQL database runs as Docker Compose stack service, since we are dealing with a PoC; for production deployment it would be advisable to use a more advanced container orchestrator, for example Docker Swarm or Kubernetes,and it would be advisable to configure a cluster to manage horizontal scaling and data redundancy.

Couchbase Server

The Couchbase Server database is also dockerized. We use a simple default configuration for the service stacked in Docker Compose. However, in a production situation Couchbase allows you to manage a cluster ensuring the redundancy and horizontal scaling capacity of the database service: here too it is possible to use more advanced tools for container orchestration.

CUBA Backend

We developed a backend service for managing personal data in the relational database by using the CUBA Platform. CUBA is an open source framework that speeds up the creation of business applications. CUBA is based on Spring framework, uses Java Persistence API (JPA) for managing data persistence and Gradle as a build and dependency management system. CUBA Platform also generates a simple UI through Vaadin, allowing you to view data graphically as well as edit.

The connection to the Postgres database is standard and handled through JPA.

However, the connection with the Couchbase server is managed through the dependencycom.couchbase.client:java-client:${VERSION}in Gradle.

The CUBA application connects to Couchbase Server at startup: a class CouchbaseManagerhandles database connection operations. Couchbase provides the objects Cluster, Bucketand Collection, which allows you to write documents (i.e., persistent data) in the database. The Clusteris the object that the backend connects to; the Bucketis instead what manages the persistence of documents in the various Collection. The CUBA backend accesses the various Collectionfrom which it reads and writes data.

Sync Gateway

The Sync Gateway is released in the Docker Compose stack; the configuration is specified in a single JSON file and includes the functions used to validate the input data (at this level, it only checks that the data is not empty), the synchronization management of only the files that the user is authorized to view (authorization) and document access via channel abstraction (a user is authorized to read public documents or documents belonging to certain channels).

Mobile App

The Android application was developed with Android Studio and includes a dependency on the Couchbase library, thus integrating the Couchbase Lite component locally for data persistence. The app implements a , DbManagera component that takes care of access to the database, and the listener functionality that configures data synchronization with the Server component via the interface provided by the Sync Gateway. In this way, when a document (i.e., a data entity) is modified on the Server component, a process update is triggered for the associated Lite component.

Synchronizing data to the Edges: a PoC with Couchbase