How to Deal with Data Management Challenges Associated with Microservices

by | Last updated Jun 14, 2023 | Published on Apr 30, 2019 | Mobile App Development

Traditional Approach vs. Microservices Model

Traditional monolithic approach to software design employs a monolithic data store like SQL server that contains a single database with many tables, and this central database is used as an engine for all data persistence. However, microservice architecture uses decentralized data management by using some of the modern architectural patterns for data management. A good microservice completely isolates its data from the outside world and only exposes it using an API (application programming interface). The focus is on avoiding data coupling by which changes in how a microservice represents its data have to be communicated and coordinated with other microservice developers. Why should data coupling be avoided? Data dependency and data coupling could lead to slowing down the introduction of new features.

However, some amount of controlled data sharing is indispensable, because no data sharing at all could prove costly and involve more complexity. The ideal solution is to meet both the needs of developers to work independently and the broader needs of the larger application team.

Data Encapsulation Challenges

There are some common data management problems associated with microservices.

  • Typically, a single microservice is responsible for a single subject such as for example, customer information and its connected data elements. All other microservices would be requesting access and updates only through an API owned by the microservice.  The challenge for developers is to consider the potential needs of other modules when they design their microservice. If a new requirement is needed, then the microservices which own the information should be updated. Developer productivity and the release of new features could be affected by the need to coordinate all prospective use of “owned” information.
  • Application-wide data analytics is required, and this constitutes another major challenge. The analytics components have to extract, transform, and load relevant data from each component as required. Typically, it is required to analyze data from multiple modules.
  • Lengthy business transactions typically include multiple components. Backing out of such transactions is a challenge. In a non-microservices world, this is done with transaction monitors. The transaction either succeeds or none of it succeeds. However, this approach is avoided in the microservices environment. What is used is the Saga pattern that can implement transactions across multiple microservices.

To improve the productivity and speed of the microservices model development teams use Oracle Database. With this, each team is free to create pluggable database or PDB using any data models like JSON/XML, key/value, spatial/graph, object, relational etc.  PDBs are small lightweight, isolated databases that are housed in larger container databases (CDB). The advantage of PDBs is that they can be independently scaled as required using specific techniques. They can be copied and relocated easily as required. As a result, there is lesser need for developers to coordinate with each other.

Six Data Management Patterns for Microservices

  • Database per service: In this pattern, each microservice manages its own data and no other microservice can access that data directly. Communication or exchange of data is possible only with a set of well-defined APIs.  Though it sounds simple, it is difficult to implement this.  The success of this pattern depends on how effectively the bounded contexts are defined in your application. Implementing this pattern could be difficult for existing monolithic systems and easier for new applications and systems. There are also other challenges such as carrying out business transactions across several microservices, and implementing queries that requires exposing data from multiple different bounded contexts. Advantages of this pattern include loose coupling between microservices and scaling up microservices individually. Developers are free to choose a particular database solution for a particular microservice.
  • Shared Database: This is a viable option if the challenges surrounding Database per Service becomes difficult to handle for your team.  This pattern tries to solve the same issue by adopting a lenient approach with the help of shared database accessed by multiple microservices. This is a safer approach in that it helps developers to work in existing ways. However, most of the advantages offered by microservices are lost. Developers across teams need to coordinate for schema changes to tables and there could be run-time conflicts when many services are trying to access the same database resources. In short, it could cause more damages than good in the long run.
  • Saga pattern: A Saga is a sequence of local transactions. The output of each transaction performed within a Saga triggers the subsequent transaction. In case one of the transactions in the chain fails, the Saga would execute a series of balancing transactions to cancel out the impact of all earlier transactions. Different from the typical point-to-point call approach, the Saga approach increases complexity and there is more chance for error.

DZone gives an example for the Saga approach. Consider a food-delivery app wherein a customer tries to order food. First the Food Order Service creates an “order.” For this pending order, a Saga manages the sequence of events. It contacts the restaurant through the Restaurant Service. The Restaurant Service places an order with the particular restaurant and sends back a reply once it receives confirmation. Receiving the reply, the Saga can “Approve” or “Reject” the order based on the reply. Then, the Food Order Service changes the state of the order from “Pending.” If the order is Approved, it informs the customer with the next details, and if Rejected, it informs the customer with an apology message.

  • CQRS: Command Query Responsibility Segregation is an attempt to get around the issues with API composition pattern. In this, the application listens to domain events from other microservices, and updates the view or query database. You can optimize the performance and scale up the query microservices as required. The disadvantage of this pattern is the increase in complexity. This can lead to latency issues where the view database is not always consistent. Code duplication could also be an issue.
  • Event sourcing: This tries to solve the problem of automatically updating the data base and publishing events. A new event is created whenever an update arises. The event store is used to store the events. This pattern can be used along with CQRS. This approach lets you manage a lot of challenges around event handling and maintaining query data. Disadvantages include imposition of an unfamiliar programming style and data that may not be always consistent.
  • API compositions: This is a direct solution to the problem of implementing complex queries in the microservice architecture. API composer invokes other microservices in the order required.  After getting the results, it performs an in memory join of the data before delivering it to the consumer.  The disadvantage of this pattern is the use of inefficient in-memory joins on potentially large data sets.

Microservices have changed the way mobile applications are developed by enhancing SDLC (Software Development Life Cycle) best practices. The main advantage of microservice architecture is speed and development teams can work independently with new features that need no coordination across multiple teams. A shift to microservice is hard. However, an experienced mobile app development company knows that using the advanced features of OracleDatabase, organizations can more easily realize the advantages of microservice-based applications by avoiding the complexities involved and cost.

Related Blogs

Share This