In order to ensure scalability of microservices, the database is mainly used to maintain data persistence rather than to process data. Data processing is primarily handled in Java at the application side. This decouples the application from the database, affecting microservices little when the database is switched to a different type or scaled up and making the most use of microservices.

One question here: is there an alternative to Java hardcoding for implementing data processing logic in microservices?

To give an answer, we need to sort through the techniques for processing data in microservices.

Database SQL

A heavy dependence on databases to process data weakens the scalability of microservices. In real-world practices, sharing one database by multiple microservices probably trigger a single point of failure, and microservices operated by line of business on a one-to-one basis will face a series of issues, such as distributed transaction, data distribution and inter-service data computation requests. The strategy is rarely used because it seriously violates the fundamental microservices design principles.

Other programming languages

In addition to SQL and Java, there are programming languages that have some forms of data processing capability, such as Scala, Python and Kotlin. They either have a high learning threshold (Scala) or are hard to be integrated with Java microservices framework (Python), or are inconvenient to use to perform data computations (Kotlin), or do not support hot deployment (Scala and Kotlin). They do not have the edges over Java. The latter is, actually, naturally suitable for processing data as the mainstream microservices frameworks are all Java-based.

Yet the high-level language also has its pros and cons.

The most obvious advantage is the seamless integration with mainstream microservices frameworks. Besides, it enables step-by-step computing processes. However, disadvantages are striking. Java lacks class libraries specifically for computing structured data, leading to great difficulties for data processing. Even a simple grouping operation needs dozens of lines of code, let alone a complicated computation, which needs unbearably large amounts of code. The problem has little improved even after Stream has been introduced since Java8. SQL, in this aspect, is far more convenient. Java, as a compiled language, is hard to achieve hot deployment, which is inconvenient to use under dynamic microservice scenarios.

But, as the other above-mentioned techniques are not suited to process data in microservices context, Java hardcoding, with its advantages, is the only choice though it is difficult.

The situation will be changed because of esProc SPL. esProc can inherit Java’s merits while improving its problems when used to process data in a microservice framework. The open-source data processing engine can be seamlessly integrated with the Java application (microservices framework) and offers all-around computational capabilities. SPL (Structured Process Language), its built-in syntax, handles computations in an incredibly effortless and simple way, far more convenient than Java (even SQL), providing a new approach to data processing in microservice framework.

esProc supports diverse data sources, including RDB, NoSQL, JSON, CSV, HDFS and RESTApi, and so on, as well as mixed computations of them.

Its own SPL syntax achieves data processing algorithms concisely. Here’s an example.

According to stock record table, find stocks that rise for over 5 days in arow and count the consecutive rising days (treat same price as rising).

For this task, data is retrieved from the database and computed in SPL. A three-level nested subquery is needed even if SQL is used to achieve the computation, let alone Java (Stream and Kotlin). SPL’s support for the stepwise computing process is also an edge over SQL.

SPL permits interpreted execution, which supports hot-swap that does not need to restart the service when there are any modifications to the script. SPL outcompetes compiled languages even if we only talk about this aspect.

esProc can be taken as embedded JDBC to be integrated into the microservices framework. The application accesses the result of executing SPL code via JDBC, as it accesses the database.

In a nutshell, esProc combines all virtues of other techniques and can be rated computing all-in-one, making it the best choice for data processing under the microservices framework. Compared with SQL, SPL supports stepwise coding to achieve complicated computations in a much simpler way. It provides rich class libraries to implement algorithms more concisely than Java and more easily than Scala. It has succinct syntax, and is easy to use thanks to the ability of seamless embedding into microservices framework when compared with Python.