State of the DolphiNDB

Software development is always moving forward, and the latest version is always the best ... until the next one arrives.  When you visit the MySQL Ndb Cluster downloads page, you are naturally recommended to pull the latest MySQL Cluster 8.0.22 (announcement), but sometimes it is good to look back at the journey taken to reach this point.

7.x release series

Prior to the 8.0 releases, MySQL Ndb Cluster had a sequence of 7.x (x=0..6) releases based on MySQL Server versions 5.1, 5.6, 5.7.  In each of the 7.x release series, MySQL Ndb Cluster was under feature development for some time, producing a number of minor releases, until eventually one minor release was validated as being acceptable as the first generally available (GA) version.  Once a release series 7.x was generally available, further changes in that series were confined to bug fixes and small improvements, with new feature development becoming only available in a new release series 7.x+1.

Each of the 7.x releases significantly improved and extended MySQL Ndb Cluster, improving SQL execution, threading, performance, async replication, checkpointing, recovery, functionality etc, as well as picking up a number of fixes and minor improvements discovered by customers, users and our own QA.

MySQL Ndb Cluster includes all of the functionality of MySQL Server, plus the unique Ndb Cluster software.  As a result, each MySQL Ndb Cluster release series has also benefited from improvements in the related MySQL Server version - new features, optimisations and bug fixes.

7.5 release series

The 7.5 release series was based on the MySQL Server 5.7 release, becoming GA (announcement) with version 7.5.4 in October 2016.

The new features included :

• Support for JSON columns.
• Stored or virtual generated columns.
• Read + Join performance optimisations : read from any replica, read from any node.
• Improved performance, memory management, usability.
• Use of InnoDB for replication state management.

Since then we have released a further sixteen minor versions of the 7.5 series, one every quarter.  Each minor version includes small bug fixes and improvements to both the Cluster and MySQL Server software, with the current latest version being 7.5.20 released in October 2020.

7.6 release series

The 7.6 release series was also based on the MySQL Server 5.7 release series, becoming GA (announcement) with version 7.6.6 in May 2018.  

The new features included :

• Optimised checkpoints giving reduced disk space usage, reduced write amplification, reduced recovery time - increased availability, increased practical data limits.
  
• Improved distributed SQL join support.
  
• Unified memory management.
• Optimised node-local communication.
• Optimised distribution awareness.
• New parallel import utility .
  
• Many performance improvements.
  

Since the 7.6.6 release of MySQL Cluster 7.6, we have released a further ten minor versions, with small bug fixes and improvements.  The current latest version is 7.6.16 released in October 2020.

8.0 release series

The MySQL Cluster 8.0 series is based on the MySQL Server 8.0 series, which first became generally available (announcement) as release 8.0.11 in April 2018.   

The MySQL Server 8.0 release included many features in this first version, and has been continuously extended in each of the eleven releases since.  MySQL Ndb Cluster is following this release model with its 8.0 releases, with new features arriving in each minor release.

The first generally available release of MySQL Cluster 8.0 was 8.0.19 (announcement), released in January 2020.

MySQL Cluster 8.0.19 (January 2020)

Features included :

• Simplified configuration, elastic memory management.
• Better support for 3 and 4 replica clusters.
• Support for up to 144 data nodes in a cluster.
• Improved multithreading for parallel backup and restore.
  
• Increase row size limit.
  
• Improved distributed SQL execution performance and efficiency.
• Disk stored columns performance and stability improvements .
  
• Integration with MySQL Server 8.0.19 (Transactional MySQL Data Dictionary, many other MySQL Server features).
• Usability improvements.
  

Since 8.0.19 there have been three further releases of MySQL Ndb Cluster 8.0 : 

MySQL Cluster 8.0.20 (April 2020)

Features included :

• Further backup restoration process parallelism
• Parallel intra-nodegroup links for high write volume applications
• Improved distributed SQL processing of outer and semi joins
• Performance improvements
• Data integrity tooling improvements
  

MySQL Cluster 8.0.21 (July 2020)

Features included :

• Support for up to 16TB of in-memory data per Data node
• Read any replica tables by default
• Further disk stored column performance and monitoring improvements
• Improved distributed schema synchronisation

MySQL Cluster 8.0.22 (October 2020)

Features included :

• Support for generating and restoring encrypted backups
• Support for IPv6 addressing between all cluster node types
  
• Improved performance for Blob column Insert, Update, Delete
  
• Improved replication applier performance for Blob columns

As you can see, MySQL Ndb Cluster is constantly evolving and improving with several new releases every quarter.  The 8.0 releases continue to bring new capabilities on a regular basis, and there is no shortage of ideas in the pipeline.

For more information about MySQL Ndb Cluster, you can check the official documentation, see the getting started guide, follow a tutorial, read blogs, watch a video, check out recent Ndb 8.0 101, architecture or performance slides, or even read a book or two.

Edit : typo