Does the Java world have a good cloud solution for distributed transactions?
@Wael from the Pivotal AppTx team asked this question which triggered the blog post ....
The traditional view of distributed transactions in the new cloud native microservices world is that they are a strict no-no. There are ways to workaround distributed transactions via compensation and eventual consistency and reverting to One-phase commit transactions. These alternatives although perfectly acceptable leave us wanting. Developers want everything and taking away distributed transactions is like pulling away a favorite teddy bear from a toddler.
But wait the tech industry evolves so fast there are other options available now ... Let me explain ...
Java world NOW does have support for distributed transactions indirectly. For Heterogeneous distributed transactions > indirectly via Kafka and MongoDB and for Homogeneous distributed transactions via Hyper Scale Secret Sauce Cloud Provider Expensive databases like Spanner DB and CosmosDB and others ...
Unless you like to pay top dollar to Microsoft or Google I would stay away from proprietary hyper-scale cloud provider databases that provide auto-magic 2 phase commit ACID transactions via atomic clocks synced across datacenters. God forbid you need to migrate away from these databases.
The other options are more palatable :
1. MongoDB has recently added ACID Transaction support see https://www.mongodb.com/transactions and Spring-mongo-data supports that via https://spring.io/blog/2018/06/28/hands-on-mongodb-4-0-transactions-with-spring-data and https://www.baeldung.com/spring-data-mongodb-transactions
2. Kafka - Online Event Processing have enabled achieving consistency where distributed transactions have failed see https://queue.acm.org/detail.cfm?id=3321612 Support for distributed transactions across heterogeneous storage technologies is either nonexistent or suffers from poor operational and performance characteristics. By building on top of immutable persistent ordered event logs OLEP systems are used to provide good performance and strong consistency guarantees in such settings. Use Online Event Processing to implement distributed transactions. For more concrete guidance around distributed transactions with Kafka checkout https://kafka-summit.org/sessions/simplifying-distributed-transactions-sagas-kafka/ which introduces the new Simple Sagas library. Built using Kafka streams, it provides a scalable fault tolerance event-based transaction processing engine. We walk through a use case of coordinating a sequence of complex financial transactions.
Bottomline is that distributed transactions in the cloud are hard due to the heterogenity of storage technologies and the transaction managers for ACID transactions were all built for the pre-cloud era. You will need to approach transactions in the cloud native era with new patterns like sagas and compensation and eventual consistency and rely on the product characteristics like immutable persistent ordered logs or proprietary features to realize your dream of distributed transactions in the cloud.
@Wael from the Pivotal AppTx team asked this question which triggered the blog post ....
The traditional view of distributed transactions in the new cloud native microservices world is that they are a strict no-no. There are ways to workaround distributed transactions via compensation and eventual consistency and reverting to One-phase commit transactions. These alternatives although perfectly acceptable leave us wanting. Developers want everything and taking away distributed transactions is like pulling away a favorite teddy bear from a toddler.
But wait the tech industry evolves so fast there are other options available now ... Let me explain ...
Java world NOW does have support for distributed transactions indirectly. For Heterogeneous distributed transactions > indirectly via Kafka and MongoDB and for Homogeneous distributed transactions via Hyper Scale Secret Sauce Cloud Provider Expensive databases like Spanner DB and CosmosDB and others ...
Unless you like to pay top dollar to Microsoft or Google I would stay away from proprietary hyper-scale cloud provider databases that provide auto-magic 2 phase commit ACID transactions via atomic clocks synced across datacenters. God forbid you need to migrate away from these databases.
The other options are more palatable :
1. MongoDB has recently added ACID Transaction support see https://www.mongodb.com/transactions and Spring-mongo-data supports that via https://spring.io/blog/2018/06/28/hands-on-mongodb-4-0-transactions-with-spring-data and https://www.baeldung.com/spring-data-mongodb-transactions
2. Kafka - Online Event Processing have enabled achieving consistency where distributed transactions have failed see https://queue.acm.org/detail.cfm?id=3321612 Support for distributed transactions across heterogeneous storage technologies is either nonexistent or suffers from poor operational and performance characteristics. By building on top of immutable persistent ordered event logs OLEP systems are used to provide good performance and strong consistency guarantees in such settings. Use Online Event Processing to implement distributed transactions. For more concrete guidance around distributed transactions with Kafka checkout https://kafka-summit.org/sessions/simplifying-distributed-transactions-sagas-kafka/ which introduces the new Simple Sagas library. Built using Kafka streams, it provides a scalable fault tolerance event-based transaction processing engine. We walk through a use case of coordinating a sequence of complex financial transactions.
Bottomline is that distributed transactions in the cloud are hard due to the heterogenity of storage technologies and the transaction managers for ACID transactions were all built for the pre-cloud era. You will need to approach transactions in the cloud native era with new patterns like sagas and compensation and eventual consistency and rely on the product characteristics like immutable persistent ordered logs or proprietary features to realize your dream of distributed transactions in the cloud.
