Gimly’s Decentralized Identity Framework Is Built for Portability and Security
Gimly built a decentralized identify framework on EOSIO that offers users portability, security, and ease.
Over the past two years Bullish has been developing, testing, and implementing a next generation version of EOSIO designed for Enterprise grade performance. We are now ready to open source these advancements and share them with the public. Meet EOSIO-Taurus.
Bullish is one of the top six cryptocurrency exchanges in the world in terms of total daily traded spot volume (Coin Metrics) and safeguarding customers’ assets without sacrificing performance has been our primary priority. We deliberately architected the Bullish Custody and Exchange systems from the ground up to provide:
We did an extensive survey of all existing blockchains and found that each provided unique subsets of capabilities but not all the required features. We therefore decided to start with the EOSIO codebase and forked it to build the unique feature set needed to satisfy our single-producer private blockchain business requirements. With these features being battle tested for the last two years by Bullish, and supporting $200+ billion trading volume, we hope other companies embarking on building applications leveraging single-producer private blockchains in a variety of domains will find them useful.
Further we are excited that this initiative helps Bullish contribute back to the open source ecosystem and strengthen the community.
Some of the key requirements at Bullish were:
In this section we cover some of the features that make the EOSIO-Taurus blockchain uniquely suited for enterprises considering building applications on blockchains:
To ensure high availability (HA) for enterprise blockchain deployments with 24×7 availability requirements, the producer_ha_plugin provides a block producer (BP) HA solution based on the RAFT consensus protocol by running multiple producer nodes to reach consensus through the Raft protocol for block production. Further, the solution also enables the blockchain to survive and recover from disasters (DR) in extreme cases such as when a whole datacenter is lost.
The producer_ha_plugin can provide:
To support DR, one or more independent producer HA groups can be deployed as standby groups in disaster independent regions/datacenters, so that the loss of a whole region/datacenter along with the producer nodes and hardware keys will not lead to significant data loss or down time of the blockchain.
The amqp_trx_plugin enables the consumption of transactions from an AMQP queue such as RabbitMQ, which is widely used in enterprise applications. The transactions are processed in a first-in first-out (FIFO) order, even when the producer (nodeos) switches during auto failover by the producer_ha_plugin.
This feature can make it easier to integrate the blockchain with enterprise applications that use queues widely. This is especially important in exchange and banking industries, with use cases where the order of transactions matter, for example, a deposit must occur before a withdrawal transaction can be executed.
Blockchains are notorious for being relatively easy to store data into the chain but provide very little tooling to extract data from the chain and act on that data in real-time. Here we provide two important innovations to address this shortcoming – a streaming interface and a high performance query interface.
The event_streamer_plugin enables streaming of messages from smart contracts. The smart contracts can call the push_event intrinsic provided to send a message to an AMQP queue. The streaming support gives the ability to smart contracts to proactively update off-chain services in real time as the state changes are recorded on chain. For example, if a customer’s account is now authorized after AML checks have been completed on the chain then an event message pushed can notify the backend system to update that status without the backend system having to constantly poll the chain. Any nodeos in a blockchain cluster can be configured to push messages, and a cluster can be configured to have one or more dedicated nodeos instances for streaming certain messages so that the streaming solution can scale up horizontally.
To support queries of the blockchain state from large scale enterprise applications, the rodeos_plugin provides a high performance multithreaded query engine and interface to run concurrent read-only queries against the blockchain state. The plugin provides a series of RPC endpoints to support queries of data concurrently, enabling queries from many micro services for a large scale system.
The plugin incorporates all the functionality formerly provided by the rodeos binary and obviates the need for running a separate state_history_plugin to source the requisite data. At startup the plugin resyncs with the latest state from nodeos’ chainbase (the nodeos blockchain state database) through memory copies. The rodeos_plugin makes use of in-memory transfers of blockchain state from chainbase to the plugin at the end of production or replay of every block. Hence, the plugin itself does not need to maintain a durable copy of the latest state on disk between restarts, and the queries are handled from data in memory directly for higher performance.
Debugging plays a critical role in the software engineering process to ensure the correctness of complex applications. This is especially useful in building financial systems where identifying and debugging edge cases is of paramount importance to ensure that the system operates as designed under all conditions.
Smart contracts are compiled to WASM code to be run on nodeos. This carries both benefits and drawbacks. One drawback is that traditional debugging is not well supported for WASM code and it is especially difficult to debug within the nodeos virtual machine used for executing the WASM binaries. For this reason, EOSIO-Taurus provides a solution consisting of generating x86 native code files for contracts and a tester tool to execute and debug the native code files on the local machine, along with support in nodeos to load the native code file when the corresponding contract code is invoked. Since the blockchain execution is deterministic, the WASM execution and x86 execution are identical. The benefit provided by the x86 instruction set is that it easily allows for debugging via standard tooling like gdb and lldb, and it allows the contract developer to single step through the contract execution at run time when it is invoked in nodeos to enable easy debugging. With this support, contract developers can use a common debugger and single step through the smart contract execution, dramatically improving efficiency of the smart contract development.
Protocol Buffers is a portable data serialization/deserialization format used widely in enterprise applications. EOSIO-Taurus supports using Protocol Buffers as the data structure encoding format for blockchain and transaction data, including the action data, table data, and return values. Protocol Buffers has certain advantages for blockchain:
With the Protocol Buffers support, the same message format can be used among microservices and blockchain, making the integration easier, the on-chain data more stable, and the smart contract development more efficient. This is in contrast to the current ABI based encoding with a customized ABIEOS library which are then integrated in applications in other languages such as by JNI for access from Java which can make integrations error prone and complex.
Enterprise applications and deployments may have requirements on the signature providers and signature algorithms for higher security requirements and easier integrations. To meet such requirements, EOSIO-Taurus provides support for advanced signature providers and extends the signature algorithms and key formats supported.
To meet security requirements for enterprise deployments where non-extractable keys in hardware devices are preferred or required, EOSIO-Taurus includes a new TPM signature provider which allows nodeos and cleos to sign transactions and/or blocks with non-extractable keys from TPM devices.
Standard ECDSA key formats are widely used by enterprise applications. EOSIO-Taurus adds support to the standard ECDSA key formats within contracts for easier integrations. The ECDSA public key follows the Standards for Efficient Cryptography.
To make it easier to integrate enterprise applications that use the RSA algorithms, EOSIO-Taurus adds support to the RSA signature verification within contracts through performant intrinsics.
Previously, the blockchain state was persisted and stored as a shared memory map file on disk. The shared memory map file solution has two major drawbacks causing issues in enterprise deployments: the shared memory file is sensitive to changes in compiler, libc, and boost versions, and changes of compiler or library version will make an existing shared memory file incompatible; the shared memory file is not fault tolerant, and if the process crashes, the shared memory file left is likely in the “Dirty DB” state and it can not be used to reload the blockchain state.
To get rid of these limitations, EOSIO-Taurus persists the state in the portable snapshot format and also makes sure the state file creation is fault tolerant and crash safe. To support persisting the blockchain state as a snapshot, EOSIO-Taurus:
With such mechanisms, the state persistent file as a snapshot is stable to successfully reload the nodeos. It could be only a little bit old if nodeos crashes. The stable snapshot-based blockchain state makes the blockchain system more reliable when running in unpredictable cloud environments where reboots may be initiated by the vendor without warning.
One of the fundamental issues in application development is the discovery of bugs that only occur in production deployments and are not reproducible in developer’s testing environment. The usage of hardware keys in the production blockchain deployment also makes it impossible to continue a blockchain from the production environment in a testing or local machine as is. The EOSIO-Taurus blockchain provides some special debug function support that enables developers to download a state snapshot from the production machines and make changes to it (including replacing block signing keys and account keys) and continue production on the local machine. This enables microservices developers to test their local instantiation of the service to explore in-depth to pinpoint the source of the issue.
Smart contracts implementing enterprise application logic may need to run on top of large scale data entries because of the complexity of the business logic and the scale of the blockchain state. For supporting such requirements, the EOSIO-Taurus producer supports long running time transactions for large contract actions by allowing the transaction execution time to exceed the block time, controlled by configuration parameters. Once the long running transaction is executed successfully, the producer can be reconfigured to respect the max transaction time appropriate for the network.
Asynchronous block signing allows EOSIO-Taurus to ensure the block producing performance, and sign blocks using more secure yet slower signature providers to enhance the security.
Those more secure signature providers, such as a TPM device or an HSM hardware based signing service may need more time to sign a block. For example, signing blocks using a TPM device may need time in the approximate range from 30 to 60 milliseconds. As a high performance blockchain, EOSIO-Taurus can not be blocked by these devices for signing the blocks.
Previously, block signing was integrated into the block building process. In EOSIO-Taurus, block signing and block construction occur in separate threads. Block signing takes place after the completion of block construction, thus it can effectively utilize a TPM signature provider without slowing down block production.
The EOSIO-Taurus blockchain has now been used by Bullish for over two years with more than 99.9% uptime and has supported well over $200B+ trading volume. It has incorporated newer innovations. For example, the zero downtime upgrades of the EOSIO-Taurus nodes including the active block producer node makes it possible for the exchange to upgrade components without any downtime to trading or custody operations. This makes it ideal, in our opinion, for enterprise scale deployments of the platform. In addition, the EOSIO-Taurus blockchain has proven itself to be a highly performant blockchain with more than 90% head room for future growth even under the heaviest load conditions on days of extreme volatility in the crypto market making it ideal for workload heavy applications. It should be noted that the EOSIO-Taurus blockchain, as deployed at Bullish, is a single producer private blockchain with unique requirements and as such other users, especially those dependent on multi-producer public blockchains, may need to perform additional due diligence before adopting these configurations verbatim.
We would also like to mention that EOSIO-Taurus continues to mature as an enterprise grade blockchain for use in the fin-tech industry and we are actively developing sophisticated features such as:
We look forward to communicating these and other exciting features that we are working on with the community in the near future.
For more details, please check the EOSIO-Taurus repository (https://github.com/EOSIO/taurus-node) and the documents included in the repository.
All repositories and other materials are provided subject to the terms of this IMPORTANT notice and you must familiarize yourself with its terms. The notice contains important information, limitations and restrictions relating to our software, publications, trademarks, third-party resources, and forward-looking statements. By accessing any of our repositories and other materials, you accept and agree to the terms of the notice.
Important: All material is provided subject to this important notice and you must familiarize yourself with its terms. The notice contains important information, limitations and restrictions, relating to our software, publications, trademarks, third-party resources and forward-looking statements. By accessing any of our material, you accept and agree to the terms of the notice.