Deep Dive - Nodes, Node JS, Node-Red and Blockchain.
Sometime back in 2013, a person working in the technology division of a financial institution asked a group of software consultants on how they could help him with an Omnichannel design the organization had brainstormed. The word ‘Omnichannel’ is riddled with hundreds of definitions, what exactly did the person mean? The banker sympathetic to our confusion, was quick to respond and define the problem - the organization used several avenues to interact with prospects, customers, vendors and other businesses. Such channels comprised the internet, brick-and-mortar branches, phone and fax systems. It was a common complaint among many customers, who would start a product application or sales-order with their Smartphones, and unable to complete, they would inevitably show up at a branch – but, to start the ‘request’ all over again. The client wanted to ensure the best use of those channels eliminating rework and providing a seamless experience. I wracked my brains that night on how I could present a simple design and suddenly a linked-list idea came to my mind. In the model, every event that happens on the channels gets ‘added’ or ‘pushed’ to the list, ‘popped’ and viewed when necessary. But a linked list is confined to the memory of an application, I thought and a new burning question flared up, how would distributed applications and computers share such a list, how could others see what was going on? A persistent linked-list? Hmm. The idea sounded crazy then and with whatever we had and knew, we proposed an architecture. It was quite elegant when finished. The UML diagram below explains the list and its iterator.
Fast forward three and a half years to today, as I look at the problem now, knowing I can use the vector-design principles to do a better prognosis of the problem. It becomes so much easier to come up with a design that meets the above needs. Let me quickly recap what I had written regarding the V-Design and PEG principle in Parts 1 and 2. If you have read them before, you can skip this paragraph and the next. The Vector Vs embodies the truth behind all ‘ideation’ and its source. It is the design’s immutable part that describes the mission – unique and distinguished, which has no parallel. Different in approach, Vr provides movement and disruption. Vr represents ‘dynamism’. Every handshake that happens by which the source gets exposed to the sensory world needs a set of dynamic Vr pathways. In contrast is the Vt vector that signifies the inert and static sections of the idea -its structure and bulk, strength and inertia. Vr and Vt deliver the support that Vs the source needs.
Ideas and design composed of the three vectors of structure, truth and disruption, turn into models that offer a trace back to the original vectors as you progress into design and its build thereof. Here is where V-design comes in genuinely useful. As I had stated in the earlier articles that any implementation always carries a percent of defects, which have the attributes of being aerial, fiery and aquatic. The product when built has a purpose and may have elevated aerial levels. One of the three defects tend towards dominance. Few products are more aquatic as they are stagnant, inert and inactive. Such products though resilient and strong, are prone to faster decay. The fiery products are the digesters. They digest the inputs into useful data and information for establishing the source, and getting stored and persisted. When the fire goes weak or is not enough to sustain the inputs (sensory or subtle), the channels through which they flow break down and they run into places and crevices which are not theirs and create a series of disorders. The aerial part allows movements, invocations, events and absorptions - how the system communicates with itself and the external world. Commercial business functions use the trinity of PEGs – People, Engagement and Goods to carve out a ‘value’. The central idea is to reduce the levels of defects and balance the three to a steady state. This ensures an optimum performance of the software application. V-Design is not disruptive, it is another set of lenses to look at an existing design or to make a new one.
Node-RED, founded on Node.js, is a web-browser flow-editor that makes it easy to wire together devices, APIs, and online services on a canvas. By connecting the nodes, flows get created, which can be hot-deployed to a Node.js runtime. Flow-based programming is a way of describing an application’s behavior as a network of ‘nodes’. Each node has a well-defined purpose – it either is the source (P), or the engagement (E) or the structural component (G). Incoming data gets digested and passed to relevant modules. The digestive network takes responsibility of the flow of data between the nodes. If we can model the digestive process from incoming raw-data to processing, storing and elimination, then the Node model lends itself extremely well to such a visual representation. It has its limitations. It is not one that fits all. A well-balanced application flow in Node-RED appears in the screen-shot below.
To begin with, I made three new Nodes, other than what came out-of-the-box. To the palette, I added the Vector nodes - Vector Engage, Vector Source and Vector Structure based on the PEG principle. You can see the node templates on the left in the screen-shot. In an ideal and ‘purist’ application, the engagement is in perfect harmony, the canvas on the right has their instances on it, joined by different wirings. The Vr receives the event or the Message Payload containing a JSON object, which goes to the Vt for filtration, security and other checks and lastly sent to the essence of Vs for analysis and establishing the source. More complex and real scenarios appear later in the article.
Let’s look at the Hyper-ledger construction for a moment. Hyper ledger comes from the Blockchain concept, which affords a shared ledger technology allowing any participant in a defined network to view a system of record or a ledger. From its inception, it has been emphasized as a source of truth Vs. Which takes priority over its structure Vt and messaging Vr per-se. Built like a persistent linked-list, it is not the greatest in structure and provides a simple API that lets you create or view blocks of business transactions. Caveat of such a design is that it grows endlessly – there’s no stopping it from increasing in size once in motion. Though, by using Blockchain technology, businesses can enjoy a more efficient transfer of goods and services. But the ledger system does not plateau, but that’s its constitutional structure and underlying policy. Imagine the Blockchain, five years down the line. Some other disruptive technology may want to change the interface and split it into block-siblings containing historical blocks of transactions, no one views anymore. It would lead to other problems. The source of truth would suffer breakage and no longer remain the ‘source’. From one defect, it would then harbor a different set of problems, but that’s another story.
Hyper-Ledger Composer digitizes business networks. Access of business network by prescribed participants in the network is allowed, some of the participants may own the responsibility in the ledger maintenance (a function of Vt), referred to as maintainers of the network. Data model gets expressed by the Composer Modeling Language defining a structure of the resources that get stored or processed as transactions in the Blockchain. Once the domain model is in place, developers can capture smart contracts functions, written in JavaScript or Python through a platform like Node-JS.
IoTs provide sensors to engage with intangible goods. Extending the PEG model as described in second part, IoT devices and sensors can be provisioned to capture data or run another device. IoTs are yet another mechanism to identify a person, as they possess a device-id, which the back-end system can associate with the person. IoTs are the ‘engaging’, ‘Vr’ principles of the three-E’s, namely its ‘enhancement’ end. It is one useful way of putting a software wrap to a human activity of ‘footfall’. The three ‘channels’ or ‘conduits of engagement’ modeled here are the internet, footfall and phone. Every event in these channels become a journal entry in the shared Hyper-ledger. Shared between the bank, the prospect, the manager and the agent, we now have a ledger that tells you, where you are, whether at the proposal, contractual, onboarding, maintenance or servicing stage using the Blockchain as the source of truth (functionally, a Vs). The nodes, the orchestration connectors, the asynchronous operations with quality of service (QoS) assigned to them provides the structure and bulk to it. The Blockchain ledger itself is immutable.
The Blue dotted rectangle in the screen-shot above shows community ‘nodes’ added to the Node-RED library to support a local instance of Hyper Ledger Composer. You can download Hyper ledger from Docker hub and run it on a local computer. This gives a tremendous flexibility of changing the chain of blocks and seeing the changes happen for real. The three channels modeled here (all Vr) have deliberately been assigned as asynchronous to mimic a real-time behavior. The delay timers ensure they do not spit out the messages instantly. Messages get filtered and aggregated at the application level (Vt, but still at the Engage level). The channels can also write directly to the Hyper Ledger Composer which serves as the immutable source of truth (Vs).
Applications are never simple. Ensuring their wellness and upkeep (even uptime) comes from balancing the three vectors. In complex scenarios, multiple flows, different messages are for real. Below is an example of different gets and posts on the REST service.
Using the Vector Principles, the design stays simple and balanced. None of the defects become elevated or overbearing.
I recently built a Cardiac Monitoring device which can send the Message Payload either on a serial port of directly to the cloud for analysis. A true Internet of Things, the Texas Instruments CC3200 Launchpad, is a SoC (System on chip) with Wi-Fi and a built in lightweight web server. It can handle TLS and SSL connections with ease. Running above 80 MHZ, the 32-bit MCU delivers sufficient power to capture the Heart Readings and transmit to a website running remotely.
Though the assembled device on the left of the laptop as shown in the above diagram stays connected through the serial port, and the MCU code managed through Energia, it can still transmit using a WiFi service to a cloud Instance as it collects the data. The three analog probes meant to read the heart’s and skin’s electrical pulses, shown in colors red, blue and black are in the left above the assembled device.
For the purposes of this article, I used a simpler design with An Arduino Uno and a Duo without any Wi-Fi, but a simple USB, Serial-Port connection.
The TI MSP430 is a similar no brainer development kit. I don’t know if they are available any more. I still have a kit which I had built a few years back.
Using an MQ Telemetry Protocol MQTT, a message broker protocol for machine-to-machine communication, the device sends the data to a broker running in the cloud. The Eclipse conference in 2014 introduced ‘Mosquito’, an open source, BSD licensed message broker, which goes now by the name of MQTT. It is a lightweight publish, subscribe protocol, with reliable messaging, and websockets useful in M2M communications. It also provides enormous stream processing and real-time analytics at the cloud with bridges to connect one MQTT broker to another broker. In our example, we have temperature, pressure, accelerometer, heart monitor sensor attached to microcontroller. MCU with Wi-Fi transmits information through MQTT to cloud (hosted by IBM Cloud DeveloperWorks Quickstart). Decision engine on the cloud responds to the message by turning on gates, blink LEDs – in our example, it plots the data.
The typical lightweight payload in MQTT uses JSON format. Broker in the cloud expects to receive the device id, the label of the data and the data as comma separated values. Device therefore sends the heart readings using the mac-id as the device id and the data collected.
I apply a few rules which are pertinent to all devices that set up the things in IoT.
Complexity may be scaled, simplicity secured. Keeping the devices simple keeps them secure. Patient information has nothing to do with heart reading and a good design must dissuade such data from getting transmitted with the data it captures.
The design must let the device do what it does well. If the device has been primarily built to capture heart information, then it should be left at that. To add a complex display to it, to let it do temperature and pressure readings defeats the purpose.
Another device should be the watchdog. A device could malfunction or the cloud could have an outage. It is sometimes not easy to monitor either of them as you may be miles away. It is advisable to set a thief to catch another. While the device is reading the heart, something else can check the battery and the connection. Node-RED also provisions their connectivity, though a little differently using MQTT nodes and Hyperledger. In the Node RED screenshot as shown, the Hyperledger Composer instance is local.
Delegation. ‘Thing’ can sense, it can read, it can write (functions of Vr). ‘Thinking’ and ‘analysis’ is best done at the application (Vt and Vs). Reliability of the network, of the model is key to the delegate model (Vt). The ‘thing’ that is a ‘do-it-all’ is a misfit for this design.
Anti-Pattern. A good design must allow ‘things’ to preserve their inherent nature of either collecting or acting upon another analog device. A balance of computation, data collection should be its entitlement. Conversions or curve fitting do not belong to the device.
Storage (Vt) is best managed at the application level. Storage at the device increases complexity and poses security problems. Segregation at the unit level – it is best left for a device act a singleton and just remain a device (Vr). Collecting data and producing output is best left to two or more worker ‘things’. Pushing information to a device may not be workable with many of the ULP (Ultra Low Power) systems conserving power between intervals.
While it is all about Collect (Vr) – Filter (Vt) – Analyze to a conclusion (Vs), the challenges posed are many. I present a few which stand out as most critical. Standard techniques used in managing things are yet to evolve. Even programming languages are still evolving and only smallest codebase like ‘C’ and ‘Assembler’ can address the small footprint of many MCU(s). Devices have different processor speeds, different timer implementations sometimes making it far from easy synchronizing these devices. UART communication between devices and / or computer is adhoc and without policies or security. People receive control through application, which appraises them of any situation. Devices get managed by the application for collection Vr, filtration Vt and actuating Vs. Substantiating through a second ‘opinion’ thing. If a temperature and pressure reading looks abnormal, there should be a thing to check the battery levels or a GPS to ensure the location has not changed.
Part 1 Part 2 Copyrights and Trademarks acknowledged. Hyperledger Composer is a copyright of the Linux Foundation Node-RED is a project of the JS Foundation CC3200 Launchpad is a product of Texas Instruments NodeJS is a copyright of NodeJS Foundation and Trademark of Joyent Inc MQTT is an OASIS standard Docker is copyright Docker Inc IBM is a Trademark of International Business Machines