Tuesday, July 7, 2015
If you missed the keynote, you can watch it on YouTube below (the keynote demo starts at 19m). In this blog post I will cover at a high level the UIs I created for the keynote demo, setting the stage for a more in-depth analysis with subsequent posts.
Rx.js in the Keynote Demo
This year’s Red Hat Summit Middleware keynote demo showcased 3 major technologies at Red Hat:
IoT using Red Hat JBoss A/MQ as the message broker, and Apache Spark for streaming analytics
Docker at Scale using OpenShift
Mobile with the Red Hat Mobile Application platform
To showcase the data we collected from the systems, I built 3 user interfaces using Rx.js to manipulate the data, and d3.js to visualize it. Developing these UIs provided me with an excellent opportunity to dive deep into these libraries and learn many of their ins and outs.
IoT Beacon Location
The data collected for the IoT portion of our keynote demo consisted of millions of scan events from 300 Gimbal BLE beacons handed out to DevNation attendees. The scanners were made using Raspberry PIs that transmitted the scan events to a Red Hat A-MQ message broker. These numerous events were then processed using Apache Spark, reducing the millions of beacon events into tens-of-thousands of meaningful "business events" representing attendee movements throughout the conference.
IoT Map Visualization
The Spark-processed data was retrieved from the message broker into a node.js back-end, was stored in a mongodb datastore, and then was re-broadcast to the browser clients. I used Rx.js throughout both the node.js back-end and in the browser to:
wrap a STOMP over Websocket protocol connection.
wrap the websocket broadcasts and clients
filter and map the data streams
bind to DOM events
The map UI uses d3.js to bind the beacon location data to SVG circles in the browser. I then used the d3.js force layout to animate the circles and move them around the map, mimicking the movement of the real-world beacons.
Watch the video of the keynote linked above to see the UI backed with real data. I additionally embedded a code-pen that shows the beacon-location visualization backed by simulated data. (If the animation is already complete, hover over the codepen and hit the "rerun" button in the bottom right).
IoT broker Visualization
The map UI worked really well to display the business value of the beacon data collected. We needed a second UI however to demonstrate both the sheer volume of data that our A-MQ message broker was receiving from the Raspberry Pis, and the dramatic reduction in data volume as Spark processed the data.
Again using a STOMP over websockets connection to the message broker, I used Rx.js to manipulate the data, and d3.js to visualize it. This broker UI is running in the codepen below backed by simulated data.
1k Containers Visualization
The power of docker, kubernetes, and OpenShift was demonstrated with a rapid scale-up of 1026 containers live on stage. To visualize this scale-up, we developed a custom "hexboard" UI, where each hexagon represents an individual kubernetes pod. The lifecycle of each individual pod was reflected as a falling hexagon indicating each stage transition, culminating with the shadowman Red Hat logo when all pods were online.
I’ve included a codepen below showing the hexboard UI, again backed by "simulated" data.
Claim you container
Audience members were then provided an opportunity to claim their own piece of "the cloud" by drawing a sketch on their mobile devices and posting those sketches to an individual kubernetes pod via the Red Hat Mobile Application Platform back-end. The audience participation was incredible, and we quickly filled the hexboard with user-generated sketches!