How to Nail Your Distributed Architecture

A new online tool to match your application to OPC UA, DDS, MQTT, Kafka, or ROS

With the rise of AI and powerful processors, system architects in almost every industry are building smart systems. Smart systems are often distributed, so to build one, you have to share data. And that’s the problem; choosing a distributed architecture that matches your problem is both fundamental to the design and really hard. Because it’s fundamental, it has to be done early. Because it’s hard, it has to be done right. So, the very first decision that most system architects make is important and risky: deciding how to share data. Choosing the right way to share data is the key to success. Choosing the wrong design is usually fatal. 

Unfortunately, most designers struggle with the choice. Many don’t think how to share even really matters, since the fundamental function of the maze of technologies and protocols is simply to get data from one place to another. So, they choose by: a) what they already know; or b) what sounds easy to learn/use; or c) what they think others are using. All these are wrong:

1. Using what you know is wrong, for the simple reason that few designers have built smart distributed systems before, so their experience may not be relevant. If all you have is a hammer, all problems look like nails. But of course, all problems are not in fact nails.
   
   
2. Using the easiest to learn is wrong, because using a simple technology leaves most of the very tricky functionality to the user. As a simple example, consider scale. What works in a lab with five systems will almost certainly fail with 50, or 500, or 5000 systems.
   
   
3. Blindly following the crowd is wrong, because there are just so many “others”! Choosing a design because you see it most online is like choosing an iPhone to run your company’s database server. Sure, iPhones are great, but they can’t run databases well.

Far worse, most architects don’t realize they made the wrong choice until it’s too late. There is perhaps nothing harder to change than architecture. And a poor choice may not be apparent for years.

The real problem is matching design requirements to realistic solutions. As in most designs, the first step is to answer questions: Is it wireless? Large? Must it be super reliable? Is there a lot of dataflow? How fast is fast enough? How will we update it? What really matters?

Of course, these probably aren’t the right questions. The real world is complicated. There are many, many different types of applications, all with different requirements. There are many technologies, each with different capabilities. To match a design to a technology, you have to ask the right questions and know the right answers. But this is exactly why it’s hard: designers don’t know the right questions. They likely don’t have all the answers. And even if they did, they don’t know the technologies, so they don’t know what those answers imply. Experts who are fluent in several connectivity technology “languages” are rare indeed. The industry needs a guide.

A few years ago, the Industrial Internet Consortium stepped forward to be that guide. Their work was published as the Industrial Internet Connectivity Framework (IICF). It analyzed the top technology choices for intelligent systems and presented capabilities of the technologies. This helped, but it was 128 pages of dense technical material inaccessible to the inexperienced users who need it most. And it didn’t address the hard issue of knowing what to ask. The IICF certainly highlighted how different these technologies really are. But it took many days and a lot of work to use.

As original authors of the IICF, and as the largest software framework company for autonomous systems, we decided to try to help.  The result is the RTI Connectivity Selection Tool. It will help you choose between the most important technologies and standards in the industry: OPC UA, DDS, MQTT, Kafka, and ROS.  

The tool works by asking you diagnostic questions and scoring them against the technologies.  Some of the questions may seem odd. They certainly don’t cover all aspects of any technology.  Rather, the questions seek to pinpoint only where the technologies differ. To do that, you can’t ask questions like “Is security important?”; all options provide some type of security. Rather, you need to ask questions like “Are you connecting devices to other devices?” or “Do you have a team of programmers?” These questions clearly separate capabilities between options. In the end, we will present a very detailed analysis behind the scores. You will then understand how your selections resulted in the Tool’s final analysis and the important guidance offered to aid you in choosing between the technologies and standards.

Of course, you may say we are biased, and it’s true. However, we are only biased towards helping users select the right technology without wasting time. In fact, the original reason we collected questions was to stop our own sales force from wasting time on applications that should not use our key standard, DDS. We found these simple questions allowed us to quickly disqualify most opportunities so we could focus on real prospects. We didn’t want those who don’t fit to try our technology; that just wastes our time and theirs. The tool, therefore, makes an honest attempt to help you choose fairly.

In the end, it doesn’t really matter if you have a hammer or not. All problems are not nails. The harder question is knowing when you need a hammer, or a screwdriver, or a welder, or glue. Choosing the right tool is the first step in any successful project.

You can try the Connectivity Selection Tool here. It should help guide you on your journey.

About the author

Stan Schneider is CEO of Real-Time Innovations (RTI), the largest software framework provider for smart machine and real-world systems. 

Stan also serves on the advisory board for IoT Solutions World Congress. Stan holds a PhD in EE/CS from Stanford University.