Plain Concepts and Boston Dynamics | First Steps with Spot

Plain Concepts has been working for years with companies in the industrial sector to help them solve production problems, quality control, and digitalization processes using the latest technologies available in the market.

The use of Mixed Reality (MR), Internet of Things (IoT), and artificial intelligence (AI) applied to manufacturing, construction, and maintenance processes offer the digitization of these processes as was not possible to date. An example of this is the digitization of construction processes. Recently techniques such as photogrammetry or scanning of facilities through devices such as cameras or lidars mounted on robots such as mobile platforms or drones are being used. This makes it possible to generate the so-called Digital Twin of a construction process that allows managers to detect anomalies, report progress, or make accurate decisions without visiting the facilities.

From Plain Concepts, we have always focused on software development, and it is where we have provided more value. But in IoT scenarios, it is essential to know the hardware in-depth to develop the best possible software. So after exploring the section, we decided to work on a mobile platform that would serve as a testbed to show our customers the possibilities of the most advanced hardware boards on the market. That allowed us to infer AI models of great complexity on the Edge (execution of AI models directly on local devices, avoiding the need to be executed on a powerful server in the cloud).

This mobile AI platform was created in 2019 on a small electric toy car, which we took to different international events to demonstrate that AI computing on Edge was already possible for most of the processes needed in factories. This tiny robot had a large number of sensors and boards for local model inference:

• 3 backfly 1.3 MP color GigE PoE
• ZED Stereo Camara
• Ouster Lidar
• Azure Kinect
• Nvidia Xavier
• Jetson nano
• Google Coral
• Raspberry Pi

It had multiple demos and could be controlled by a Tablet or directly from the HoloLens, an MR device that could guide it through the space while keeping the operator’s hands free to perform other tasks.

The vehicle had obstacle avoidance route planning developed in ROS and using information captured by a lidar sensor in real-time:

This device was very useful to demonstrate the latest advances in IoT and the importance of having software optimized for the hardware used and to reach a more profound understanding from the point of view of the hardware available in the market.

Plain Concepts & Boston Dynamics | Partnership

At Plain Concepts, we are committed to providing the best software solutions using the best hardware on the market. Today, we announce that we have recently closed a partnership agreement with Boston Dynamics, the world leader in advanced robotics. Boston Dynamics has been working for more than ten years to produce and certify robots for the industrial sector, and our mission will be to add the best possible software to this hardware.

We have started to work with Boston Dynamics robot, which has incredible characteristics in different industries. The following are some of the highlights:

• It is a robot that can walk on any kind of surface.
• Can climb stairs and ramps.
• Can move through small spaces avoiding obstacles.
• It is capable of carrying up to 14kg of weight.
• It has perimeter cameras that offer a 360-degree view of the perimeter.

So the hardware already solves like no other robot to be able to move around any facility. This is a great starting point, and on this basis, our mission will be to add sensors, AI on Edge devices with enough versatility to adapt to the needs of our customers in different industries, and Wireless connectivity to allow use in remote installations.

After showing it to several clients, we have detected several immediate uses in different sectors. One of the clearest is the digitization of the construction process, which is currently a tedious process. This robot is terrestrial but can reach any corner of a facility, so providing it with lidar sensors and 360 cameras can be used for periodic and automatic scanning of a facility using an autonomous guidance system. This will allow us to create a Digital Twin, precisely the same in different periods, which will help us to:

• Early detection of errors in construction processes.
• Automatic progress verification after comparison with BIM models.
• Creation of geolocalized instructions.
• Remote spatial measurements
• Visual progress history.

First steps with Spot of Boston Dynamics

In initial tests, we mounted a power bank-powered Azure Percept with 4G and 5G connectivity via MiFi on top of the Spot robot, which we used as a testbed to explore a large number of Vision and Audio-based models.

Vision Algorithms

• People detection.
• Identification of construction machinery.
• Detection of fire extinguishers for automatic checks.
• QR and barcode scanning.
• Troubleshooting

Audio Algorithms

• Detection of anomalies in audio-based machinery.
• Speech recognition to receive commands.

Holo-Robot

After these tests and the feedback from several companies, we have started developing a software solution called Holo-Robot.

The Holo-Robot project proposes researching and developing a tool to control the Spot quadruped robot through a mixed reality-based interface using the HoloLens 2 device.

To achieve this purpose, the system will be equipped with a navigation system that allows the operator to move the robot without worrying about obstacles in its path. A higher-level layer will be added to this navigation system to manage the movements on a virtual map (mission management).

The operator will control the system through an immersive experience implemented through mixed reality technology using HoloLens 2, which will allow operation in both a first-person and third-person view.

The system will include computer vision capabilities that allow the robot to detect people, read meter figures or determine the status of a switch, all of which will be developed in a perception module. Finally, to provide the system with more outstanding capabilities, a series of embedded components will be incorporated, such as: a 360º camera to provide panoramic vision, a LIDAR sensor to provide precision, a Jetson computer with an NVIDIA graphics card for image processing, the Google Coral accelerator component to accelerate the computational vision processing of the perception module, a SpotCore global processor to perform the central processing and a Spot GXP power system to provide power ports to feed the rest of the systems and all this connected by a communications system to provide robustness and security to the system.

The need covered by the new product is clear, to provide a new tool to increase safety in large industries, automating the reviews and inspections of hazardous areas of their plants.

Conclusions

Technology is advancing rapidly, so it is essential to know the latest advances in cutting-edge areas such as robotics, mixed reality, Internet of Things, and Artificial Intelligence.

From Plain Concepts, we have the eternal commitment to bring the latest technological advances to our clients and advise them on which ones have the most incredible future to help them increase their competitiveness. Becoming a Boston Dynamics partner will allow us to bring the best hardware with the best software to the industrial sector.