Digital Twin – Command and Control

The umajin engine allows multiple users and devices to participate in a shared visualisation of live IOT data, enterprise data and 3D geospatial data including terrain, buildings, people, assets and systems.  Uses include tactical, security and logistics scenarios. 

Some users can spectate just with a smartphone or laptop, while some users can experience an immersive VR or AR view of the landscape as a table top view. Each active participant is tracked in real time in the real world –this allows their interactions and their VR avatar (or AR HUD) to be visible to the other users as they collaborate on command and control style scenarios.

The Umajin enterprise layer is able to marshal real time data sources and relay real time commands for execution, based on the interaction in the spatial simulation.

Umajin also provides a full stack of real world fusion systems for GPS tracking outside, and radio interpolation for inside tracking. This can be used to provide location for thousands of people and assets in real time in three dimensions. Once in the command and control environment it’s possible to scrub the timeline and see the historical position of items.

IoT Module

Umajin supports industry standard IOT sensors and data integration frameworks.  We also provide standard firmware for an ESP32 module that can be attached to physical assets and supports both WiFi and Bluetooth Low Energy (BLE) in the default package. The Umajin Embedded framework also supports attaching sensors like GPS and other networks such as LoRa and GSM.

Visuals

The Umajin UI layer allows for fully functional forms and dialogs using ray picking from dual high accuracy controllers. Umajin can also accept analog-stick, trackpad and trigger inputs.

Umajin also supports voice input and eye fixation where the hardware is available. The combination really improves usability with the Umajin look menus and Umajin speak commands making context sensitive discoverability and positive actions extremely simple, fast and natural.

Commands can include literal directions or instructions to users or assets, or higher level commands at a system level to perhaps increase the production level for a given plant in an industrial scenario.

Maps can be range from illustrative to realistic and from terrain to urban. Real time shadow and other lighting effects can be used to help users clearly perceive scale and position. 

Assets

3D assets such as vehicles or buildings can be added to scenes and scenarios. Formats BLEND, DAE, STL , PLY, OBJ, FBX and glTF.

Scenarios

Geospatial command and control scenarios can be created to manage real time situations like civil defence emergencies.  A smartphone based application for users in the field can send in reports with GPS location, description, photos and videos of emergencies emerging on the ground.  These are all reported back in on the 3D geospatial command map. The command team can then review a list of resources such as fire, police, EMT, volunteers, sandbags & bottled water and decide what can be deployed or re-deployed.

Extensible

Additional logic can be added to scenarios with Javascript.