MOVERIO
BT-35E Documentation

3. Developing Windows Apps

3.1. Summary of Developing Windows Apps

The following descriptions are the steps necessary to develop a Windows app for the Moverio.

Introduction of Windows SDK

Developing Apps

Visual Studio Documentation

3.1.1. Introduction of Windows SDK

The following steps indicate the procedures for introducing the Windows SDK to a personal computer equipped with Windows 10.

3.1.1.1. Acquiring Visual Studio 2019

Download Visual Studio 2019 from the following website. (The most recent version as of May 2019 is Visual Studio 2019)

https://visualstudio.microsoft.com/vs/

3.1.1.2. Installing Visual Studio 2019

Start the Visual Studio Installer and follow the instructions to install Visual Studio 2019.

3.1.1.3. Adding Workloads

Start the Visual Studio Installer. Add and install "Universal Windows Platform development", ".NET desktop development", and "Desktop development with C++" at the Workloads page. See the website below for the procedure details.

https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio?view=vs-2019

3.1.1.4. Troubleshooting

Check the Visual Studio documentation from Microsoft.

https://docs.microsoft.com/en-us/visualstudio/?view=vs-2019

3.1.2. Developing Apps

This documentation indicates how to develop apps for desktop computers. Select the workloads that match the type of apps you are developping.

3.1.2.1. Developing Apps for Desktop Computers

See the following website for the procedure details.

https://docs.microsoft.com/en-us/windows/desktop/index

3.2. Display management

The Moverio is a smart glass equipped with a see-through (transparent) display. To use the display control from a Windows desktop app, you can use the System.IO.Ports.SerialPort namespace from the Windows standard COM port access API, and then control can be performed by sending dedicated commands.

You can use dedicated commands to control the brightness of the display, and switch between 2D/3D display mode. You can adjust the brightness of the display by using the dedicated commands and by using the auto adjust method according to the brightness of the surroundings. The Side-by-Side method for displaying 3D content is also supported.

For more information please refer to the related information.

Adjusting the display brightness

Switching between 2D and 3D display modes

Model Correspondence Table for Display Management

3.2.1. Adjusting the display brightness

Since the Moverio has a see-through display, the visibility of the images on the screen may be affected by the brightness of the surroundings. To view the displayed images clearer, please increase the display brightness if you are in a bright environment, and decrease it if you are in a dark environmnet.

To adjust display brightness, use the System.IO.Ports.SerialPort (Windows standard COM port access API) to execute a dedicated command "setbright xx" (specify a value 0 to 20 for "xx".) The Display brightness can be adjusted throuh 21 levels from 0 to 20. Execute the "setbright 50" dedicated command to specify automatic mode as the display brightness adjustment mode. You can switch the display brightness adjustment mode from automatic to manual by executing a "setbright xx" (specify a value 0 to 20 for "xx") dedicated command if automatic mode is specified as the display brightness adjustment mode.

You can also execute the "getbright" dedicated command to obtain the current display brightness. The command returns a value from 0 to 20 providing the display brightness adjustment mode is set to manual mode, otherwise the value 50 is returned for automatic mode.

3.2.2. Switching between 2D and 3D display modes

The Moverio display supports the Side-by-Side method for displaying 3D content. If you want to display 3D content in the Moverio using the Side-by-Side feature when managing operations from the app, then use the System.IO.Ports.SerialPort from the Windows standard COM port access API to execute the "set2d3d 0" (2D display mode) dedicated command or the "set2d3d 1" (3D display mode) dedicated command.

The side by side method places images on the left and right of the screen.

When creating images using the Side-by-Side method, place the left eye image and the right eye image on the left and right sides of the image respectively.

For example, when creating a 1280 × 720 (HD size) Side-by-Side image, place a 640 x 720 image for the left eye and a 640 × 720 image for the right eye as shown in the following figure to build the Side-by-Side image.

If you want to switch between 2D/3D display mode, then use the System.IO.Ports.SerialPort from the Windows standard COM port access API to execute the "set2d3d 0" (2D display mode) dedicated command or the "set2d3d 1" (3D display mode) dedicated command.

You can also use the "get2d3d" dedicated command to obtain the current 2D/3D display mode. The command returns 0 when 2D/3D display mode is in 2D display mode, and returns 1 when in 3D display mode.

3.2.3. Creating images using the see-through type display

The Moverio is equipped with a see-through type display that allows the wearer to superimpose images over their real-time surroundings. To superimpose the image of an object you want to display in the Moverio over the wearer's real-time surroundings, you can create an image that makes use of the see-through display by setting the background of the object you want to display in black.

3.2.4. Available brightness adjustment range

See following table on the available brightness adjustment range.

model brightness adjustment range
BT-35E 0 to 20
BT-30C 0 to 20

3.3. Camera Management

The Moverio headset is equipped with a camera for uses such as normal image capturing and marker recognition. For camera control on a Windows desktop app, you can use the Microsoft Media Foundation, which is a Windows standard camera control API. This API can capture video, display previews, and similar functions.

Please refer to the related information.

Microsoft Media Foundation information

Model Correspondence Table for Camera Management

API reference

Sample code

3.4. Sensor Management

The Moverio is equipped with a variety of sensors that detect movement, direction, and environment. For sensor control from a Windows desktop app, you can use the standard Sensor API for control. The sensor data is used to calculate the head movement of the user, and the brightness of the surrounding environment.

Please refer to the related information.

Sensor types

Sensor axises

Sensor API information

Model Correspondence Table for Sensor Management

API reference

Sample code

3.4.1. Sensor types

The Moverio is equipped with various types of sensors. These sensors consist of both hardware and software-based sensor types. Hardware-based sensors include acceleration, magnetic field, angular velocity, and surrounding ambient light. Software-based sensors are virtual sensors that derive values output by single or multiple hardware-based sensors. Software-based sensors can calculate items such as rotation vectors and gravity.

Sensor Type Description Use example
SENSOR_TYPE_ACCELEROMETER_3D Hardware Measures the acceleration of the Moverio, including gravity, along the three axises (X, Y, and Z) in [G]. Motion detection (slanting ...)
SENSOR_TYPE_COMPASS_3D Hardware Measures the surrounding magnetic field along the three axises (X, Y, and Z) in [mG]. Position detection
SENSOR_TYPE_GYROMETER_3D Hardware Measures the angular velocity of the Moverio along the three axises (X, Y, and Z) in [deg/s]. Motion detection (rotation ...)
SENSOR_TYPE_AMBIENT_LIGHT Hardware Measures the surrounding ambient light in [lx]. To change the Display brightness adjustment according to the surrounding ambient light.
GUID_SensorType_GravityVector Software Measures the acceleration of gravity along the three axises (X, Y, and Z) in [m/s²]. Motion detection (slanting ...)
GUID_SensorType_LinearAccelerometer Software Measures the acceleration along the three axises (X, Y, and Z) with gravity removed in [m/s²]. Used for detecting tapping, walking, and similar actions
SENSOR_TYPE_AGGREGATED_DEVICE_ORIENTATION Software Measures the Moverio orientation in quaternions. Head tracking

See the following websites for detailed information.

https://docs.microsoft.com/en-us/windows/desktop/sensorsapi/sensor-categories--types--and-datafields

https://docs.microsoft.com/en-us/windows-hardware/drivers/sensors/sensor-types

3.4.2. Sensor axises

The coordinate system for the Moverio acceleration sensor is different from the coordinate system of all other sensors. When wearing the Moverio, the X axis of the acceleration sensor indicates leftward movement, the Y axis indicates downward movement, and the Z axis indicates the direction of the user's line of sight. The X axis for all other sensors apart from the acceleration sensor indicates a rightward movement, the Y axis indicates upward movement, and the Z axis indicates the direction toward the user.

3.5. Audio Management

The Moverio can be connected with CTIA-compliant earphones and microphone to listen to a sound source or for having a voice conversation, or similar functionality. For audio control from a Windows desktop app, you can use the Core Audio API, which is a Windows standard audio control API. This API can be used to output a sound source to the earphones, input sound from a microphone, or set parameters to both the earphones and microphones.

Please refer to the following related information.

Core Audio APIs information

Outputting sound source from the earphones

Inputting sound to the microphone

Parameter settings

Mute settings

Model Correspondence Table for Audio Management

API reference

3.5.1. Outputting sound source from the earphones

The Moverio allows you to use Core Audio APIs to listen to a sound source from the connected earphones. See the following website for detailed information.

https://docs.microsoft.com/en-us/windows/desktop/coreaudio/rendersharedeventdriven

3.5.2. Inputting sound from the microphone

The Moverio allows you to use Core Audio APIs to input a sound from connected microphones. See the following website for detailed information.

https://docs.microsoft.com/en-us/windows/desktop/coreaudio/capturesharedeventdriven

3.5.3. Parameter settings

The Moverio allows you to use Core Audio APIs to specify various parameters for the connected earphones and microphone. See the following website for detailed information.

https://docs.microsoft.com/en-us/windows/desktop/CoreAudio/programming-guide

3.5.4. Mute settings

The Moverio allows you to use Core Audio APIs to temporarily stop earphone output and microphone input (mute setting). See the following website for detailed information.

https://docs.microsoft.com/en-us/windows/desktop/api/audiopolicy/nf-audiopolicy-iaudiosessionevents-onsimplevolumechanged

Go to Top