Tag Archives: Surface Dial

18May 2017

Surface Dial Revisited

by bsonnino ⋅ 1 Comment

Some time ago, I’ve written a post about Surface Dial programming. With the introduction of the Creator’s Update, some things have improved. In this post, I will show some changes and will revisit the program that was developed and add these to the app. To use these changes, you need to have the Windows 10 Creator’s Update installed (you will have it if you are using the Windows Insider builds), Visual Studio 2017 and the Creator’s Update SDK (build 15063) installed. Once you have these pre requisites installed, go to the project properties in Visual Studio and change the target version to 15063:

With that, the new APIs will be opened.

Menu icons from font glyphs

A welcome change was the possibility to use font glyphs for the menu items. In the previous version, we had to add PNG files to the project, create the icon and then create the menu item, with some code like this:

var iconResize = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/Resize.png"));
var itemResize = RadialControllerMenuItem.CreateFromIcon("Resize", iconResize);

This is not needed anymore: now, we can use the CreateFromFontGlyph method, that creates the menu item from a font glyph, that can come both from an installed font or for a custom font for the app. This code creates the menu icons for the app:

// Create the items for the menu
var itemResize = RadialControllerMenuItem.CreateFromFontGlyph("Resize", "\xE8B9", "Segoe MDL2 Assets");
var itemRotate = RadialControllerMenuItem.CreateFromFontGlyph("Rotate", "\xE7AD", "Segoe MDL2 Assets");
var itemMoveX = RadialControllerMenuItem.CreateFromFontGlyph("MoveX", "\xE8AB", "Segoe MDL2 Assets");
var itemMoveY = RadialControllerMenuItem.CreateFromFontGlyph("MoveY", "\xE8CB", "Segoe MDL2 Assets");
var itemColor = RadialControllerMenuItem.CreateFromFontGlyph("Color", "\xE7E6", "Segoe MDL2 Assets");

We only have to add the Unicode character and the font name and that’s all. The new Surface Dial menu is like this:

Once we have the new menu, we can add new features to the app.

Changing behavior when the button is pressed

One change that we can make to the project is to detect when the button is pressed and change the behavior when the dial is rotated. This is done by using the new IsButtonPressed property in the RadialControllerRotationChangedEventArgs class. We can use it in the RotationChanged event to move diagonally when the button is pressed. We only need to do a small change to the code:

private void ControllerRotationChanged(RadialController sender, 
    RadialControllerRotationChangedEventArgs args)
{
    switch (_currentTool)
    {
        case CurrentTool.Resize:
            Scale.ScaleX += args.RotationDeltaInDegrees / 10;
            Scale.ScaleY += args.RotationDeltaInDegrees / 10;
            break;
        case CurrentTool.Rotate:
            Rotate.Angle += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.MoveX:
            Translate.X += args.RotationDeltaInDegrees;
            if (args.IsButtonPressed)
                Translate.Y += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.MoveY:
            Translate.Y += args.RotationDeltaInDegrees;
            if (args.IsButtonPressed)
                Translate.X += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.Color:
            _selBrush += (int)(args.RotationDeltaInDegrees / 10);
            if (_selBrush >= _namedBrushes.Count)
                _selBrush = 0;
            if (_selBrush < 0)
                _selBrush = _namedBrushes.Count-1;
            Rectangle.Fill = _namedBrushes[(int)_selBrush];
            break;
        default:
            break;
    }
}

When the tool is MoveX or MoveY, we check for IsButtonPressed and, if it’s on, we move both in the X and Y directions at the same time. Now, if you run the code and press the button while rotating the dial, you will see that the rectangle moves in the diagonal.

Hiding the menu

The menu is a great tool, but some times, it’s too intrusive and you would like to hide it. Until now, there was no way to hide it, but the Creator’s Update introduced the possibility to do that. You just have to set the IsMenuSuppressed property of the RadialControllerConfiguration class to true and the menu will not appear anymore.

In this case, there will be an extra problem – the controller won’t send messages to the app anymore. You will need some extra steps to regain control:

Set the ActiveControllerWhenMenuIsSuppressed property of RadialControllerConfiguration to point to the controller
Capture the ButtonHolding event of the controller to set the actions when the user is holding the button
Give feedback to the user, as he won’t see the menu anymore

With these steps, you’re ready and don’t need to show the menu.

We will add a new TextBlock to the window to show the tool the user is using. In MainPage.xaml, add this code:

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Rectangle ...
    </Rectangle>
    <TextBlock x:Name="ToolText" HorizontalAlignment="Right" VerticalAlignment="Bottom" 
               Margin="5" Foreground="Red" FontSize="20" FontWeight="Bold"/>
</Grid>

Then, in the constructor of MainPage, add this code:

// Leave only the Volume default item - Zoom and Undo won't be used
RadialControllerConfiguration config = RadialControllerConfiguration.GetForCurrentView();
config.SetDefaultMenuItems(new[] { RadialControllerSystemMenuItemKind.Volume });
config.ActiveControllerWhenMenuIsSuppressed = controller;
config.IsMenuSuppressed = true;
controller.ButtonHolding += (s, e) => _isButtonHolding = true;
controller.ButtonReleased += (s, e) => _isButtonHolding = false;
ToolText.Text = _currentTool.ToString();

It will suppress the menu, set the controller as the active controller when the menu is suppressed and set the _isButtonHolding field to true when the button is pressed. The next step is to change tools when the button is pressed. This is done in the RotationChanged event:

private void ControllerRotationChanged(RadialController sender,
    RadialControllerRotationChangedEventArgs args)
{
    if (_isButtonHolding)
    {
        _currentTool = args.RotationDeltaInDegrees > 0 ? 
            MoveNext(_currentTool) : MovePrevious(_currentTool); 
        ToolText.Text = _currentTool.ToString();
        return;
    }
    switch (_currentTool)
    ...

When the button is pressed, we change the current tool by calling the MoveNext and MovePrevious (depending of the direction of the rotation) methods:

private CurrentTool MoveNext(CurrentTool currentTool)
{
    return Enum.GetValues(typeof(CurrentTool)).Cast()
        .FirstOrDefault(t => (int)t > (int)currentTool);
}

private CurrentTool MovePrevious(CurrentTool currentTool)
{
    return currentTool == CurrentTool.Resize ? CurrentTool.Color :
        Enum.GetValues(typeof(CurrentTool)).Cast()
        .OrderByDescending(t => t)
        .FirstOrDefault(t => (int)t < (int)currentTool);
}

These methods use LINQ to get the next tool and, if there isn’t one, reset to the first (or last). With this code in place, you can run the app and see that the menu isn’t shown, but the tool is changed when the dial is pressed and rotated, and the TextBlock reflects the current tool.

Haptic Feedback

Another improvement in the SDK is the ability to control the haptic feedback, the vibration the dial sends for every change in the rotation. You can disable this feedback by setting the UseAutomaticHapticFeedback property of the controller to false. You can then send your feedback using the methods SendHapticFeedback, SendHapticFeedbackForDuration and SendHapticFeedbackForPlayCount of the SimpleHapticsController class. You can get an instance of this class using the SimpleHapticsController property of the RadialControllerRotationChangedEventArgs class in the RotationChanged event.

We will add the feedback when the user tries to move the rectangle beyond the window’s limits. The first step is to remove the feedback, in the constructor of MainPage:

controller.UseAutomaticHapticFeedback = false;

The next step is to check if the movement is beyond the window’s limits and send the feedback if it is:

case CurrentTool.MoveX:
    if (CanMove(Translate, Scale, args.RotationDeltaInDegrees))
    {
        Translate.X += args.RotationDeltaInDegrees;
        if (args.IsButtonPressed)
            Translate.Y += args.RotationDeltaInDegrees;
    }
    else
        SendHapticFeedback(args.SimpleHapticsController,3);
    break;
case CurrentTool.MoveY:
    if (CanMove(Translate, Scale, args.RotationDeltaInDegrees))
    {
        Translate.Y += args.RotationDeltaInDegrees;
        if (args.IsButtonPressed)
            Translate.X += args.RotationDeltaInDegrees;
    }
    else
        SendHapticFeedback(args.SimpleHapticsController,3);
    break;

The code uses the CanMove method to check if the user is trying to move outside the limits and then, if that’s the case, it calls the SendHapticFeedback method to send the feedback. The CanMove method uses the translation, scale and rotation delta to check it the rectangle can be moved:

private bool CanMove(TranslateTransform translate, ScaleTransform scale,
    double delta)
{
    var canMove = delta > 0 ?
        translate.X + 60 * scale.ScaleX + delta < ActualWidth / 2 &&
        translate.Y + 60 * scale.ScaleY + delta < -ActualWidth / 2 &&
        translate.X - 60 * scale.ScaleX + delta > -ActualWidth / 2 &&
        translate.Y - 60 * scale.ScaleY + delta > -ActualHeight / 2;
    return canMove;
}

SendHapticFeedback receives the SimpleHapticsController as a parameter and sends the feedback:

private void SendHapticFeedback(SimpleHapticsController simpleHapticsController, int count)
{
    var feedback = simpleHapticsController.SupportedFeedback.FirstOrDefault(f =>; 
        f.Waveform == KnownSimpleHapticsControllerWaveforms.Click);
    if (feedback != null)
        simpleHapticsController.SendHapticFeedbackForPlayCount(feedback, 1, count, 
            TimeSpan.FromMilliseconds(100));
}

This method uses LINQ to get the feedback of type Click and, if it finds it, it uses the SendHapticFeedbackForPlayCount to send the feedback three times with an interval of 100ms. We just need to make an extra change: as we removed completely the feedback, there will be no feedback when the user changes tools. One way to restore it is to send the feedback every time we change tools:

if (_isButtonHolding)
{
    _currentTool = args.RotationDeltaInDegrees > 0 ?
        MoveNext(_currentTool) : MovePrevious(_currentTool);
    ToolText.Text = _currentTool.ToString();
    SendHapticFeedback(args.SimpleHapticsController, 1);
    return;
}

Now, if you run the app, you will see that there is no more feedback for every rotation, but you still get feedback when you change tools or when you try to move beyond window’s limits.

Conclusions

We could see a lot of improvements in the Surface Dial API, now we have a lot of control on the menu and on the feedback sent to the user. The Surface Dial brings a new way to input data into your apps, and using it can improve the user experience for your users and make your app stand above the competition.

The full source code for this project is at https://github.com/bsonnino/SurfaceDial2

31Dec 2016

Working with the Surface Dial in UWP

by bsonnino ⋅ Leave a Comment

Not long ago, Microsoft released its new device, the Surface Studio, a powerhorse with a 28” touch screen, with characteristics that make it a go in the wish list for every geek (https://www.microsoft.com/en-us/surface/devices/surface-studio/overview).

With it, Microsoft released the Surface Dial, a rotary wheel that redefines the way you work when you are doing some creative design (https://www.microsoft.com/en-us/surface/accessories/surface-dial). Although it was created to interact directly with the Surface Studio (you can put it on the screen and it will open new ways to work, you can also use it even if you don’t have the Surface Studio.

In fact, you can use it with any device that has bluetooth connection and Windows 10 Anniversary edition. This article will show you how to use the Surface Dial in a UWP application and enhance the ways your user can interact with your app.

Working with the Surface Dial

Work with the Surface Dial is very easy. Just open the “Manage Bluetooth Devices” window and, in the Surface Dial, open the bottom cover to show the batteries and press the button there until the led starts blinking. The Surface Dial will appear in the window:

Just click on it and select “Connect”. Voilà, you are ready to go. And what can you do, now? Just out of the box, many things:

Go to a browser window and turn the wheel, and you’ll start scrolling the page content
Click on the dial and a menu opens:

If you turn the wheel, you can change the function the wheel does:

Volume, to increase or decrease the sound volume of you machine
Scroll, to scroll up and down the content (the default setting)
Zoom, to zoom in and out
Undo, to undo or redo the last thing

The best thing here is that it works in the program you are in, no matter if it’s a Windows 10 program or no (for example, you can use it for Undo/Redo in Word or to scroll in the Chrome browser). All of this comes for free, you don’t have to do anything.

If you want to give an extra usability for your Surface Dial, you can go to Windows Settings/Devices/Wheel and set new tools for your apps. For example, if you have a data entry form and you want to move between the text boxes, you can create a Custom Tool like this:

That way, the user will be able to move between the boxes using the Surface Dial. Nice, no?

But what if you want to create a special experience for your user? We’ll do that now, creating a UWP program that will be able to change properties of a rectangle: change its size, position, angle or color, with just one tool: the Surface Dial.

Creating a UWP program for the Surface Dial

Initially, let’s create a UWP program in Visual Studio. You must target the Anniversary Edition. If you don’t have it, you must install Visual Studio Update 3 on your system.

Then, add the rectangle to MainPage.xaml:

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Rectangle Width="100" Height="100" Fill =" Red" x:Name="Rectangle" RenderTransformOrigin="0.5,0.5">
        <Rectangle.RenderTransform>
            <TransformGroup>
                <ScaleTransform x:Name="Scale" ScaleX="1"/>
                <RotateTransform x:Name="Rotate" Angle="0"/>
                <TranslateTransform x:Name="Translate" X="0" Y="0"/>
            </TransformGroup>
        </Rectangle.RenderTransform>
    </Rectangle>
</Grid>

We have added a RenderTransform to the rectangle, so we can scale, rotate or translate it later with the Surface Dial.

The next step is add some png files for the Surface Dial icons. Get some Pngs that represent Rotate, Resize, Move in X Axis, Move in Y Axis and Change Colors (I’ve got mine from https://www.materialui.co) and add them to the Assets folder in the project.

In MainPage.xaml.cs, get the Dial controller, initialize the Surface Dial menu and set its RotationChanged event handler:

public enum CurrentTool
{
    Resize,
    Rotate,
    MoveX,
    MoveY,
    Color
}

private CurrentTool _currentTool;
private readonly List<SolidColorBrush> _namedBrushes;
private int _selBrush;

public MainPage()
{
    this.InitializeComponent();
    // Create a reference to the RadialController.
    var controller = RadialController.CreateForCurrentView();

    // Create the icons for the dial menu
    var iconResize = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/Resize.png"));
    var iconRotate = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/Rotate.png"));
    var iconMoveX= RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/MoveX.png"));
    var iconMoveY = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/MoveY.png"));
    var iconColor = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/Color.png"));

    // Create the items for the menu
    var itemResize = RadialControllerMenuItem.CreateFromIcon("Resize", iconResize);
    var itemRotate = RadialControllerMenuItem.CreateFromIcon("Rotate", iconRotate);
    var itemMoveX = RadialControllerMenuItem.CreateFromIcon("MoveX", iconMoveX);
    var itemMoveY = RadialControllerMenuItem.CreateFromIcon("MoveY", iconMoveY);
    var itemColor = RadialControllerMenuItem.CreateFromIcon("Color", iconColor);

    // Add the items to the menu
    controller.Menu.Items.Add(itemResize);
    controller.Menu.Items.Add(itemRotate);
    controller.Menu.Items.Add(itemMoveX);
    controller.Menu.Items.Add(itemMoveY);
    controller.Menu.Items.Add(itemColor);

    // Select the correct tool when the item is selected
    itemResize.Invoked += (s,e) =>_currentTool = CurrentTool.Resize;
    itemRotate.Invoked += (s,e) =>_currentTool = CurrentTool.Rotate;
    itemMoveX.Invoked += (s,e) =>_currentTool = CurrentTool.MoveX;
    itemMoveY.Invoked += (s,e) =>_currentTool = CurrentTool.MoveY;
    itemColor.Invoked += (s,e) =>_currentTool = CurrentTool.Color;

    // Get all named colors and create brushes from them
    _namedBrushes = typeof(Colors).GetRuntimeProperties().Select(c => new SolidColorBrush((Color)c.GetValue(null))).ToList();
    
    controller.RotationChanged += ControllerRotationChanged;
    
    // Leave only the Volume default item - Zoom and Undo won't be used
    RadialControllerConfiguration config = RadialControllerConfiguration.GetForCurrentView();
    config.SetDefaultMenuItems(new[] { RadialControllerSystemMenuItemKind.Volume });

}

The first step is to get the current controller with RadialController.CreateForCurrentView(). Then, create the icons, initialize the items and add them to the menu. For each item, there will be an Invoked event that will be called when the menu item is invoked. At that time, we select the tool we want. The last steps are to set up the RotationChanged event handler and to remove the default menu items we don’t want.

The RotationChanged event handler is

private void ControllerRotationChanged(RadialController sender, RadialControllerRotationChangedEventArgs args)
{
    switch (_currentTool)
    {
        case CurrentTool.Resize:
            Scale.ScaleX += args.RotationDeltaInDegrees / 10;
            Scale.ScaleY += args.RotationDeltaInDegrees / 10;
            break;
        case CurrentTool.Rotate:
            Rotate.Angle += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.MoveX:
            Translate.X += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.MoveY:
            Translate.Y += args.RotationDeltaInDegrees;
            break;
        case CurrentTool.Color:
            _selBrush += (int)(args.RotationDeltaInDegrees / 10);
            if (_selBrush >= _namedBrushes.Count)
                _selBrush = 0;
            if (_selBrush < 0)
                _selBrush = _namedBrushes.Count-1;
            Rectangle.Fill = _namedBrushes[(int)_selBrush];
            break;
        default:
            break;
    }
    
}

Depending on the tool we have, we work with the desired transform. For changing colors, I’ve used an array of brushes, created from the named colors in the system. When the user rotates the dial, I select a new color in the array.

Now, when you run the program, you have something like this:

Conclusions

As you can see, we’ve created a completely new interaction for the user. Now, he can change size, angle, position or even color with the dial, without having to select things with the mouse. That’s really a completely new experience and that can be achieved just with a few lines of code. With the Surface Dial you can give to your users a different experience, even if they are not using the Surface Studio.

The full source code for the project is at https://github.com/bsonnino/SurfaceDial.

M	T	W	T	F	S	S
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30