You have developed your Asp.Net Core MVC, it’s working fine and then you want to take it into the next level: to make it installable and allow it to be distributed in the platform store. But a web app is a web app and, as so, it should run in the browser. That would be true some years ago, but not anymore.

There are several ways to transform your web app into an installable app, without rewriting it. One easy way to do it is to transform it into a Progressive Web App (PWA). A PWA is an app that allows to be installed in the hosting OS, is multiplatform, and doesn’t need the browser to be ran. That seems too good to be true, but there is more: it can also use the resources of the hosting OS, as a native app and can be sent and distributed in the platform store.

That seems very nice, no ? And what are the requirements to make a PWA ?

  • It should be launchable and installable
  • It should be fast, even on slow networks
  • It should serve data using HTTPS, not HTTP
  • It should offer offline experience
  • It should run in all browsers
  • It should have responsive design

The full checklist for a PWA can be found here. There is a resource to audit your app, it’s called Lighthouse and can be accessed by opening the Dev Tools (F12):

If you click on Generate report, it will scan your website to see if you have everything to run your app as a PWA:

Ok. Now we know how to check if our website is ready for being a PWA, but what does it take to transform it into a PWA? Basically, two things: a Service Worker and a manifest file.

The Service worker is a script that runs separately from your web page and offers extra functionality, like push notifications and background sync. You can read more about it here.

The app manifest is a json file that will add metadata information for your app. in a way that it can be used to install the app. Sound simple, no ? Well, it’s not that difficult, we’ll do it right now.

The first step is to create a service worker, we’ll use the tutorial described in  https://developers.google.com/web/tools/workbox/guides/get-started. Let’s create a file named serviceWorker.js and add it to the wwwroot folder:

console.log('Hello from serviceWorker.js');

Then, we’ll register it in the end of the _Layout.cshtml file:

<script>
    // Check that service workers are supported
    if ('serviceWorker' in navigator) {
        // Use the window load event to keep the page load performant
        window.addEventListener('load', () => {
            navigator.serviceWorker.register('/serviceWorker.js');
        });
    }
</script>

Once you do that, you can open the dev tools and, in the console, you will see the message:

You can also check in the Application tab, under Service Workers, that it’s up and running:

This works, but it’s not very useful. You can create your own service worker by using Workbox, a tool developed by Google, to develop something that matches your needs. For now, we’ll use workbox cli to generate the service worker (for that, you will have to have npm installed in your machine).  Install the workbox cli with:

npm install workbox-cli --global

Then run the wizard, to generate a configuration file:

workbox wizard

Once the wizard has ran, you can generate the service worker with:

workbox generateSW workbox-config.js

That will generate a sw.js file in wwwroot. You have to change the register procedure in _Layout.cshtml to reflect this change:

<script>
    // Check that service workers are supported
    if ('serviceWorker' in navigator) {
        // Use the window load event to keep the page load performant
        window.addEventListener('load', () => {
            navigator.serviceWorker.register('/sw.js');
        });
    }
</script>

 

Once you do that, the next step is to add a manifest file to your app. The easiest way to do it is to use a manifest generator, like this one:

You will need an icon for the app. I went to https://freeicons.io and downloaded an image for my icon and added it to the manifest generator. Then I downloaded a zip with the manifest file and the icons for the app. The manifest file is something like this:

{
    "theme_color": "#3558f6",
    "background_color": "#2a10b1",
    "display": "standalone",
    "scope": "/",
    "start_url": "/",
    "name": "MyMVCApp",
    "short_name": "mymvcapp",
    "description": "Sample MVC PWA app",
    "icons": [
        {
            "src": "/images/icon-192x192.png",
            "sizes": "192x192",
            "type": "image/png"
        },
        {
            "src": "/images/icon-256x256.png",
            "sizes": "256x256",
            "type": "image/png"
        },
        {
            "src": "/images/icon-384x384.png",
            "sizes": "384x384",
            "type": "image/png"
        },
        {
            "src": "/images/icon-512x512.png",
            "sizes": "512x512",
            "type": "image/png"
        }
    ]
}

This manifest file must be put on the wwwroot folder of the app with the name of manifest.json and the images, on wwwroot/images folder.

Your PWA is ready. You can run it and see if it’s installable, by checking if this icon is present in Edge (there should be a similar one in the other browsers):

Clicking on it, you will have an install prompt:

If you choose to install it, you will have an app that will run in a window, will have an icon (which you can pin in the taskbar in Windows) and an icon in the start menu:

This app is multi-platform, meaning that you can install it on Linux, Mac or any phone platform. If you press F12 to open the Dev Tools, they will open and then, in the Application tab, you can see the manifest data:

We can check the app in Lighthouse for the PWA compatibility:

From it, you can see three errors

  • Redirects HTTP traffic to HTTPS Error!
  • Does not provide a valid apple-touch-icon
  • Manifest doesn’t have a maskable icon
If you see the details, you can see they are easy to fix:
The second one needs just adding a non transparent icon to the images folder and add this line in the head section of _Layout.cshtml:
<link rel="apple-touch-icon" href="/images/apple-touch-icon.png">

This will provide a valid apple-touch-icon

The third is just provide a maskable icon for Android devices. This is done adding a purpose in the icon on the manifest:

"icons": [
  {
    "src": "/images/icon-192x192.png",
    "sizes": "192x192",
    "type": "image/png",
    "purpose" :  "maskable" 
  },

The first error is already fixed: on the Startup.cs, we already have this:

app.UseHttpsRedirection();

This will allow redirection from http to https, but we are using a port, and we cannot use the same port for http and https (every protocol must have its own port). To test it, we should have it stored in a web site and use the default ports (80 for http and 443 for https). That way we wouldn’t need to use the ports in the address and the test could be done with no problems. If we have IIS installed in our machine, then the problem is solved.

If we don’t have it, we still have an alternative: launch the app in the correct ports. One way to do it is to use the dotnet cli and launch Kestrel with the correct ports. This is done with this command line:

dotnet run --urls "http://localhost:80;https://localhost:443"

It will lauch the app with the default ports (that won’t run if you already have some web server listening in those ports), so we can lauch our app with http://localhost or https://localhost.

Once you make these changes and run Lighthouse again, you will get:

Now the app passes the PWA test. But this is not everything that a PWA can do. It can interact with the OS, like a native app, and we will see how to do it.

The first thing that can be done is to add a shortcut to the taskbar. This is done in the manifest. by adding a Shortcuts clause. We will add a shortcut for the privacy policy:

"shortcuts": [
  {
    "name": "Privacy policy",
    "short_name": "Privacy",
    "description": "View the privacy policy",
    "url": "/Home/Privacy",
    "icons": [
      {
        "src": "/images/icon-lock.png",
        "sizes": "128x128"
      }
    ]
  }
]

You will need to add an icon in the images folder, named icon-lock.png. Once you add these and reinstall the app, when you right click in the taskbar icon, you will see a new menu option:

If you click in this new menu option, it will open a new window with the privacy policy.

One other thing that you can do with PWAs is to access the file system. In the Views\Home folder, create a new FileSystem.cshtml file:

@{
    ViewData["Title"] = "File System";
}
<div class="container">
    <pre id="textBox" class="pre-scrollable" contenteditable="true">Click on the button to load a file</pre>
    <button class="btn btn-secondary" onclick="loadFile()">Load file</button>
    <button class="btn btn-secondary" onclick="saveFile()">Save file</button>
</div>
<script>
    loadFile = async () => {
        const [handle] = await window.showOpenFilePicker();
        const file = await handle.getFile();
        const text = await file.text();
        $("#textBox").text(text);
    }
    saveFile = async () => {
        const options = {
            types: [
                {
                    description: 'Text Files',
                    accept: {
                        'text/plain': ['.txt'],
                    },
                },
            ],
        };
        const handle = await window.showSaveFilePicker(options);
        const writable = await handle.createWritable();
        await writable.write($("#textBox").text());
        await writable.close();
    }

</script>

This file will show a text area with two buttons, for load and save a file. The loadFile function will open a OpenFilePicker, get a file handle, open the file and get its text and load the text area. The saveFile function will open a SaveFilePicker, create a writable file and save the textarea text to it. The next step is to add the link to this page in the main menu. In _Layout.cshtml, add this between the Home and the Privacy links:

<li class="nav-item">
    <a class="nav-link text-dark" asp-area="" asp-controller="Home" 
      asp-action="FileSystem">File System</a>
</li>

When you run the project, you will have a new option that will take you to the file system page:

Now, you have the option to open and save the files to your file system.

Another thing that can be done is to share data with other apps. In the File System page, add a new button:

<button class="btn btn-secondary" onclick="shareData()">Share</button>

The code for the button is:

shareData = async () => {
    const data = {
        title: 'Sharing data with PWA',
        text: $("#textBox").text(),
        url: 'https://localhost:5001',
    }
    try {
        await navigator.share(data)
    } catch (err) {
        alert('Error: ' + err);
    }
}

This code will load the text in the text area and will share it with the registered apps. When you run this app and click the Share button, you will see something like this:

 

 

If you choose the Mail application, it will open the mail app with the text data:

One other feature that can be enabled is to run the app as a tabbed app. For that, the only thing to do is to enable that, opening the url edge://flags and enabling Desktop PWA tab strips:

When you run the installed app, it will open as a tabbed window:

These are some features available for the PWAs, and more are being added every day. You can check some experimental features in Edge here

All the source code for this project is at https://github.com/bsonnino/MvcPwa

One criticism about Asp.Net web api is the ceremony to create a simple web api. This criticism has some reason: when you create a bare-bones web api with this command:

dotnet new webapi

You will get something like this:

As you can see, there is a lot of plumbing going on in Program.cs and Startup.cs and the real meat is in Controllers\WeatherForecastController.cs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;

namespace WebApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private static readonly string[] Summaries = new[]
        {
            "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
        };

        private readonly ILogger<WeatherForecastController> _logger;

        public WeatherForecastController(ILogger<WeatherForecastController> logger)
        {
            _logger = logger;
        }

        [HttpGet]
        public IEnumerable<WeatherForecast> Get()
        {
            var rng = new Random();
            return Enumerable.Range(1, 5).Select(index => new WeatherForecast
            {
                Date = DateTime.Now.AddDays(index),
                TemperatureC = rng.Next(-20, 55),
                Summary = Summaries[rng.Next(Summaries.Length)]
            })
            .ToArray();
        }
    }
}

Even that has a lot going on: the ApiControllerRoute and HttpGet attributes, besides the code (well, that’s needed in any platform, anyway :-)). For comparison, we could create the same web api using node.js with Express with this code:

const express = require("express");

const app = express();

const nextElement = (n) => Math.floor(Math.random() * n);

const summaries = [
    "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
];

app.get("/weatherforecast", (req, res, next) => {

    const forecasts = [...Array(5).keys()].map(i => (
        {
            "date": new Date() + i,
            "temperatureC": nextElement(75) - 20,
            "summary": summaries[nextElement(summaries.length)]
        }));
    
    res.json(forecasts);
});

app.listen(3000, () => {
    console.log("Server running on port 3000");
});

 

As you can see, there is a lot less code for the same api. The Express library takes care of setting up the server, listening to the port and mapping the route. The response for the request is the same as the one generated by the Asp.Net api.

To fix this issue, David Fowler, a Microsoft software architect designed a way to create Asp.Net web apis with low code. In fact, you can do it with a comparable amount of code than with Node. It’s called FeatherHTTP and it’s available here.

To use it, you must install the template and you can use it with dotnet new.

To install the template you can use this command:

dotnet new -i FeatherHttp.Templates::0.1.83-alpha.g15473de7d1 --nuget-source https://f.feedz.io/featherhttp/framework/nuget/index.json

When you do it, the option feather appears in dotnet new:

 

You can now create a new project with the command:

dotnet new feather -n FeatherApi

When the project is created, you can see that it’s much lighter than the standard code: there is only the csproj file and Program.cs with this code:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;

var app = WebApplication.Create(args);

app.MapGet("/", async http =>
{
    await http.Response.WriteAsync("Hello World!");
});

await app.RunAsync();

Much smaller, no? As you can see, it uses the new C#9 feature “Top level statements” to reduce a little bit the code. We can change the code to have the same output of the standard code:

using System;
using System.Linq;
using System.Text.Json;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;

var app = WebApplication.Create(args);
var Summaries = new[]
{
   "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};

app.MapGet("/weatherforecast", async http =>
{
    var rng = new System.Random();
    var weatherForecast = Enumerable.Range(1, 5)
        .Select(index => new WeatherForecast(
            DateTime.Now.AddDays(index), rng.Next(-20, 55), 
            Summaries[rng.Next(Summaries.Length)])).ToArray();
    http.Response.ContentType = "application/json";    
    await http.Response.WriteAsync(JsonSerializer.Serialize(weatherForecast));
});

await app.RunAsync();

public record WeatherForecast(DateTime Date, int TemperatureC,string Summary)
{
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

Now we have something comparable to Node! I’ve used here another feature of C#9, Records, to reduce a little the code size.

As we can see, with FeatherHttp, we can reduce the code size for an Asp.Net web api app, but can we still use it for a more complex API ?

We’ll try to reproduce the code used in my HttpRepl article here.

The first step is to copy the Customer class to the project. We’ll convert it to a record:

public record Customer(string CustomerId, string CompanyName,
  string ContactName, string ContactTitle, string Address,
  string City, string Region, string PostalCode, string Country,
  string Phone, string Fax);

Then we’ll copy the repository:

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using System.Xml.Linq;

namespace CustomerApi.Model
{
    public class CustomerRepository
    {
        private readonly IList<Customer> customers;

        public CustomerRepository()
        {
            var doc = XDocument.Load("Customers.xml");
            customers = new ObservableCollection<Customer>(
                doc.Descendants("Customer").Select(c => new Customer(
                    GetValueOrDefault(c, "CustomerID"), GetValueOrDefault(c, "CompanyName"),
                    GetValueOrDefault(c, "ContactName"), GetValueOrDefault(c, "ContactTitle"),
                    GetValueOrDefault(c, "Address"), GetValueOrDefault(c, "City"),
                    GetValueOrDefault(c, "Region"), GetValueOrDefault(c, "PostalCode"),
                    GetValueOrDefault(c, "Country"), GetValueOrDefault(c, "Phone"),
                    GetValueOrDefault(c, "Fax")
            )).ToList());
        }

        #region ICustomerRepository Members

        public bool Add(Customer customer)
        {
            if (customers.FirstOrDefault(c => c.CustomerId == customer.CustomerId) == null)
            {
                customers.Add(customer);
                return true;
            }
            return false;
        }

        public bool Remove(Customer customer)
        {
            if (customers.IndexOf(customer) >= 0)
            {
                customers.Remove(customer);
                return true;
            }
            return false;
        }

        public bool Update(Customer customer)
        {
            var currentCustomer = GetCustomer(customer.CustomerId);
            if (currentCustomer == null)
                return false;
            var index = customers.IndexOf(currentCustomer);
            currentCustomer = new Customer(customer.CustomerId, customer.CompanyName,
              customer.ContactName, customer.ContactTitle, customer.Address, customer.City,
              customer.Region, customer.PostalCode, customer.Country, customer.Phone, customer.Fax);
            customers[index] = currentCustomer;
            return true;
        }

        public bool Commit()
        {
            try
            {
                var doc = new XDocument(new XDeclaration("1.0", "utf-8", "yes"));
                var root = new XElement("Customers");
                foreach (Customer customer in customers)
                {
                    root.Add(new XElement("Customer",
                                          new XElement("CustomerID", customer.CustomerId),
                                          new XElement("CompanyName", customer.CompanyName),
                                          new XElement("ContactName", customer.ContactName),
                                          new XElement("ContactTitle", customer.ContactTitle),
                                          new XElement("Address", customer.Address),
                                          new XElement("City", customer.City),
                                          new XElement("Region", customer.Region),
                                          new XElement("PostalCode", customer.PostalCode),
                                          new XElement("Country", customer.Country),
                                          new XElement("Phone", customer.Phone),
                                          new XElement("Fax", customer.Fax)
                                 ));
                }
                doc.Add(root);
                doc.Save("Customers.xml");
                return true;
            }
            catch (Exception)
            {
                return false;
            }
        }

        public IEnumerable<Customer> GetAll() => customers;

        public Customer GetCustomer(string id) => customers.FirstOrDefault(c => string.Equals(c.CustomerId, id, StringComparison.CurrentCultureIgnoreCase));

        #endregion

        private static string GetValueOrDefault(XContainer el, string propertyName)
        {
            return el.Element(propertyName) == null ? string.Empty : el.Element(propertyName).Value;
        }
    }
}

One note here: as we declared Customer as a Record, it’s immutable, so we cannot simply replace its data with the new data in the Update method. Instead, we create a new customer and replace the current one with the newly created in the customers list.

We also need the customers.xml file that can be obtained at https://github.com/bsonnino/HttpRepl/blob/main/Customers.xml and add it to the project by adding this clause in the csproj file:

<ItemGroup >
  <None Update="customers.xml" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>

Now, we will change the main program to create the API:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using CustomerApi.Model;
using System.Text.Json;
using System.Threading.Tasks;

var app = WebApplication.Create(args);

app.MapGet("/", GetAllCustomers);
app.MapGet("/{id}", GetCustomer);
app.MapPost("/", AddCustomer);
app.MapPut("/", UpdateCustomer);
app.MapDelete("/{id}", DeleteCustomer);

await app.RunAsync();

async Task GetAllCustomers(HttpContext http)
{
    var customerRepository = new CustomerRepository();
    http.Response.ContentType = "application/json";
    await http.Response.WriteAsync(JsonSerializer.Serialize(customerRepository.GetAll()));
}

async Task GetCustomer(HttpContext http)
{
    if (!http.Request.RouteValues.TryGet("id", out string id))
    {
        http.Response.StatusCode = 400;
        return;
    }
    var customerRepository = new CustomerRepository();
    Customer customer = customerRepository.GetCustomer(id);
    if (customer != null)
    {
        http.Response.ContentType = "application/json";
        await http.Response.WriteAsync(JsonSerializer.Serialize(customer));
    }
    else
        http.Response.StatusCode = 404;
    return;
}

async Task AddCustomer(HttpContext http)
{
    var customer = await http.Request.ReadFromJsonAsync<Customer>();
    if (string.IsNullOrWhiteSpace(customer.CustomerId))
    {
        http.Response.StatusCode = 400;
        return;
    }
    var customerRepository = new CustomerRepository();
    if (customerRepository.Add(customer))
    {
        customerRepository.Commit();
        http.Response.StatusCode = 201;
        http.Response.ContentType = "application/json";
        await http.Response.WriteAsync(JsonSerializer.Serialize(customer));
        return;
    }
    http.Response.StatusCode = 409;
    return;
}

async Task UpdateCustomer(HttpContext http)
{
    var customer = await http.Request.ReadFromJsonAsync<Customer>();
    if (string.IsNullOrWhiteSpace(customer.CustomerId))
    {
        http.Response.StatusCode = 400;
        return;
    }
    var customerRepository = new CustomerRepository();
    var currentCustomer = customerRepository.GetCustomer(customer.CustomerId);
    if (currentCustomer == null)
    {
        http.Response.StatusCode = 404;
        return;
    }
    if (customerRepository.Update(customer))
    {
        customerRepository.Commit();
        http.Response.ContentType = "application/json";
        await http.Response.WriteAsync(JsonSerializer.Serialize(customer));
        return;
    }
    http.Response.StatusCode = 204;
    return;
}


async Task DeleteCustomer(HttpContext http)
{
    if (!http.Request.RouteValues.TryGet("id", out string id))
    {
        http.Response.StatusCode = 400;
        return;
    }
    var customerRepository = new CustomerRepository();
    var currentCustomer = customerRepository.GetCustomer(id);
    if (currentCustomer == null)
    {
        http.Response.StatusCode = 404;
        return;
    }
    if (customerRepository.Remove(currentCustomer))
    {
        customerRepository.Commit();
        http.Response.StatusCode = 200;
        return;
    }
    http.Response.StatusCode = 204;
    return;
}

We are mapping the routes for all the desired actions. For each mapping, we’ve created a method to use when the corresponding route is being called:

  • For GetAll, we ar using the same method we’ve used with the WeatherForecast, getting all customers, serializing the array into a json string and writing the response
  • To get just one customer, where the id is in the route, we get the id with http.Request.RouteValues.TryGet, get the customer and write the response
  • To add and update the customer, we must get it from the body with await http.Request.ReadFromJsonAsync<Customer>(); Once we get it, we can add or update the customer repository
  • Delete is similar to getting just one customer, but we delete it from the repository, instead of returning it.

You can test the API by using the same methods we’ve used in the HttpRepl article, the API will be the same, but in this case, I’m not using the customers route. Another minor issue is that Swagger is still not available in FeatherHttp (https://github.com/featherhttp/framework/issues/34), but this doesn’t change the functionality at all.

FeatherHttp is not production ready, it’s still in alpha, but it shows a direction to lightweight Asp.Net web api applications. As you can see, all the ceremony from Asp.Net web api can be removed and you can finish with an app as small as a Node one, but written in C#.

All the source code for this article is at https://github.com/bsonnino/FeatherHttp

 

 

 

 

Introdução

O Office 2007 trouxe mudanças significativas para esta suíte de aplicativos: a mais visível foi o Ribbon, uma nova interface de usuário que facilita muito o uso dos aplicativos.

Outra mudança, menos visível à primeira vista, permite que os aplicativos do Office se integrem com uma grande variedade de programas: o novo formato de arquivo. Até as versões anteriores, o formato de arquivo era proprietário: quando se queria abrir ou gravar um documento do Office em nossas aplicações, era necessário usar automação OLE, o que requeria que o Office estivesse instalado na máquina do cliente, ou então tentar descobrir o formato dos arquivos, que não era completamente documentado e podia ser alterado a cada nova versão.

O novo formato de arquivo, além de documentado, é baseado em padrões abertos, o que permite que qualquer aplicação, em qualquer linguagem ou plataforma possa abrir ou criar arquivos Office 2007. Este novo padrão, chamado de Open XML, é baseado em compactação zip e arquivos XML,  gera arquivos com menor tamanho que os anteriores e permite que outras aplicações abram e alterem estes arquivos.

As possibilidades que se abrem são inúmeras:

  • Programas que permitem indexar e pesquisar textos a partir dos arquivos no disco
  • Programas para geração de documentos em lotes, a partir de bancos de dados e modelos
  • Programas para substituição de textos em lotes
  • Processadores de texto simplificados que geram arquivos Office
  • Geradores de planilhas eletrônicas a partir de dados provenientes de diversas fontes

Neste artigo, iremos mostrar o novo formato de arquivos e como podemos acessá-los e criá-los usando o Delphi, sem a necessidade de instalar o Office.

Analisando um arquivo OpenXML

Um arquivo OpenXML, seja ele um documento (docx), planilha (xlsx) ou apresentação (pptx) é, na realidade, um arquivo zip composto de diversas pastas e arquivos XML. Podemos ver isso na prática, criando um documento no Word composto de algum texto e uma imagem. Ao salvar este documento, podemos renomear o arquivo docx para zip e abri-lo com qualquer programa que abra arquivos zip:

Como podemos ver na Figura acima, o arquivo contém, na raiz, três diretórios, _rels, docProps e word e um arquivo, [Content_Types].xml. Esta estrutura de diretórios é criada pelo Word, e não é obrigatório que ela seja mantida. A localização dos arquivos fica no arquivo .rels, que está no diretório _rels. Este arquivo contém as relações entre o pacote e os arquivos de nível superior.  O seguinte código mostra o arquivo .rels criado em nosso exemplo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
  <Relationship Id="rId3" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties" Target="docProps/core.xml"/>
  <Relationship Id="rId2" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/thumbnail" Target="docProps/thumbnail.wmf"/>
  <Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
  <Relationship Id="rId4" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/extended-properties" Target="docProps/app.xml"/>
</Relationships>

Analisando este arquivo, vemos o seguinte:

  • As propriedades principais (core-properties) estão no arquivo docProps/core.xml.
  • A imagem reduzida (thumbnail) está em docProps/thumbnail.wmf.
  • O documento principal (officeDocument) está em word/document.xml.
  • As propriedades estendidas (extended-properties) estão em docProps/app.xml.

Com base nestas informações, podemos abrir os arquivos que compõem o pacote OpenXML.

Adicionalmente, podemos ver que no diretório word existem um subdiretório _rels, que contém as relações para o documento. No arquivo document.txt.rels, encontramos as seguintes relações:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
  <Relationship Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/webSettings" Target="webSettings.xml"/>
  <Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/settings" Target="settings.xml"/>
  <Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>
  <Relationship Id="rId6" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme" Target="theme/theme1.xml"/>
  <Relationship Id="rId5" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/fontTable" Target="fontTable.xml"/>
  <Relationship Id="rId4" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/image1.png"/>
</Relationships>

Aqui são encontradas as relações para o documento. Vemos que os estilos aplicados no documento encontram-se no arquivo styles.xml e que a imagem que adicionamos está em media/image1.png. Assim, podemos acessar qualquer parte do documento.

A seguir iremos criar um pequeno programa em Delphi que abre um arquivo do Office e lista suas propriedades (tanto as principais como as estendidas) em um componente TValueListEditor.

Acessando os arquivos OpenXML

Para acessar os arquivos OpenXML , precisamos dividir nosso programa nas seguintes partes:

  • Abrir o pacote OpenXML com um componente que permite ler e gravar arquivos zip
  • Abrir o arquivo .rels em _rels e ler as relações, extraindo a localização das partes que nos interessam
  • Acessar as partes, executando a funcionalidade desejada.

Para abrir os arquivos Zip, usaremos o componente TZipFile, que está disponível no Delphi a partir da versão XE2. Este componente permite manipular arquivos zip de maneira relativamente simples. Crie um novo projeto no Delphi e adicione à Form um botão, um OpenDialog e um Memo.

Configure a propriedade Caption para Abrir. Configure a propriedade Filter do OpenDialog para “Arquivos Word (*.docx, *.docm)|*.docx;*.docm| Arquivos Excel (*.xlsx, *.xlsm)|*.xlsx;*.xlsm| Arquivos Powerpoint (*.pptx, *.pptm)|*.pptx;*.pptm”.

No handler do evento OnClick do botão, coloque o seguinte código:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipStream: TStream;
  XmlNode: IXMLNode;
  i: Integer;
  AttType: String;
  ZipFile: TZipFile;
  LocalHeader: TZipHeader;
begin
  if OpenDialog1.Execute then begin
    ZipFile := TZipFile.Create();
    try
      ZipFile.Open(OpenDialog1.FileName, TZipMode.zmRead);
      try
        // Lê relações
        ZipFile.Read('_rels/.rels', ZipStream, LocalHeader);
        ZipStream.Position := 0;
        XMLDocument1.LoadFromStream(ZipStream);
        Memo1.Text := XMLDoc.FormatXMLData(XMLDocument1.XML.Text);
      finally
        ZipStream.Free;
      end;
    finally
      ZipFile.Close();
      ZipFile.Free;
    end;
  end;
end;

Se o usuário escolher um arquivo, indicamos o nome do arquivo para o componente ZipFile e depois extraímos o arquivo .rels para um stream e carregamos as linhas do Memo com este stream, formatado com a função FormatXMLData. A figura a seguir mostra o resultado da execução.

Uma vez que temos o arquivo .rels, devemos lê-lo e interpretar as relações. Poderíamos usar as funções de leitura de arquivos texto e interpretar o documento, porém essa não é a melhor maneira de fazer esta operação. O ideal é usar um componente próprio para a leitura de arquivos XML, como o componente TXMLDocument, que vem com o Delphi na guia Internet da paleta de componentes.

Coloque dois componentes TXMLDocument e um componente TValueListEditor na Form.  Modifique a propriedade TitleCaptions do TValueListEditor para Propriedade/Valor. No evento OnClick do botão, modifique o código para:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipStream: TStream;
  XmlNode: IXMLNode;
  i: Integer;
  AttType: String;
  ZipFile: TZipFile;
  LocalHeader: TZipHeader;
begin
  if OpenDialog1.Execute then begin
    ZipFile := TZipFile.Create();
    try
      ZipFile.Open(OpenDialog1.FileName, TZipMode.zmRead);
      // Lê relações
      ZipFile.Read('_rels/.rels', ZipStream, LocalHeader);
      try
        ZipStream.Position := 0;
        XMLDocument1.LoadFromStream(ZipStream);
        Memo1.Text := XMLDoc.FormatXMLData(XMLDocument1.XML.Text);
        ValueListEditor1.Strings.Clear;
        // Processa nós
        for i := 0 to XMLDocument1.DocumentElement.ChildNodes.Count - 1 do begin
          XmlNode := XMLDocument1.DocumentElement.ChildNodes.Nodes[i];
          // Pega o tipo de relação.
          // Ela é a parte final do atributo Type
          AttType := ExtractFileName(XmlNode.Attributes['Type']);
          if AttType.EndsWith('core-properties') or
             AttType.EndsWith('extended-properties') then
            // Adiciona as propriedades no ValueListEditor
            LePropriedades(ZipFile, XmlNode.Attributes['Target']);
        end;
      finally
        ZipStream.Free;
      end;
    finally
      ZipFile.Close();
      ZipFile.Free;
    end;
  end;
end;

Carregamos o stream em XMLDocument1 e processamos os nós, para encontrar aqueles com os tipos que queremos (core-properties ou extended-properties). Quando os encontramos, passamos o nome do arquivo (que está no atributo Target) para a função LePropriedades, que irá ler o arquivo de propriedades e adicioná-las ao ValueListEditor. A função LePropriedades é:

procedure TMainFrm.LePropriedades(ZipFile: TZipFile; const Arquivo: String);
var
  ZipStream: TStream;
  i: Integer;
  XmlNode: IXMLNode;
  LocalHeader: TZipHeader;
begin
  ZipFile.Read(Arquivo, ZipStream, LocalHeader);
  try
    ZipStream.Position := 0;
    XMLDocument2.LoadFromStream(ZipStream);
    // Lê as propriedades
    for i := 0 to XMLDocument2.DocumentElement.ChildNodes.Count - 1 do
    begin
      XmlNode := XMLDocument2.DocumentElement.ChildNodes.Nodes[i];
      try
        // Achou nova propriedade adiciona
        ValueListEditor1.InsertRow(XmlNode.NodeName, XmlNode.NodeValue, True);
      except
        // Propriedade não é um valor simples - despreza.
        On EXMLDocError do;
        // Propriedade é nula - adiciona sting nulo
        On EVariantTypeCastError do
          ValueListEditor1.InsertRow(XmlNode.NodeName, '', True);
      end;
    end;
  finally
    ZipStream.Free;
  end;
end;

Esta função é parecida com a anterior. Iremos ler o arquivo de propriedades no segundo TXMLDocument e inserir uma linha no ValueListEditor para cada propriedade encontrada. Tratamos aqui dois tipos de exceção: EXMLDocError, que pode acontecer quando o tipo de dado não é um tipo simples, como um string ou inteiro e EVariantTypeCastError, que acontece quando o valor é nulo. Desta maneira, adicionamos as propriedades na lista, como mostra a figura a seguir.

Como podemos ver, o acesso aos dados de um arquivo OpenXML é relativamente simples, e pode ser feito usando componentes disponíveis no Delphi, mas isso não é tudo que pode ser feito: como estamos trabalhando com pacotes zip normais e arquivos XML, usando tecnologia aberta, podemos também modificar os arquivos, usando as mesmas técnicas.

Até agora, vimos como acessar um arquivo. Porém, nada impede que possamos criar um arquivo a partir de nossos dados.

Criação de um arquivo OpenXML

Para gerar um arquivo OpenXML, precisamos criar alguns arquivos que irão compor o pacote. O pacote mínimo deve conter três arquivos:

  • [Content_Types].xml
  • _rels/.rels
  • xml

Não é necessário criar uma estrutura de diretórios como a do Word, basta apenas que indiquemos no arquivo .rels a localização dos dados. À medida que  vamos adicionando novas funcionalidades, como imagens, cabeçalhos, temas e estilos, precisamos incluir novos arquivos para adicionar estas partes ao documento. Inicialmente, iremos criar um arquivo simples, para mostrar o processo de geração de um arquivo e, em seguida, mostraremos como criar um arquivo mais complexo.

Crie um novo projeto e coloque um Label, um Memo e um botão na Form. Mude a propriedade Caption do Label para Texto:, a propriedade Caption do Button para Criar e limpe a propriedade Lines do Memo.

Coloque um componente XMLDocument. No evento OnClick do botão, coloque o seguinte código:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  zipFile: TZipFile;
  contentTypes: TStream;
  rels: TStream;
  doc: TStream;
begin
  zipFile := TZipFile.Create();
  try
    zipFile.Open('ArquivoSimples.docx', TZipMode.zmWrite);
    contentTypes := CriaContentTypes();
    try
      zipFile.Add(contentTypes, '[Content_Types].xml');
    finally
      contentTypes.Free;
    end;
    rels := CriaRels();
    try
      zipFile.Add(rels, '_rels\.rels');
    finally
      rels.Free;
    end;
    doc := CriaDocumento();
    try
      zipFile.Add(doc, 'word\document.xml');
    finally
      doc.Free;
    end;
  finally
    zipFile.Close();
    zipFile.Free;
  end;
end;

O programa irá criar os diversos arquivos necessários, adicionar os streams para o arquivo zip e criar um arquivo com o nome ArquivoSimples.docx. A função que cria o arquivo [Content_Types.xml] é:

function TMainFrm.CriaContentTypes(): TStream;
var
  Root: IXmlNode;
  Tipo: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  // Nó raiz
  Root := XMLDoc.addChild('Types',
    'http://schemas.openxmlformats.org/package/2006/content-types');
  // Definição de tipos
  Tipo := Root.addChild('Default');
  Tipo.Attributes['Extension'] := 'rels';
  Tipo.Attributes['ContentType'] :=
    'application/vnd.openxmlformats-package.relationships+xml';
  Tipo := Root.addChild('Default');
  Tipo.Attributes['Extension'] := 'xml';
  Tipo.Attributes['ContentType'] :=
    'application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml';
  // Grava no stream de saída
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

A função que cria o arquivo de relações é:

function TMainFrm.CriaRels(): TStream;
var
  Root: IXmlNode;
  Rel: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  // Nó raiz
  Root := XMLDoc.addChild('Relationships',
    'http://schemas.openxmlformats.org/package/2006/relationships');
  // Definição de relações
  Rel := Root.addChild('Relationship');
  Rel.Attributes['Id'] := 'rId1';
  Rel.Attributes['Type'] :=
    'http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument';
  Rel.Attributes['Target'] := 'word/document.xml';
  // Grava no stream de saída
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

O código para gravar o documento com o texto digitado no Memo é:

function TMainFrm.CriaDocumento(): TStream;
var
  Root: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  // Nó raiz
  Root := XMLDoc.addChild('wordDocument',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  // Grava texto
  Root.addChild('body').addChild('p').addChild('r').addChild('t').NodeValue :=
    Memo1.Text;
  // Grava no stream de saída
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

Aqui apenas precisamos gravar um nó dentro do nó raiz wordDocument: ele  é o corpo do documento, que tem um parágrafo (nó p), um “run” (nó r) e o texto, que é o conteúdo do Memo. Ao compilar e executar o programa, podemos digitar um texto no Memo e clicar no botão Criar. O arquivo docx é criado com o texto digitado.

Colocando mais informações no arquivo

Uma vez que sabemos como criar nossos arquivos, podemos adicionar mais informações no que está sendo criado. Criaremos agora um exemplo que mostra as fontes disponíveis no sistema. Este documento será gerado em formato paisagem, e colocaremos um cabeçalho com três colunas e o número da página.

Crie um novo projeto e coloque um botão, um XmlDocument  Altere a propriedade Caption do botão para Criar. No evento OnClick do botão, coloque:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipFile: TZipFile;
  MemStream: TMemoryStream;
begin
  ZipFile := TZipFile.Create();
  try
    ZipFile.Open('ArquivoComplexo.docx', TZipMode.zmWrite);
    MemStream := TMemoryStream.Create();
    try
      CriaContentTypes(MemStream);
      ZipFile.Add(MemStream, '[Content_Types].xml');
      MemStream.Clear;
      CriaRels(MemStream);
      ZipFile.Add(MemStream, '_rels\.rels');
      MemStream.Clear;
      CriaDocumento(MemStream);
      ZipFile.Add(MemStream, 'word\document.xml');
    finally
      MemStream.Free;
    end;
  finally
    ZipFile.Close();
    ZipFile.Free;
  end;
end;

As funções CriaRels e CriaContentTypes são as mesmas da rotina anterior. A função CriaDocumento é a seguinte:

procedure TMainFrm.CriaDocumento(AStream: TStream);
var
  Root, Body, PgSz: IXMLNode;
  i: Integer;
  SectPr: IXMLNode;
  Header: IXMLNode;
begin
  LimpaXML;
  CriaCabecalho;
  // Nó raiz
  Root := XMLDocument1.addChild('w:wordDocument');
  Root.DeclareNamespace('w',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  Body := Root.addChild('w:body');
  for i := 0 to Screen.Fonts.Count - 1 do
    AdicionaFonte(Body, Screen.Fonts[i]);
  
  // Grava no stream de saída
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

Iremos varrer as fontes do sistema, chamando a função AdicionaFonte, que irá adicionar o texto formatado no arquivo Document.xml:

procedure TMainFrm.AdicionaFonte(Body: IXMLNode; NomeFonte: String);
var
  Fonte: IXMLNode;
  Run: IXMLNode;
  RunPr: IXMLNode;
begin
  Run := Body.addChild('w:p').addChild('w:r');
  RunPr := Run.addChild('w:rPr');
  Fonte := RunPr.addChild('w:rFonts');
  Fonte.Attributes['w:ascii'] := NomeFonte;
  Fonte.Attributes['w:hAnsi'] := NomeFonte;
  Fonte.Attributes['w:cs'] := NomeFonte;
  RunPr.addChild('w:sz').Attributes['w:val'] := 30;
  Run.addChild('w:t').NodeValue := NomeFonte;
  Run.addChild('w:tab');
  Run.addChild('w:t').NodeValue :=
    'The quick brown fox jumps over the lazy dog';
end;

Para cada fonte do sistema, adicionamos um parágrafo e, nele, um Run. O Run deve ser formatado com o elemento rPr, colocando-se como filho o elemento rFonts e o nome da fonte como valores dos atributos ascii,  hAnsi e cs. Também mudamos o tamanho da fonte adicionando o elemento sz. Em seguida, colocamos o nome da fonte como texto, adicionando o elemento tab para gerar uma tabulação e um texto de exemplo. Ao rodar o programa, vemos que a lista de fontes é gerada no documento.

O próximo passo é fazer que o documento seja colocado em paisagem. Para isso, devemos adicionar ao final do documento um elemento sectPr (propriedades da seção), que indica a formatação da seção. Coloque o seguinte código ao final de CriaDocumento, antes da linha   XMLDocument1.SaveToStream(AStream):

SectPr := Body.addChild('sectPr');
PgSz := SectPr.addChild('w:pgSz');
PgSz.Attributes['w:w'] := Round(297 / 25.4 * 1440);
PgSz.Attributes['w:h'] := Round(210 / 25.4 * 1440);
PgSz := SectPr.addChild('w:pgMar');
PgSz.Attributes['w:top'] := 1440;
PgSz.Attributes['w:bottom'] := 1440;
PgSz.Attributes['w:left'] := 720;
PgSz.Attributes['w:right'] := 720;
PgSz.Attributes['w:header'] := 720;
PgSz.Attributes['w:footer'] := 720;

Neste código adicionamos o elemento pgSz (Page size), dando os atributos w e h para a largura e altura da página. Estas medidas são em twips (1/1440 de polegada), assim fazemos a conversão do tamanho da página A4 para twips. Em seguida, colocamos o elemento pgMar (Page margins), que determina as margens da página e a posição do cabeçalho e rodapé. Ao rodarmos o programa e abrirmos o documento, vemos que ele está em paisagem.

O último passo é colocar o cabeçalho. Colocamos o cabeçalho em um arquivo separado e, desta maneira, devemos alterar todas as referências para que este novo documento seja lido.

Inicialmente, criamos uma referência para o cabeçalho na seção, como filho de sectPr. Coloque o seguinte código em CriaDocumento, após a linha SectPr := Body.AddChild(‘sectPr’):

Header := SectPr.addChild('w:headerReference');
Header.Attributes['w:type'] := 'default';
Header.Attributes['r:id'] := 'rId1';

Para usar as referências, devemos adicionar um novo namespace ao documento. Isto é feito adicionando a seguinte linha após a declaração do namespace em CriaDocumento:

Root.DeclareNamespace('r', 'http://schemas.openxmlformats.org/officeDocument/2006/relationships');

Criamos uma referência rId1 no documento. Devemos então criar uma função que cria este relacionamento no arquivo word\_rels\document.xml.rels:

procedure TMainFrm.CriaDocRels(AStream: TStream);
var
  Root: IXMLNode;
  Rel: IXMLNode;
begin
  LimpaXML;
  CriaCabecalho;
  // Nó raiz
  Root := XMLDocument1.addChild('Relationships',
    'http://schemas.openxmlformats.org/package/2006/relationships');
  // Definição de relações
  Rel := Root.addChild('Relationship');
  Rel.Attributes['Id'] := 'rId1';
  Rel.Attributes['Type'] :=
    'http://schemas.openxmlformats.org/officeDocument/2006/relationships/header';
  Rel.Attributes['Target'] := 'header1.xml';
  // Grava no stream de saída
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

Esta função é semelhante à que cria o relacionamento do pacote. A função que cria o cabeçalho no arquivo header1.xml é:

procedure TMainFrm.CriaHeader(AStream: TStream);
var
  Root, Header, PTab: IXMLNode;
begin
  LimpaXML;
  CriaCabecalho;
  // Nó raiz
  Root := XMLDocument1.addChild('w:hdr');
  Root.DeclareNamespace('w',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  Header := Root.addChild('w:p');
  Header.addChild('w:r').addChild('w:t').NodeValue := 'Texto 1';
  PTab := Header.addChild('w:r').addChild('w:ptab');
  PTab.Attributes['w:relativeTo'] := 'margin';
  PTab.Attributes['w:alignment'] := 'center';
  PTab.Attributes['w:leader'] := 'none';
  Header.addChild('w:r').addChild('w:t').NodeValue := 'Texto 2';
  PTab := Header.addChild('w:r').addChild('w:ptab');
  PTab.Attributes['w:relativeTo'] := 'margin';
  PTab.Attributes['w:alignment'] := 'right';
  PTab.Attributes['w:leader'] := 'none';
  Header.addChild('w:fldSimple').Attributes['w:instr'] := 'PAGE \* MERGEFORMAT';
  // Grava no stream de saída
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

Aqui criamos o cabeçalho com um texto alinhado à esquerda, uma tabulação para alinhar o texto centralizado e outra tabulação para alinhar o número da página à direita. O número da página é dado pelo elemento fldSimple, usando-se  o atributo instr com o valor PAGE \* MERGEFORMAT. Após criar estas funções devemos colocar o código para chamá-las, ao final do evento OnClick do botão:

MemStream.Clear;
CriaDocRels(MemStream);
ZipFile.Add(MemStream, 'word\_rels\document.xml.rels');
MemStream.Clear;
CriaHeader(MemStream);
ZipFile.Add(MemStream, 'word\header1.xml');

Agora, precisamos apenas fazer uma pequena mudança em [Content_Types].xml, adicionando o elemento Override, para determinar o tipo de header1.xml. Coloque o seguinte código em CriaContentTypes, antes da linha XMLDocument1.SaveToStream(AStream):

Tipo := Root.addChild('Override');
Tipo.Attributes['PartName'] := '/word/header1.xml';
Tipo.Attributes['ContentType'] :=
  'application/vnd.openxmlformats-officedocument.wordprocessingml.header+xml';

Com isso, nosso programa está pronto. Ao executá-lo, geramos um documento semelhante ao mostrado na figura a seguir:

Conclusões

O formato OpenXML traz grandes vantagens para quem quer processar e abrir arquivos do Office. Como este formato usa tecnologias abertas e está completamente documentado, podemos acessar, alterar ou mesmo criar arquivos do Office usando quaisquer ferramentas de desenvolvimento (ou mesmo alterando-se os arquivos manualmente), em qualquer plataforma ou linguagem.

Não são necessárias APIs proprietárias ou programas especiais, o que permite que a informação esteja disponível para qualquer um que queira acessá-la. Mostramos aqui como manipular os arquivos em Delphi, fazendo notar que utilizamos apenas componentes padrão do Delphi, utilizando apenas arquivos  zip e XML.

O código fonte para este projeto está em https://github.com/bsonnino/OpenXmlDelphiPort

 

Introduction

Office 2007 brought many significant changes for this app suite. The most visible was the Ribbon, an new user interface that eases the use of the applications.

Another change, less visible, allows the integration of Office applications with a large variety of programs: the new file format. Until the previous version, the file format was proprietary: when you wanted to open or save an Office document in our applications, you should use Ole Automation, what required that Office was installed in the client’s machine, or try to discover the internal file format, which wasn’t documented and could be changed at any time.

The new file format, besides being documented, is based in open standards, thus allowing that any aplication for any platform, written in any language to open or create Office 2007 files. This new standard, named OpenXML, is based on the zip packaging and XML files. It creates smaller files and allows other applications to open and change these files.

This opens a lot of possibilities:

  • Programs to index and search text from the files in the machine
  • Programs for batch generation of documents, based on databases and templates
  • Programs for batch text replacement
  • Simple text editors that generate Office files
  • Spreadsheet generation using data from many sources

In this article, we will show the new file format and how we can read and crete them using Delphi, with no need to install Ofiice.

Analyzing an OpenXML file

Any OpenXML file is, in fact, a zip file with many folders and XML files. We can see that in practice, by creating a Word file with some text and save it. If we rename this file to zip, we can open it with any program that can open zip files:

As you can see in the figure above, the file contains in the root three directories,  _rels, docProps and word and a file, [Content_Types].xml. This directory structure is created by Word and it’s not obligatory to maintain it. The files location is in the .rels file, located in the  _rels folder. This file contais the relations between the package and the files in the upper level. The following code shows the rels file from the example:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
   <Relationship Id="rId3" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties" Target="docProps/core.xml"/>
   <Relationship Id="rId2" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/thumbnail" Target="docProps/thumbnail.wmf"/>
   <Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
   <Relationship Id="rId4" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/extended-properties" Target="docProps/app.xml"/>
 </Relationships>

Analyzing this file, we see the following:

  • The core properties are in the docProps/core.xml file.
  • The thumbnail is in the docProps/thumbnail.wmf file.
  • The main document is in word/document.xml.
  • The extended-properties are in docProps/app.xml.

Besides that, we can see that the word directory contains a _rels subdirectory, which contains the relations for the document. In the file document.txt.rels, we find the following relations:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">   
  <Relationship Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/webSettings" Target="webSettings.xml"/>   
  <Relationship Id="rId2" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/settings" Target="settings.xml"/>   
  <Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>   
  <Relationship Id="rId6" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme" Target="theme/theme1.xml"/>   
  <Relationship Id="rId5" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/fontTable" Target="fontTable.xml"/>   
  <Relationship Id="rId4" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/image" Target="media/image1.png"/> 
</Relationships>

Here you can find the relations for the document. We see that the styles used in the document are in styles.xml and any image in the document is in the media folder. That way, we can access any part of the document.

Based on these informations, we can open the files that are in the OpenXML package. Now, we will create a small Delphi program that opens an Office file and lists its properties in a component TValueListEditor.

Accessing OpenXml files

To open OpenXml files, we need to divide our program in the following parts:

  • Open the OpenXml package with a component that allows to read and write zip files
  • Open the .rels file and read the relations, extracting the parts that interest us
  • Access the parts, executing what we want

To open the zip files, we will use the TZipFile component, which is available in Delphi since version XE2. This component allows to manipulate zip files in a relatively simple way. You should create a new Delphi project and add to the main form a button, an OpenDialog and a Memo.

Configure the Caption property of the button to Open. Configure the Filter property of the OpenDialog to “Word Files (*.docx, *.docm)|*.docx;*.docm| Excel Files(*.xlsx, *.xlsm)|*.xlsx;*.xlsm| Powerpoint Files(*.pptx, *.pptm)|*.pptx;*.pptm”.

On the OnClick event handler of the button, add the following code:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipStream: TStream;
  XmlNode: IXMLNode;
  i: Integer;
  AttType: String;
  ZipFile: TZipFile;
  LocalHeader: TZipHeader;
begin
  if OpenDialog1.Execute then begin
    ZipFile := TZipFile.Create();
    try
      ZipFile.Open(OpenDialog1.FileName, TZipMode.zmRead);
      try
        ZipFile.Read('_rels/.rels', ZipStream, LocalHeader);
        ZipStream.Position := 0;
        XMLDocument1.LoadFromStream(ZipStream);
        Memo1.Text := XMLDoc.FormatXMLData(XMLDocument1.XML.Text);
      finally
        ZipStream.Free;
      end;
    finally
      ZipFile.Close();
      ZipFile.Free;
    end;
  end;
end;

If the user chooses a file, we open the file with the TZipFile and extract the .rels file to a stream and load the lines of the memo with this stream formatted with the FormatXMLData. The following figure shows the result of this operation:

Once we have the .rels file, we must read it and interpret the relations. We could use the functions to read text files and interpret the document, but this is not the best way to do this operation. The ideal is to use a component to read XML files, like the TXMLDocument component that comes with Delphi.

Put two TXMLDocument components and a TValueListEditor on the form. Modify the TileCaptions property of the TValueListEditor to Property/Value. On the OnClick handler of the button modify the code to this one:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipStream: TStream;
  XmlNode: IXMLNode;
  i: Integer;
  AttType: String;
  ZipFile: TZipFile;
  LocalHeader: TZipHeader;
begin
  if OpenDialog1.Execute then begin
    ZipFile := TZipFile.Create();
    try
      ZipFile.Open(OpenDialog1.FileName, TZipMode.zmRead);
      ZipFile.Read('_rels/.rels', ZipStream, LocalHeader);
      try
        ZipStream.Position := 0;
        XMLDocument1.LoadFromStream(ZipStream);
        Memo1.Text := XMLDoc.FormatXMLData(XMLDocument1.XML.Text);
        ValueListEditor1.Strings.Clear;
        for i := 0 to XMLDocument1.DocumentElement.ChildNodes.Count - 1 do begin
          XmlNode := XMLDocument1.DocumentElement.ChildNodes.Nodes[i];
          AttType := ExtractFileName(XmlNode.Attributes['Type']);
          if AttType.EndsWith('core-properties') or
             AttType.EndsWith('extended-properties') then
            ReadProperties(ZipFile, XmlNode.Attributes['Target']);
        end;
      finally
        ZipStream.Free;
      end;
    finally
      ZipFile.Close();
      ZipFile.Free;
    end;
  end;
end;

We load the stream in XMLDocument1 and process the nodes, to find the ones with the types we want (core-properties or extended-properties). When we find them, we pass the name of the file (which is in the Target attribute) to the ReadProperties function, which will read the property file and add them to the ValueListEditor. The ReadProperties  function is:

procedure TMainFrm.ReadProperties(ZipFile: TZipFile; const FileName: String);
var
  ZipStream: TStream;
  i: Integer;
  XmlNode: IXMLNode;
  LocalHeader: TZipHeader;
begin
  ZipFile.Read(FileName, ZipStream, LocalHeader);
  try
    ZipStream.Position := 0;
    XMLDocument2.LoadFromStream(ZipStream);
    for i := 0 to XMLDocument2.DocumentElement.ChildNodes.Count - 1 do begin
      XmlNode := XMLDocument2.DocumentElement.ChildNodes.Nodes[i];
      try
        ValueListEditor1.InsertRow(XmlNode.NodeName, XmlNode.NodeValue, True);
      except
        On EXMLDocError do;
        On EVariantTypeCastError do
          ValueListEditor1.InsertRow(XmlNode.NodeName, '', True);
      end;
    end;
  finally
    ZipStream.Free;
  end;
end;

 

This function is similar with the previous one. We will read the properties file in the second TXMLDcoument and insert a line in the ValueListEditor for each property found. We treat here two types of exceptions: EXMLDocError, which can be raised when the type of the information is not a single type, like a string or an integer and EVariantTypeCastError, which happens when the value is null. This way, we add the properties on the list, like in the next figure:

As we can see, the access to the data of an OpenXml file is relatively simple and can be made using components available in Delphi, but this isn’t everything that can be done: as we are working with zip and xml files, using open technology, we can also modify the files, using the same techniques. In the next section, we’ll see how to create a file from our data.

Creating an OpenXml file

To create an OpenXml file, we need to create some files that will be added to the package. The package should contain at least three files:

  • [Content_Types].xml
  • _rels/.rels
  • xml

It’s not necessary to create a folder structure like the one created by Word, we just need to point the location of the files in the .rels file. When adding new functionalities, like images, headers, themes and styles, we must add new files to add these parts to the document. Initially, we will create a simple file, to show the file generation process and then, we will show how to create a more complex file.

Create a new project and place a Label, a Memo and a button on the Form. Change the Caption property of the Label to Text :, the Caption property of the Button to Create and clear the Lines property of the Memo.

Place an XMLDocument component. In the button’s OnClick event, place the following code:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  zipFile: TZipFile;
  contentTypes: TStream;
  rels: TStream;
  doc: TStream;
begin
  zipFile := TZipFile.Create();
  try
    zipFile.Open('SimpleFile.docx', TZipMode.zmWrite);
    contentTypes := CreateContentTypes();
    try
      zipFile.Add(contentTypes, '[Content_Types].xml');
    finally
      contentTypes.Free;
    end;
    rels := CreateRels();
    try
      zipFile.Add(rels, '_rels\.rels');
    finally
      rels.Free;
    end;
    doc := CreateDoc();
    try
      zipFile.Add(doc, 'word\document.xml');
    finally
      doc.Free;
    end;
  finally
    zipFile.Close();
    zipFile.Free;
  end;
end;

The program will create the necessary files, add the streams to the zip file and create a file with the name SimpleFile.docx. The function that creates the file [Content_Types.xml] is:

function TMainFrm.CreateContentTypes(): TStream;
var
  Root: IXmlNode;
  Type: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  Root := XMLDoc.addChild('Types',
    'http://schemas.openxmlformats.org/package/2006/content-types');
  Type := Root.addChild('Default');
  Type.Attributes['Extension'] := 'rels';
  Type.Attributes['ContentType'] :=
    'application/vnd.openxmlformats-package.relationships+xml';
  Type := Root.addChild('Default');
  Type.Attributes['Extension'] := 'xml';
  Type.Attributes['ContentType'] :=
    'application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml';
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

The function that create the relations file is:

function TMainFrm.CreateRels(): TStream;
var
  Root: IXmlNode;
  Rel: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  Root := XMLDoc.addChild('Relationships',
    'http://schemas.openxmlformats.org/package/2006/relationships');
  Rel := Root.addChild('Relationship');
  Rel.Attributes['Id'] := 'rId1';
  Rel.Attributes['Type'] :=
    'http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument';
  Rel.Attributes['Target'] := 'word/document.xml';
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

The code to write the document with the text entered in the Memo is:

function TMainFrm.CreateDoc(): TStream;
var
  Root: IXmlNode;
  XMLDoc: IXmlDocument;
begin
  Result := TMemoryStream.Create();
  XMLDoc := CriaXml;
  Root := XMLDoc.addChild('wordDocument',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  Root.addChild('body').addChild('p').addChild('r').addChild('t').NodeValue :=
    Memo1.Text;
  XMLDoc.SaveToStream(Result);
  Result.Position := 0;
end;

Here we just need to write a node inside the wordDocument root node: it is the body of the document, which has a paragraph (node p), a “run” (node r) and the text, which is the content of the Memo. When compiling and running the program, we can type some text in the Memo and click on the Create button. The docx file is created with the typed text.

Putting more information in the file

Once we know how to create our files, we can add more information to what is being created. We will now create an example that shows all the fonts available in the system. This document will be generated in landscape format, and we will put a header with three columns and the page number.

Create a new project and place a button and an XmlDocument Change the Caption property of the button to Create. In the button’s OnClick event, put:

procedure TMainFrm.Button1Click(Sender: TObject);
var
  ZipFile: TZipFile;
  MemStream: TMemoryStream;
begin
  ZipFile := TZipFile.Create();
  try
    ZipFile.Open('ComplexFile.docx', TZipMode.zmWrite);
    MemStream := TMemoryStream.Create();
    try
      CreateContentTypes(MemStream);
      ZipFile.Add(MemStream, '[Content_Types].xml');
      MemStream.Clear;
      CreateRels(MemStream);
      ZipFile.Add(MemStream, '_rels\.rels');
      MemStream.Clear;
      CreateDoc(MemStream);
      ZipFile.Add(MemStream, 'word\document.xml');
    finally
      MemStream.Free;
    end;
  finally
    ZipFile.Close();
    ZipFile.Free;
  end;
end;

The functions CreateRels and CreateContentTypes are the same as the previous routine. The function CreateDocument is the following:

procedure TMainFrm.CreateDocument(AStream: TStream);
var
  Root, Body, PgSz: IXMLNode;
  i: Integer;
  SectPr: IXMLNode;
  Header: IXMLNode;
begin
  LimpaXML;
  CreateHeader;
  Root := XMLDocument1.addChild('w:wordDocument');
  Root.DeclareNamespace('w',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  Body := Root.addChild('w:body');
  for i := 0 to Screen.Fonts.Count - 1 do
    AddFont(Body, Screen.Fonts[i]);
  
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

We will loop for the system fonts, calling the function AddFont, which will add the formatted text in the Document.xml file:

procedure TMainFrm.AddFont(Body: IXMLNode; NomeFonte: String);
var
  Fonte: IXMLNode;
  Run: IXMLNode;
  RunPr: IXMLNode;
begin
  Run := Body.addChild('w:p').addChild('w:r');
  RunPr := Run.addChild('w:rPr');
  Fonte := RunPr.addChild('w:rFonts');
  Fonte.Attributes['w:ascii'] := NomeFonte;
  Fonte.Attributes['w:hAnsi'] := NomeFonte;
  Fonte.Attributes['w:cs'] := NomeFonte;
  RunPr.addChild('w:sz').Attributes['w:val'] := 30;
  Run.addChild('w:t').NodeValue := NomeFonte;
  Run.addChild('w:tab');
  Run.addChild('w:t').NodeValue :=
    'The quick brown fox jumps over the lazy dog';
end;

For each font in the system, we add a paragraph and, in it, a Run. The Run must be formatted with the rPr element, placing the rFonts element and the font name as values of the ascii, hAnsi and cs attributes as children. We also changed the font size by adding the sz element. Then, we put the name of the font as text, adding the tab element to generate a tab and sample text. When running the program, we see that the list of fonts is generated in the document.

The next step is to have the document placed in landscape. To do this, we must add a sectPr element (section properties) to the end of the document, which indicates the formatting of the section. Place the following code at the end of CreateDocument, before the XMLDocument1.SaveToStream (AStream) line:

SectPr := Body.addChild('sectPr');
PgSz := SectPr.addChild('w:pgSz');
PgSz.Attributes['w:w'] := Round(297 / 25.4 * 1440);
PgSz.Attributes['w:h'] := Round(210 / 25.4 * 1440);
PgSz := SectPr.addChild('w:pgMar');
PgSz.Attributes['w:top'] := 1440;
PgSz.Attributes['w:bottom'] := 1440;
PgSz.Attributes['w:left'] := 720;
PgSz.Attributes['w:right'] := 720;
PgSz.Attributes['w:header'] := 720;
PgSz.Attributes['w:footer'] := 720;

In this code we add the element pgSz (Page size), giving the attributes w and h for the width and height of the page. These measurements are in twips (1/1440 of an inch), so we convert the page size from A4 to twips. Then, we put the pgMar element (Page margins), which determines the page margins and the position of the header and footer. When we run the program and open the document, we see that it is in landscape.

The last step is to place the header. We put the header in a separate file and, therefore, we must change all references so that this new document can be read.

Initially, we created a reference to the header in the section, as a child of sectPr. Place the following code in CreateDocument, after the line SectPr: = Body.AddChild (‘sectPr’):

Header := SectPr.addChild('w:headerReference');
Header.Attributes['w:type'] := 'default';
Header.Attributes['r:id'] := 'rId1';

To use the references, we must add a new namespace to the document. This is done by adding the following line after declaring the namespace in CreateDocument:

Root.DeclareNamespace ('r', 'http://schemas.openxmlformats.org/officeDocument/2006/relationships');

We created a reference rId1 in the document. We must then create a function that creates this relationship in the file word\_rels\document.xml.rels:

procedure TMainFrm.CreateDocRels(AStream: TStream);
var
  Root: IXMLNode;
  Rel: IXMLNode;
begin
  CleanXML;
  CreateHeader;
  Root := XMLDocument1.addChild('Relationships',
    'http://schemas.openxmlformats.org/package/2006/relationships');
  // Definição de relações
  Rel := Root.addChild('Relationship');
  Rel.Attributes['Id'] := 'rId1';
  Rel.Attributes['Type'] :=
    'http://schemas.openxmlformats.org/officeDocument/2006/relationships/header';
  Rel.Attributes['Target'] := 'header1.xml';
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

This function is similar to the one that creates the package relationship. The function that creates the header in the file  header1.xml is:

procedure TMainFrm.CreateHeader(AStream: TStream);
var
  Root, Header, PTab: IXMLNode;
begin
  CleanXML;
  CriaCabecalho;
  Root := XMLDocument1.addChild('w:hdr');
  Root.DeclareNamespace('w',
    'http://schemas.openxmlformats.org/wordprocessingml/2006/main');
  Header := Root.addChild('w:p');
  Header.addChild('w:r').addChild('w:t').NodeValue := 'Texto 1';
  PTab := Header.addChild('w:r').addChild('w:ptab');
  PTab.Attributes['w:relativeTo'] := 'margin';
  PTab.Attributes['w:alignment'] := 'center';
  PTab.Attributes['w:leader'] := 'none';
  Header.addChild('w:r').addChild('w:t').NodeValue := 'Texto 2';
  PTab := Header.addChild('w:r').addChild('w:ptab');
  PTab.Attributes['w:relativeTo'] := 'margin';
  PTab.Attributes['w:alignment'] := 'right';
  PTab.Attributes['w:leader'] := 'none';
  Header.addChild('w:fldSimple').Attributes['w:instr'] := 'PAGE \* MERGEFORMAT';
  XMLDocument1.SaveToStream(AStream);
  AStream.Position := 0;
end;

Here we create the header with text left-aligned, a tab to align the centered text and another tab to align the page number to the right.

The page number is given by the fldSimple element, using the instr attribute with the value PAGE\*MERGEFORMAT. After creating these functions, we must place the code to call them, at the end of the button’s OnClick event:

MemStream.Clear;
CreateDocRels(MemStream);
ZipFile.Add(MemStream, 'word\_rels\document.xml.rels');
MemStream.Clear;
CreateHeader(MemStream);
ZipFile.Add(MemStream, 'word\header1.xml');

Now, we need only to make a small change in [Content_Types].xml, adding the Override element, to show the type of the header1.xml file. Put the following code in the  CreateContentTypes, before the line XMLDocument1.SaveToStream(AStream):

Tipo := Root.addChild('Override');
Tipo.Attributes['PartName'] := '/word/header1.xml';
Tipo.Attributes['ContentType'] :=
  'application/vnd.openxmlformats-officedocument.wordprocessingml.header+xml';

With that, our program is ready. When executing it, we generate a document similar to the one shown in the following figure:

Conclusions

The OpenXML format has great advantages for those who want to process and open Office files. As this format uses open technologies and is fully documented, we can access, change or even create Office files using any development tools (or even changing files manually), on any platform or language.

No proprietary APIs or special programs are required, which allows the information to be available to anyone who wants to access it. We show here how to manipulate Delphi files, noting that we use only standard Delphi components, using only zip and XML files.

The source code for this project is in https://github.com/bsonnino/OpenXmlDelphiEng

 

Introduction

One of the interesting things that C#9 brought is the introduction of Code Generators. When compiling the code, the C# compiler can generate extra code and add it to your project, thus complementing your code. This has a lot of possible uses:

  • Add an attribute to your code to generate boilerplate code: you can add an attribute to a private member and the code generator will add the property and the INotifyPropertyChanged to your class.
  • Discover the dependencies needed at compile time and wire them in the executable. This can improve execution time, because wiring the dependencies using reflection is very slow. In this case, the dependencies would be already set up.
  • Parse files and generate code for them: you could have a json file with data and the compiler would add the classes without the need of creating them manually. Once the file has changed (and thus the class structure), a new class would be created.

One interesting use of this feature was used by the Windows SDK Team for using the Win32 APIs in our C# code. Instead of using the traditional PInvoke (which you must go to http://pinvoke.net to get the signatures and structures, add them as external methods and call them), you can use C#/Win32, developed to simplify the usage of Win32 APIs.

Using C#/Win32

To use C#/Win32, you must have .NET 5.102 installed in your machine and you must be using Visual Studio 16.8 or newer. With these prerequisites, you can create a new console application in the command line with:

dotnet new console -o UseWin32

This line will create a new console project in the UseWin32  folder. Then you must add the CsWin32  NuGet package with:

dotnet add package Microsoft.Windows.CsWin32 --prerelease

Now you are ready to open the project in Visual Studio or Visual Studio Code and use it. This first project will  enumerate the files in the current directory. Yes, I know that .NET already has methods to do that, but we can also do it using Win32.

Create a new text file and name it NativeMethods.txt. In this file, add the names of the thre API functions that we need:

FindFirstFile
FindNextFile
FindClose

Now, we are ready to use these methods in our program. In Program.cs, erase everything and add this code:

using System;
using System.Linq;
using Microsoft.Windows.Sdk;

var handle = PInvoke.FindFirstFile("*.*", out var findData);
if (handle.IsInvalid)
    return;

We are using another feature of C#9, Top Level Statements. We can see that CsWin32 has generated a new class named PInvoke  and has added the FindFirstFile  method. If you hoveer the mouse over it, you will see something like this:

As you can see, it has added the function and  also added the documentation for it. If you right-click in the function and select Go to Definition, it will open the generated code:

We can then continue the code to enumerate the files:

using System;
using System.Linq;
using Microsoft.Windows.Sdk;

var handle = PInvoke.FindFirstFile("*.*", out var findData);
if (handle.IsInvalid)
    return;
bool result;
do
{
    Console.WriteLine(ConvertFileNameToString(findData.cFileName.AsSpan()));
    result = PInvoke.FindNextFile(handle, out findData);
} while (result);
PInvoke.FindClose(handle);

string ConvertFileNameToString(Span<ushort> span)
{
    return string.Join("", span.ToArray().TakeWhile(i => i != 0).Select(i => (char)i));
}

This code opens the enumeration with FindFirstFile. If the returned handle is invalid, then there are no files in the folder, so the program exits. Then, it will print the file name to the console and continue the enumeration until the last file, when it calls FindClose, to close the handle. The filename is returned as a structure named __ushort_260, that can be converted to a Span<ushort> with the AsSpan method. To convert this to a string, we use the ConvertFileNameToString method, that uses Linq to convert it to a string: it takes all the items until it finds a 0, converts them to an IEnumerable<char> and then uses string.Join to convert this to a string.

If you use this code, you will see that FindClose(handle) has an error. That’s because the FileClose function receives a parameter of type HANDLE, while the handle variable is of the FileCloseSafeHandle type and both are not compatible (FileCloseSafeHandle has a handle field, but it’s protected and cannot be used). The solution, in this case, is to dispose the handle variable, that will call FindClose. This code shows how this is done:

using var handle = PInvoke.FindFirstFile("*.*", out var findData);
if (handle.IsInvalid)
    return;
bool result;
do
{
    Console.WriteLine(ConvertFileNameToString(findData.cFileName.AsSpan()));
    result = PInvoke.FindNextFile(handle, out findData);
} while (result);

We are using here the C#8’s Using Statement, so we don’t need to use a block. When you run this code, you will see the enumeration of the files in the console:

Function callbacks

As you can see, there is no need to dig to use the Win32 APIs, but there are some APIs that are more complex and use a callback function. These can also be used in the same way. to see that, we can enumerate all resources in an executable. To do that, we load the executable with LoadLibraryEx. Once loaded, we use EnumerateResourceTypes to enumerate all resource types and, for every resource type, we enumerate the resources with EnumerateResourceNames.

Create a new project with

dotnet new console -o EnumerateResources

Then add the CsWin32 NuGet package with

dotnet add package Microsoft.Windows.CsWin32 --prerelease

Then, open the project in Visual Studio and add a new text file and name it NativeMethods.txt. Add these functions in the file:

LoadLibraryEx
FreeLibrary
EnumResourceTypes
EnumResourceNames

In Program.cs, erase all text and add this code:

using System;
using Microsoft.Windows.Sdk;

var hInst = PInvoke.LoadLibraryEx(@"C:\Windows\Notepad.exe",
    null, LoadLibraryEx_dwFlags.LOAD_LIBRARY_AS_DATAFILE);
try
{

}
finally
{
    PInvoke.FreeLibrary(hInst);
}

This code calls LoadLibraryEx to load Notepad. The LOAD_LIBRARY_AS_DATAFILE constant was changed to an enum. This function returns the Instance handle, that will be used to enumerate the resources. At the end, we free the instance using FreeLibrary.

Now, we’ll start to enumerate the resources with:

var hInst = PInvoke.LoadLibraryEx(@"C:\Windows\Notepad.exe",
    null, LoadLibraryEx_dwFlags.LOAD_LIBRARY_AS_DATAFILE);
try
{
    PInvoke.EnumResourceTypes(hInst, EnumerateTypes, 0);
}
finally
{
    PInvoke.FreeLibrary(hInst);
}

You can see that the second parameter in EnumerateResourceTypes is a callback function (which I don’t even know the signature :-)). Let’s see if Visual Studio will help us with this. We type Ctrl-. and it will propose us to create the method. We select it and it is created:

BOOL EnumerateTypes(nint hModule, PWSTR lpType, nint lParam)
{
    throw new NotImplementedException();
}

We can add our code to enumerate the types:

BOOL EnumerateTypes(nint hModule, PWSTR lpType, nint lParam)
{
    Console.WriteLine(PwStrToString(lpType));
    PInvoke.EnumResourceNames(hModule, lpType, EnumNames, 0);
    return true;
}

This code will write the resource type to the console and enumerate the resources for that type. We are using a function, PwStrToString to convert the resource name (a PWSTR) to a string:

string PwStrToString(PWSTR str)
{
    unsafe
    {
        return ((ulong)str.Value & 0xFFFF0000) == 0 ?
            ((ulong)str.Value).ToString() :
            str.AsSpan().ToString();
    }
}

This struct has a Value property, that can be an integer or a string. To know which of them we must use, we test against the high order byte and see if it’s empty. If it is, then the value is an integer and we convert it to a string. If not, we convert it to a Span and get the string from it. All this code must be marked as unsafe, as we are working with the pointers.

EnumerateResourceNames has a callback, which we get the signature in the same way we did before, by using Visual Studio refactoring:

BOOL EnumNames(nint hModule, PCWSTR lpType, PWSTR lpName, nint lParam)
{
    Console.WriteLine("  " + PwStrToString(lpName));
    return true;
}

Now the program is complete and we can run it to list all notepad’s resources:

As you can see, working with the Win32 API is much easier now, we don’t have to use custom P/Invokes, everything is at one place and all we have to do is to add the functions we want to the NativeMethods file. This is really a great and welcome improvement.

Ah, and if you want to know what those resource type numbers are, you can find them here.

  • 4 – Menu
  • 5 – Dialog
  • 6 – String
  • 9 – Accelerator
  • 16 – Version
  • 24 – Manifest

All the source code for this article is at https://github.com/bsonnino/UseWin32

 

With .NET 5.0, two small features were introduced to Asp.NET and were almost unnoticed: Open Api and HTTPRepl. Open Api is not something new, it’s been available for a long time, but it had to be included explicitly in a new project. Now, when you create a new project, it’s automatically included in the project and you can get the Api documentation using Swagger.

Now, when you create a new project with

dotnet new webapi

You will create a new WebApi project with a Controller, WeatherController, that shows 10 values of a weather forecast:

It’s a simple app, but it already comes with the OpenApi (Swagger) for documentation. Once you type the address:

https://localhost:5001/swagger

You will get the Swagger documentation and will be able to test the service:

But there is more. Microsoft introduced also HttpRepl, a REPL (Read-Eval-Print Loop) for testing REST services. It will scan your service and allow you to test the service using simple commands, like the file commands.

To test this new feature, in  new folder create a webapi app with

dotnet new webapi

Then, open Visual Studio Code with

code .

You will get something like that:

Then, delete the WeatherForecast.cs file and add a new folder and name it Model. In it, add a new file and name it Customer.cs and add this code:

public class Customer
{
    public string CustomerId { get; set; }
    public string CompanyName { get; set; }
    public string ContactName { get; set; }
    public string ContactTitle { get; set; }
    public string Address { get; set; }
    public string City { get; set; }
    public string Region { get; set; }
    public string PostalCode { get; set; }
    public string Country { get; set; }
    public string Phone { get; set; }
    public string Fax { get; set; }
}

Create a new file and name it CustomerRepository.cs and add this code:

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using System.Xml.Linq;

namespace HttpRepl.Model
{
    public class CustomerRepository
    {
        private readonly IList<Customer> customers;

        public CustomerRepository()
        {
            var doc = XDocument.Load("Customers.xml");
            customers = new ObservableCollection<Customer>((from c in doc.Descendants("Customer")
                                                            select new Customer
                                                            {
                                                                CustomerId = GetValueOrDefault(c, "CustomerID"),
                                                                CompanyName = GetValueOrDefault(c, "CompanyName"),
                                                                ContactName = GetValueOrDefault(c, "ContactName"),
                                                                ContactTitle = GetValueOrDefault(c, "ContactTitle"),
                                                                Address = GetValueOrDefault(c, "Address"),
                                                                City = GetValueOrDefault(c, "City"),
                                                                Region = GetValueOrDefault(c, "Region"),
                                                                PostalCode = GetValueOrDefault(c, "PostalCode"),
                                                                Country = GetValueOrDefault(c, "Country"),
                                                                Phone = GetValueOrDefault(c, "Phone"),
                                                                Fax = GetValueOrDefault(c, "Fax")
                                                            }).ToList());
        }

        #region ICustomerRepository Members

        public bool Add(Customer customer)
        {
            if (customers.FirstOrDefault(c => c.CustomerId == customer.CustomerId) == null)
            {
                customers.Add(customer);
                return true;
            }
            return false;
        }

        public bool Remove(Customer customer)
        {
            if (customers.IndexOf(customer) >= 0)
            {
                customers.Remove(customer);
                return true;
            }
            return false;
        }

        public bool Update(Customer customer)
        {
            var currentCustomer = GetCustomer(customer.CustomerId);
            if (currentCustomer == null)
                return false;
            currentCustomer.CustomerId = customer.CustomerId;
            currentCustomer.CompanyName = customer.CompanyName;
            currentCustomer.ContactName = customer.ContactName;
            currentCustomer.ContactTitle = customer.ContactTitle;
            currentCustomer.Address = customer.Address;
            currentCustomer.City = customer.City;
            currentCustomer.Region = customer.Region;
            currentCustomer.PostalCode = customer.PostalCode;
            currentCustomer.Country = customer.Country;
            currentCustomer.Phone = customer.Phone;
            currentCustomer.Fax = customer.Fax;
            return true;    
        }

        public bool Commit()
        {
            try
            {
                var doc = new XDocument(new XDeclaration("1.0", "utf-8", "yes"));
                var root = new XElement("Customers");
                foreach (Customer customer in customers)
                {
                    root.Add(new XElement("Customer",
                                          new XElement("CustomerID", customer.CustomerId),
                                          new XElement("CompanyName", customer.CompanyName),
                                          new XElement("ContactName", customer.ContactName),
                                          new XElement("ContactTitle", customer.ContactTitle),
                                          new XElement("Address", customer.Address),
                                          new XElement("City", customer.City),
                                          new XElement("Region", customer.Region),
                                          new XElement("PostalCode", customer.PostalCode),
                                          new XElement("Country", customer.Country),
                                          new XElement("Phone", customer.Phone),
                                          new XElement("Fax", customer.Fax)
                                 ));
                }
                doc.Add(root);
                doc.Save("Customers.xml");
                return true;
            }
            catch (Exception)
            {
                return false;
            }
        }

        public IEnumerable<Customer> GetAll() => customers;

        public Customer GetCustomer(string id) => customers.FirstOrDefault(c => string.Equals(c.CustomerId, id, StringComparison.CurrentCultureIgnoreCase));

        #endregion

        private static string GetValueOrDefault(XContainer el, string propertyName)
        {
            return el.Element(propertyName) == null ? string.Empty : el.Element(propertyName).Value;
        }
    }
}

This code will use a file, named Customers.xml and will use it to serve the customers repository. With it, you will be able to get all customers, get, add, update or delete one customer. We will use it to serve our controller. The Customers.xml can be obtained at . You should add this file to the main folder and then add this code:

<ItemGroup >
  <None Update="customers.xml" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>

This will ensure that the xml file is copied to the output folder when the project is built.

Then, in the Controllers folder, delete the WeatherForecastController and add a new file, CustomerController.cs and add this code:

using System.Collections.Generic;
using HttpRepl.Model;
using Microsoft.AspNetCore.Mvc;

namespace HttpRepl.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class CustomerController : ControllerBase
    {
        [HttpGet]
        public IEnumerable<Customer> Get()
        {
            var customerRepository = new CustomerRepository();
            return customerRepository.GetAll();
        }

        [HttpGet("{id}")]
        public IActionResult GetCustomer(string id)
        {
            var customerRepository = new CustomerRepository();
            Customer customer = customerRepository.GetCustomer(id);
            return customer != null ? Ok(customer) : NotFound();
        }

        [HttpPost]
        public IActionResult Add([FromBody] Customer customer)
        {
            if (string.IsNullOrWhiteSpace(customer.CustomerId))
              return BadRequest();
            var customerRepository = new CustomerRepository();
            if (customerRepository.Add(customer))
            {
                customerRepository.Commit();
                return CreatedAtAction(nameof(Get), new { id = customer.CustomerId }, customer);
            }
            return Conflict();
        }

        [HttpPut]
        public IActionResult Update([FromBody] Customer customer)
        {
            if (string.IsNullOrWhiteSpace(customer.CustomerId))
              return BadRequest();
            var customerRepository = new CustomerRepository();
            var currentCustomer = customerRepository.GetCustomer(customer.CustomerId);
            if (currentCustomer == null)
                return NotFound();
            if (customerRepository.Update(customer))
            {
                customerRepository.Commit();
                return Ok(customer);
            }
            return NoContent();
        }

        [HttpDelete("{id}")]
        public IActionResult Delete([FromRoute]string id)
        {
            if (string.IsNullOrWhiteSpace(id))
              return BadRequest();
            var customerRepository = new CustomerRepository();
            var currentCustomer = customerRepository.GetCustomer(id);
            if (currentCustomer == null)
                return NotFound();
            if (customerRepository.Remove(currentCustomer))
            {
                customerRepository.Commit();
                return Ok();
            }
            return NoContent();
        }
    }
}

This controller will add actions to get all customers, get add, update or delete one customer. The project is ready to be run. You can run it with dotnet run and, when you open a new browser window and type this address:

https://localhost:5001/swagger

You will get something like this:

You can test the service with the Swagger page (as you can see, it was generated automatically wen you compiled and ran the app), but there is still another tool: HttpRepl. This tool was added with .NET 5 and you can install it with the command:

dotnet tool install -g Microsoft.dotnet-httprepl

Once you install it, you can run it with

httprepl https://localhost:5001

When you run it, you will get the REPL prompt:

If you type the uicommand, a new browser window will open with the Swagger page. You can type lsto list the available controllers and actions:

As you can see, it has a folder structure and you can test the service using the command line. For example, you can get all the customers with get customer or get one customer with get customer/blonp:

You can also “change directory” to the Customer “directory” with cd Customer. In this case, you can query the customer with get blonp:

If the body content is simple, you can use the -c parameter, like in:

post -c "{"customerid":"test"}"

This will add a new record with just the customer id:

If the content is more complicated, you must set the default editor, so you can edit the customer that will be inserted in the repository. You must do that with the command:

pref set editor.command.default "c:\windows\system32\notepad.exe"

This will open notepad when you type a command that needs a body, so you can type the body that will be sent when the command is executed. If you type post in the command line, notepad will open to edit the data. You can type this data:

{
  "customerId": "ABCD",
  "companyName": "Abcd Inc.",
  "contactName": "A.B.C.Dinc",
  "contactTitle": "Owner",
  "address": "1234 - Acme St - Suite A",
  "city": "Abcd",
  "region": "AC",
  "postalCode": "12345",
  "country": "USA",
  "phone": "(501) 555-1234"
}

When you save the file and close notepad, you will get something like this:

If you try to add the same record again, you will get an error 409, indicating that the record already exists in the database:

As you can see, the repository is doing the checks and sending the correct response to the REPL. You can use the same procedure to update or delete a customer. For the delete, you just have to pass the customer Id:

Now that we know all the commands we can do with the REPL, we can go one step further: using scripts. You can write a text file with the commands to use and run the script. Let’s say we want to exercise the entire API, by issuing all commands in a single run. We can create a script like this (we’ll name it TestApi.txt)

connect https://localhost:5001
ls
cd Customer
ls
get 
get alfki
post --content "{"customerId": "ABCD","companyName": "Abcd Inc.","contactName": "A.B.C.Dinc"}"
get abcd
put --content "{"customerId": "ABCD","companyName": "ACME Inc"}"
delete abcd
get abcd
cd ..
ls

And then open HttpRepl and run the command

run testapi.txt

We’ll get this output:

As you can see, with this tool, you get a very easy way to exercise your API. It’s not something fancy as a dedicated program like Postman, but it’s easy to use and does its job.

The full code for the project is at https://github.com/bsonnino/HttpRepl

 

In the last post I’ve shown how to use the new WebView2 control in a WPF app and said that it could be used in any Windows version and in any platform. As a matter of fact, I can create a 64 bit VCL native application with Delphi, that uses the Win32 API with the same control, offering the same functionality that the previous app did. To do that, you must have Delphi 10.4 Sydney installed in your machine. It offers the TEdgeBrowser component, that can be used to browse the web using the Chromium component.

In Delphi, create a new VCL application. Add the 64 bit platform to it and set it default. Add a TPanel, docked at the top. Then add a TEdgeBrowser and doc it to fill the window. In the panel, add a TLabel and change the Caption property to Search Text, add a TEdit and clear its Text property. Then, add two buttons and set their property Caption to Find and Copy. Add two more buttons at the right of the panel and set their sizes to 23×23, and their font Height to 10, Name to Segoe MDL2 Assets and their caption to the Back and Forward icons in the Character Map:

You should have something like this:

Now you can set the OnCreate event of the main form to

EdgeBrowser1.Navigate('https://docs.microsoft.com');

and run the application, but you won’t see nothing in the window. You can check the initialization of the WebViewer in the OnCreatedWebViewCompleted event and check the AResult parameter.

procedure TForm1.EdgeBrowser1CreateWebViewCompleted(Sender: TCustomEdgeBrowser;
  AResult: HRESULT);
begin
  if AResult <> 0 then
    ShowMessage('Error initializing WebView: $'+IntToHex(AResult));
end;

You will see that there is an error code (in my machine it’s $80004005). This is due to the fact that the WebView dll is missing. The dll is in the Redist folder and you have to add a PostBuild command to the project. Go to Project/Options and select Build options and, in the command “Value from All Configurations – Windows 64 bits”, add the following command:

copy /Y "C:\Program Files (x86)\Embarcadero\Studio\21.0\Redist\win64\WebView2Loader.dll" $(OUTPUTDIR)

This will copy the dll to the output directory and will initialize the WebView correctly(you will see that there is no message in the OnCreatedWebViewCompleted event and the page is loaded. Now, we can add the event handlers for the buttons. The Find button will set up the address and call the Navigate method (You must add the NetEncoding unit to the Uses clause):

procedure TForm1.FindClick(Sender: TObject);
begin
  if Edit1.Text <> '' then begin
    var address := 'https://docs.microsoft.com/en-us/search/?terms=' +
      TNetEncoding.URL.Encode(Edit1.Text);
    EdgeBrowser1.Navigate(address);
  end;
end;

The Back and Forward buttons events are very simple to implement:

procedure TForm1.BackClick(Sender: TObject);
begin
  if EdgeBrowser1.CanGoBack then
    EdgeBrowser1.GoBack;
end;

procedure TForm1.ForwardClick(Sender: TObject);
begin
  if EdgeBrowser1.CanGoForward then
    EdgeBrowser1.GoForward;
end;

To remove the parts of the page that we don’t want, we use the OnNavigationCompleted event to inject the JavaScript code:

procedure TForm1.EdgeBrowser1NavigationCompleted(Sender: TCustomEdgeBrowser;
  IsSuccess: Boolean; WebErrorStatus: TOleEnum);
begin
  if IsSuccess then
    EdgeBrowser1.ExecuteScript(
' var rss = document.querySelector(''[data-bi-name="search-rss-link"]'');'+ #13#10 +
' console.log(rss);'+ #13#10 +
' if (rss)'+ #13#10 +
'   rss.style.display = "none";'+ #13#10 +
' var form = document.getElementById("facet-search-form");'+ #13#10 +
' console.log(form);'+ #13#10 +
' if (form)'+ #13#10 +
'   form.style.display = "none";'+ #13#10 +
' var container = document.getElementById("left-container");'+ #13#10 +
' console.log(container);'+ #13#10 +
' if (container)'+ #13#10 +
'   container.style.display = "none";'+ #13#10 +
' var hiddenClasses = ["header-holder", "footerContainer"];'+ #13#10 +
' var divs = document.getElementsByTagName("div");'+ #13#10 +
' for( var i = 0; i < divs.length; i++) {'+ #13#10 +
'   if (hiddenClasses.some(r=> divs[i].classList.contains(r))){'+ #13#10 +
'     divs[i].style.display = "none";'+ #13#10 +
'   }'+ #13#10 +
' }');
end;

You can see here the same JavaScript code we used in the previous post, it also logs to the console the value of the variables. You can also open the Developer Tools of the browser component by pressing F12.

Now, there is only the Copy button to get the results and copy them to the clipboard. We will inject the code that gets all the results and send them to the application:

procedure TForm1.CopyClick(Sender: TObject);
begin
EdgeBrowser1.ExecuteScript(
'var results = [...document.querySelectorAll(''[data-bi-name="result"]'')]'+
'.map(a => {'+ #13#10 +
' let aElement = a.querySelector("a");'+ #13#10 +
' return {'+ #13#10 +
' title: aElement.innerText,'+ #13#10 +
' link: aElement.getAttribute("href")'+ #13#10 +
' };'+ #13#10 +
'});'+ #13#10 +
'console.log(results);'+ #13#10 +
'if (results.length >= 1){'+ #13#10 +
' window.chrome.webview.postMessage(results);'+ #13#10 +
'}'+ #13#10 +
'else {'+ #13#10 +
' alert("There are no results in the page");'+ #13#10 +
'}');
end;

This is the same code that is injected in the WPF app, with a slight difference: in the WPF program we’ve sent a message and added a listener in the JavaScript code. Here, we are running the code when the user clicks the button. This will send a message to the app, that will be processed in the OnMessageReceived event:

procedure TForm1.EdgeBrowser1WebMessageReceived(Sender: TCustomEdgeBrowser;
  Args: TWebMessageReceivedEventArgs);
var
  json : PWideChar;
begin
  var msg := Args as ICoreWebView2WebMessageReceivedEventArgs;
  msg.Get_webMessageAsJson(json);
  Clipboard.AsText := json;
  ShowMessage('Results sent to clipboard');
end

Now, when you run the program, you will have the same results as in the WPF program:

As you can see, with the new WebView2 component you have a lot of flexibility, you can use it in .NET or Win32 programs with almost no change. You can use this component to browse the Web and get data from the browsing, or you can use it to complement your current app: let’s say you have parts of your app that are already written for the web and you don’t want to rewrite them, but use them to interact with your app, you can add these parts and include them in your desktop app.

The full source code for this project is at https://github.com/bsonnino/WebViewDelphi

 

 

Times have changed and Microsoft is not the same: Edge, the new Microsoft browser has been remodeled and now it’s using the Chromium engine, an open source browser engine developed by Google.

With that, it has also changed the way you can develop browser apps – you will be able to use the same browser engine Microsoft uses in its browser to develop your browser apps. In order to do that, you will have to use the new WebView control. The new WebView control is not tied to a specific Windows version or development platform: You can use it in any Windows version, from 7 to 10 or use it in a Win32, .NET (core or full framework) or UWP app.

This article will show how to use it and interact with it in a WPF app. We will develop a program that will search in the Microsoft docs site, so you will be able to easily search there for information.

Introduction

Using the new WebView2 control in a .NET app is very simple, it’s just a matter of adding a NuGet package and you’re already setup. In Visual Studio, create a new WPF app. Right click the dependencies node in the Solution Explorer and select “Manage NuGet Packages” , the select the Microsoft.Web.WebView2 package. Then, in MainWindow.xaml, add the main UI:

<Grid>
    <Grid.RowDefinitions>
        <RowDefinition Height="40"/>
        <RowDefinition Height="*"/>
    </Grid.RowDefinitions>
    <StackPanel Orientation="Horizontal">
        <TextBlock Text="Search Text" Margin="5" VerticalAlignment="Center"/>
        <TextBox Text="" x:Name="SearchText" Margin="5" VerticalAlignment="Center"
                 Width="400" Height="30" VerticalContentAlignment="Center"/>
        <Button Content="Find" Width="65" Height="30" Margin="5" Click="ButtonBase_OnClick"/>
    </StackPanel>
    <wpf:WebView2 x:Name="WebView" Grid.Row="1" Source="" />
</Grid>

You will have to add the wpf  namespace to the xaml:

xmlns:wpf="clr-namespace:Microsoft.Web.WebView2.Wpf;assembly=Microsoft.Web.WebView2.Wpf"

As you can see, we are adding a textbox for the text to search and a button to activate the search. In the WebView control, we left the Source property blank, as we don’t want to go to a specific site. If we wanted to start with some page, we would have to fill this property. For example, if you fill the Source property with  https://docs.microsoft.com you would have something like this:

 

Navigating with the WebView

As you can see, it’s very easy to add a browser to your app, but we want to add more than a simple browsing experience. We want to make our app a custom way of browsing. To do that, we will use the following button click event handler:

private void ButtonBase_OnClick(object sender, RoutedEventArgs e)
{
    if (string.IsNullOrWhiteSpace(SearchText.Text))
        return;
    var searchAddress =
        $"https://docs.microsoft.com/en-us/search/?terms={HttpUtility.UrlEncode(SearchText.Text)}";
    WebView?.CoreWebView2?.Navigate(searchAddress);
}

We’ll take the text the user will want to search, create an URL for searching in the Microsoft docs website and then navigate to it. Once we do that, we can search the docs, just by typing the wanted text and clicking the button:

We can also add back and forward navigation by adding these two buttons:

<StackPanel Orientation="Horizontal" Grid.Row="0" HorizontalAlignment="Right" TextElement.FontFamily="Segoe MDL2 Assets">
    <Button Content="&amp;#xE0A6;" Width="30" Height="30" Margin="5" ToolTip="Back" Click="GoBackClick"/>
    <Button Content="&amp;#xE0AB;" Width="30" Height="30" Margin="5" ToolTip="Forward" Click="GoForwardClick"/>
</StackPanel>

The click event handler that will allow the browser to navigate to the previous or next page in history is:

private void GoBackClick(object sender, RoutedEventArgs e)
{
    if (WebView.CanGoBack)
        WebView.GoBack();
}

private void GoForwardClick(object sender, RoutedEventArgs e)
{
    if (WebView.CanGoForward)
        WebView.GoForward();
}

Once you have this code in place, you are able to use the two buttons to navigate in the browser history.

Customizing the page

One thing that bothers me in this app is the fact that we have several items in the page that don’t belong to our search: the top bar, the footer bar, the search box, and so on. Wouldn’t it be nice to clean these items from the page when we are showing it? Well, there is a way to do that, but we’ll have to resort to JavaScript to do that: when the page is loaded, we’ll inject a JavaScript script in the page that will remove the parts we don’t want. For that, we must create this code:

        async void InitializeAsync()
        {
            await WebView.EnsureCoreWebView2Async(null);
            WebView.CoreWebView2.DOMContentLoaded += CoreWebView2_DOMContentLoaded;
        }

        private async void CoreWebView2_DOMContentLoaded(object sender, CoreWebView2DOMContentLoadedEventArgs e)
        {

            await WebView.ExecuteScriptAsync(
                @"
window.onload = () => {
  var rss = document.querySelector('[data-bi-name=""search-rss-link""]');
  console.log(rss);
  if (rss)
    rss.style.display = 'none'; 
  var form = document.getElementById('facet-search-form');
  console.log(form);
  if (form)
    form.style.display = 'none';  
  var container = document.getElementById('left-container');
  console.log(container);
  if (container)
    container.style.display = 'none';  
  var hiddenClasses = ['header-holder', 'footerContainer'];
  var divs = document.getElementsByTagName('div');
  for( var i = 0; i < divs.length; i++) {
    if (hiddenClasses.some(r=> divs[i].classList.contains(r))){
      divs[i].style.display = 'none';
    }
  }
}");
        }

We have two parts in this code: initially, we ensure that the CoreWebView2 component is created and then we set a DomContentLoaded event handler, that will be called when the HTML content is loaded in the browser. In the event handler, we will inject the script and execute it with ExecuteScriptAsync. That is enough to remove the parts we don’t want from the page. The JavaScript code will retrieve the parts we want to hide and set their display style to none. This code is called from the constructor of the main window:

public MainWindow()
{
    InitializeComponent();
    InitializeAsync();
}

 

You can also see the console.log commands in the code. You can debug the JavaScript code when browsing by using the F12 key. The developer window will open in a separate window:

As you can see from the image, the top bar was removed, we now have a cleaner page. You can use the same technique to remove the context menu or add other functionality to the browser. Now we will get some extra info from the page.

Communicating between the app and the browser

When we are browsing the results page, we can get something from it. We can copy the results to the clipboard, so we can use them later. To do that we must use the communication between the app and the WebView. This is done by a messaging process. The app can send a message to the WebView, using something like

WebView?.CoreWebView2?.PostWebMessageAsString("message");

The WebView will receive the message and can process it by adding an event listener like in

window.chrome.webview.addEventListener('message', event => {
  if (event.data === 'message') {
    // process message
  }
});

When we want to send messages in the other direction, from the WebView to the app, we can send it from JavaScript, using

window.chrome.webview.postMessage(message);

It will be received by the app with an event handler like

WebView.CoreWebView2.WebMessageReceived += (s, args) =>
{
  data = args.WebMessageAsJson;
  // Process data
}

That way, we can have full communication between the app and the WebView and we can add the functionality we want: copy the results from the results page to the clipboard. The first step is to add the button to copy the results in MainWindow.xaml:

<StackPanel Orientation="Horizontal">
    <TextBlock Text="Search Text" Margin="5" VerticalAlignment="Center"/>
    <TextBox Text="" x:Name="SearchText" Margin="5" VerticalAlignment="Center"
             Width="400" Height="30" VerticalContentAlignment="Center"/>
    <Button Content="Find" Width="65" Height="30" Margin="5" Click="ButtonBase_OnClick"/>
    <Button Content="Copy" Width="65" Height="30" Margin="5" Click="CopyClick"/>
</StackPanel>

The click event handler will send a message to the WebView:

private void CopyClick(object sender, RoutedEventArgs e)
{
    WebView?.CoreWebView2?.PostWebMessageAsString("copy");
}

We must inject some code in the web page to receive and process the message, this is done by using the AddScriptToExecuteOnDocumentCreatedAsync method to inject the code when the page is loaded, in the InitializeAsync method:

        async void InitializeAsync()
        {

            await WebView.EnsureCoreWebView2Async(null);
            WebView.CoreWebView2.DOMContentLoaded += CoreWebView2_DOMContentLoaded;
            await WebView.CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync(@"
window.chrome.webview.addEventListener('message', event => {
  if (event.data === 'copy') {
    var results = [...document.querySelectorAll('[data-bi-name=""result""]')].map(a => {
            let aElement = a.querySelector('a');
            return {
                title: aElement.innerText,
                link: aElement.getAttribute('href')
            };
        });
    if (results.length >= 1){
      window.chrome.webview.postMessage(results);
    }
    else {
      alert('There are no results in the page');
    }
  }
});");
            WebView.CoreWebView2.WebMessageReceived += DataReceived;
        }

The JavaScript code will add the event listener, that will get all results in the page using the querySelectorAll  method, and then it will map it to an array of objects that have the title and link of the result, then will send this array to the app with postMessage. In the case that there are no results in the page, an alert message is shown. The code also sets the event handler for the WebMessageReceived event:

void DataReceived(object sender, CoreWebView2WebMessageReceivedEventArgs args)
{
    var data = args.WebMessageAsJson;
    Clipboard.SetText(data);
    MessageBox.Show("Results copied to the clipboard");
}

The handler will get the sent data with the args.WebMessageAsJson, then it will send it to the clipboard as text, where it can be copied to any program. Now, when you run the program, do a search and click the Copy  button, you will have something like this in the clipboard:

[
    {
        "title": "Getting started with WebView2 for WinForms apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/winforms"
    },
    {
        "title": "Microsoft Edge WebView2 Control - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/"
    },
    {
        "title": "Getting started with WebView2 for WinUI apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/winui"
    },
    {
        "title": "Getting started with WebView2 for WPF apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/wpf"
    },
    {
        "title": "Versioning of Microsoft Edge WebView2 - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/versioning"
    },
    {
        "title": "Distribution of Microsoft Edge WebView2 apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution"
    },
    {
        "title": "Getting started with WebView2 for Win32 apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/win32"
    },
    {
        "title": "Release Notes for Microsoft Edge WebView2 for Win32, WPF, and WinForms - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/releasenotes"
    },
    {
        "title": "Use JavaScript in WebView2 apps - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/js"
    },
    {
        "title": "Microsoft Edge WebView2 API Reference - Microsoft Edge Development",
        "link": "https://docs.microsoft.com/en-us/microsoft-edge/webview2/webview2-api-reference"
    }
]

Now we have an app that can browse to a page and interact with it.

Conclusion

There are many uses to this feature and the new Chromium WebView is a welcome addition to our toolbox, we can interact with the web page, retrieving and sending data to it.

The full source code for this article is at https://github.com/bsonnino/WebViewWpf

Introduction

You have an old, legacy app (with no tests), and its age is starting to show – it’s using an old version of the .NET Framework, it’s difficult to maintain and every new feature introduced brings a lot of bugs. Developers are afraid to change it, but the users ask for new features. You are at a crossroad: throw the code and rewrite everything or refactor the code. Does this sound familiar to you ?

I’m almost sure that you’re leaning to throw the code and rewrite everything. Start fresh, use new technologies and create the wonderful app you’ve always dreamed of. But this comes with a cost: the old app is still there, functional, and must be maintained while you are developing the new one. There are no resources to develop both apps in parallel and the new app will take a long time before its finished.

So, the only way to go is to refactor the old app. It’s not what you wanted, but it can still be fun – you will be able to use the new technologies, introduce good programming practices, and at the end, have the app you have dreamed. No, I’m not saying it will be an easy way, but it will be the most viable one.

This article will show how to port a .NET 4 WPF app and port it to .NET 5, introduce the MVVM pattern and add tests to it. After that, you will be able to change its UI, using WinUI3, like we did in this article.

The original app

The original app is a Customer CRUD, developed in .NET 4, with two projects – the UI project, CustomerApp, and a library CustomerLib, that access client’s data in an XML file (I did that just for the sake of simplicity, but this could be changed easily for another data source, like a database). You can get the app from here, and when you run it, you get something like this:

The first step will be converting it to .NET 5. Before that, we will see how portable is our app, using the .NET Portability analyzer. It’s a Visual studio extension that you can download from here. Once you download and install it, ou can run it in Visual Studio with Analyze/Portability Analyzer Settings:

You must select the platforms you want and click OK. Then, you must select Analyze/Analyze Assembly Portability, select the executables for the app and click OK. That will generate an Excel file with the report:

As you can see, our app can be ported safely to .NET 5. If there are any problems, you can check them in the Details tab. There, you will have a list of all APIs that you won’t be able to port, where you should find a workaround. Now, we’ll start converting the app.

Converting the app to .NET 5

To convert the app, we’ll start converting the lib to .NET Standard 2.0. To do that, right-click in the Lib project in the Solution Explorer and  select Unload Project, then edit the project file and change it to:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netstandard2.0</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup>
    <Content Include="Customers.xml">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
  </ItemGroup>
</Project>

The project file is very simple, just set the taget framework to netstandard2.0 and copy the item group relative to the xml file, so it’s included in the final project. Then, you must reload the project and remove the AssemblyInfo  file from the Properties folder, as it isn’t needed anymore (if you leave it, it will generate an error, as it’s included automatically by the new project file).

Then, right click the app project in the Solution Explorer and select Unload Project, to edit the project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net5.0-windows</TargetFramework>
    <UseWPF>true</UseWPF>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
  </PropertyGroup>
  <ItemGroup>
    <ProjectReference Include="..\CustomerLib\CustomerLib.csproj">
      <Name>CustomerLib</Name>
    </ProjectReference>
  </ItemGroup>
</Project>

We are setting the output type to WinExe, setting the target framework to net5.0-windows, telling that we will use the .net 5 features, plus the ones specific to Windows. If we don’t do that, we wouldn’t be able to use the WPF features, that are specific to Windows and set UseWPF to true. Then, we copy the lib’s ItemGroup to the project. When we reload the project, we must delete the Properties folder and we can build our app, converted to .NET 5. It should work exactly the way it did before. We are in the good path, porting the app to the newest .NET version, but we still have a long way to go. Now it’s time to add good practices to our app, using the MVVM Pattern.

The MVVM Pattern

The MVVM (Model-View-ViewModel) pattern was created on 2005 by John Gossman, a Microsoft Architect on Blend team, and it makes extensive use of the DataBinding feature existent in WPF and other XAML platforms (like UWP or Xamarin). It provides separation between data (Model) and its visualization (View), using a binding layer, the ViewModel.

The  ViewModel  is a class that implements the INotifyPropertyChanged interface:

public interface INotifyPropertyChanged 
{ 
  event PropertyChangedEventHandler PropertyChanged; 
}

It has just one event, PropertyChanged that is activated when there is a change in a property. The Data binding mechanism present in WPF (and in other XAML platforms) subscribes this event and updates the view with no program intervention. So, all we need to do is to create a class that implements INotifyPropertyChanged and call this event when there is a change in a property to WPF update the view.

The greatest advantage is that the ViewModel is a normal class and doesn’t have any dependency on the view layer. That way, we don’t need to initialize a window when we test the ViewModel. This image shows the basic structure of this pattern:

The model communicates with the ViewModel by its properties and methods. The ViewModel communicates with the View mainly using Data Binding, it receives Commands from the View and can send messages to it. When there are many ViewModels that must communicate , they usually send messages, to maintain a decoupled architecture. That way, the Model (usually a POCO class – Plain Old CSharp Object) doesn’t know about the ViewModel, the ViewModel isn’t coupled with the View or other ViewModels, and the View isn’t tied to a ViewModel directly (the only tie is the View’s DataContext property, that will bind the View and the ViewModel. The rest will be done by Data Binding).

We could implement all this infrastructure by ourselves, it’s not a difficult task, but it’s better to use a Framework for that. There are many MVVM frameworks out there, each one chooses a different approach to implement the infrastructure and select one of them is just a matter of preference. I’ve used MVVM Light toolkit for years, it’s very lightweight and easy to use, but it isn’t maintained anymore, so I decided to search another framework. Fortunately, the Windows Community Toolkit has provided a new framework, inspired on MVVM Light, the MVVM Community Toolkit.

Implementing the MVVM Pattern

In our project, the Model is already separated from the rest: it’s in the Lib project and we’ll leave that untouched, as we don’t have to change anything in the model. In order to use the MVVM Toolkit, we must add the Microsoft.Toolkit.Mvvm NuGet package.

Then, we must create the ViewModels to interact between the View and the Model. Create a new folder and name it ViewModel. In it, add a new class and name it MainViewModel.cs.  This class will inherit from ObservableObject, from the toolkit, as it already implements the interface. Then we will copy and adapt the code that is found in the code behind of MainWindow.xaml.cs:

 

public class MainViewModel : ObservableObject
{
    private readonly ICustomerRepository _customerRepository;
    private Customer _selectedCustomer;

    public MainViewModel()
    {
        _customerRepository =  new CustomerRepository();
        AddCommand = new RelayCommand(DoAdd);
        RemoveCommand = new RelayCommand(DoRemove, () => SelectedCustomer != null);
        SaveCommand = new RelayCommand(DoSave);
        SearchCommand = new RelayCommand<string>(DoSearch);
    }

    public IEnumerable<Customer> Customers => _customerRepository.Customers;

    public Customer SelectedCustomer
    {
        get => _selectedCustomer;
        set 
        { 
            SetProperty(ref _selectedCustomer, value); 
            RemoveCommand.NotifyCanExecuteChanged(); 
        } 
    }

    public IRelayCommand AddCommand { get; }
    public IRelayCommand RemoveCommand { get; }
    public IRelayCommand SaveCommand { get; }
    public IRelayCommand<string> SearchCommand { get; }
    private void DoAdd()
    {
        var customer = new Customer();
        _customerRepository.Add(customer);
        SelectedCustomer = customer;
        OnPropertyChanged("Customers");
    }

    private void DoRemove()
    {
        if (SelectedCustomer != null)
        {
            _customerRepository.Remove(SelectedCustomer);
            SelectedCustomer = null;
            OnPropertyChanged("Customers");
        }
    }

    private void DoSave()
    {
        _customerRepository.Commit();
    }

    private void DoSearch(string textToSearch)
    {
        var coll = CollectionViewSource.GetDefaultView(Customers);
        if (!string.IsNullOrWhiteSpace(textToSearch))
            coll.Filter = c => ((Customer)c).Country.ToLower().Contains(textToSearch.ToLower());
        else
            coll.Filter = null;
    }
}

We have two properties, Customers and SelectedCustomerCustomers  will contain the list of customers shown in the DataGrid. SelectedCustomer will be the selected customer in the DataGrid, that will be shown in the detail pane. There are four commands, and we will use the IRelayCommand interface, declared in the toolkit. Each command will be initialized with the method that will be executed when the command is invoked. The RemoveCommand uses an overload for the constructor, that uses a predicate as the second parameter. This predicate will only enable the button when there is a customer selected in the DataGrid. As this command is dependent on the selected customer, when we change this property, we call the NotifyCanExecuteChanged method to notify all the elements that are bound to this command.

Now we can remove all the code from MainWindow.xaml.cs and leave only this:

public MainWindow()
{
    InitializeComponent();
    DataContext = new MainViewModel();
}

We can run the program and see that it runs the same way it did before, but we made a large refactoring to the code and now we can start implementing unit tests in the code.

Implementing tests

Now that we’ve separated the code from the view, we can test the ViewModel without the need to initialize a Window. That is really great, because we can have testable code and be assured that we are not breaking anything when we are implementing new features. For that, add a new test project and name it CustomerApp.Tests. In the Visual Studio version I’m using, there is no template for the .net 5.0 test project available, so I added a .Net Core test project, then I edited the project file and changed the TargetFramework to net5.0-windows. Then, you can add a reference to the CustomerApp project and rename UnitTest1 to MainViewModelTests.

Taking a look at the Main ViewModel, we see that there is a coupling between it and the Customer Repository. In this case, there is no much trouble, because we are reading the customers from a XML file located in the output directory, but if we decide to replace it with some kind of database, it can be tricky to test the ViewModel, because we would have to do a lot of setup to test it.

We’ll remove the dependency using Dependency Injection. Instead of using another framework for the the dependency injection, we’ll use the integrated one, based on Microsoft.Extensions.DependencyInjection. You should add this NuGet package in the App project to use the dependency injection. Then, in App.xaml.cs, we’ll add code to initialize the location of the services:

public partial class App
{
    public App()
    {
        Services = ConfigureServices();
    }

    public new static App Current => (App) Application.Current;

    public IServiceProvider Services { get; }

    private static IServiceProvider ConfigureServices()
    {
        var services = new ServiceCollection();

        services.AddSingleton<ICustomerRepository, CustomerRepository>();
        services.AddSingleton<MainViewModel>();

        return services.BuildServiceProvider();
    }

    public MainViewModel MainVM => Services.GetService<MainViewModel>();
}

We declare a static property Current to ease using the App  object and declare a IServiceProvider, to provide our services. They are configured in the ConfigureServices method, that creates a ServiceCollection and add the CustomerRepository and the main ViewModel to the collection. ConfigureServices  is called in the constructor of the application. Finally we declare the property MainVM, which will get the ViewModel from the Service Collection.

Now, we can change MainWindow.xaml.cs to use the property instead of instantiate directly the ViewModel:

public MainWindow()
{
    InitializeComponent();
    DataContext = App.Current.MainVM;
}

The last change is to remove the coupling between the ViewModel and the repository using Dependency Injection, in MainViewModel.cs:

public MainViewModel(ICustomerRepository customerRepository)
{
    _customerRepository = customerRepository ??  
                          throw new ArgumentNullException("customerRepository");
    _customerRepository = customerRepository;
    AddCommand = new RelayCommand(DoAdd);
    RemoveCommand = new RelayCommand(DoRemove, () => SelectedCustomer != null);
    SaveCommand = new RelayCommand(DoSave);
    SearchCommand = new RelayCommand<string>(DoSearch);
}

With that, we’ve gone one step further and removed the coupling between the ViewModel and the repository, so we can start our tests.

For the tests, we will use two libraries, FluentAssertions, for better assertions and FakeItEasy, to generate fakes. You should install both NuGet packages to your test project. Now, we can start creating our tests:

public class MainViewModelTests
{
    [TestMethod]
    public void Constructor_NullRepository_ShouldThrow()
    {
        Action act = () => new MainViewModel(null);

        act.Should().Throw<ArgumentNullException>()
            .Where(e => e.Message.Contains("customerRepository"));
    }

    [TestMethod]
    public void Constructor_Customers_ShouldHaveValue()
    {
        var repository = A.Fake<ICustomerRepository>();
        var customers = new List<Customer>();
        A.CallTo(() => repository.Customers).Returns(customers);
        var vm =  new MainViewModel(repository);

        vm.Customers.Should().BeEquivalentTo(customers);
    }

    [TestMethod]
    public void Constructor_SelectedCustomer_ShouldBeNull()
    {
        var repository = A.Fake<ICustomerRepository>();
        var vm = new MainViewModel(repository);

        vm.SelectedCustomer.Should().BeNull();
    }
}

Here we created three tests for the constructor, testing the values of the properties after the constructor. We can continue, testing the commands in the ViewModel:

[TestMethod]
public void AddCommand_ShouldAddInRepository()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);

    vm.AddCommand.Execute(null);
    A.CallTo(() => repository.Add(A<Customer>._)).MustHaveHappened();
}

[TestMethod]
public void AddCommand_SelectedCustomer_ShouldNotBeNull()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.AddCommand.Execute(null);
    vm.SelectedCustomer.Should().NotBeNull();
}

[TestMethod]
public void AddCommand_ShouldNotifyCustomers()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    var wasNotified = false;
    vm.PropertyChanged += (s, e) =>
    {
        if (e.PropertyName == "Customers")
            wasNotified = true;
    };
    vm.AddCommand.Execute(null);
    wasNotified.Should().BeTrue();
}

[TestMethod]
public void RemoveCommand_SelectedCustomerNull_ShouldNotRemoveInRepository()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.RemoveCommand.Execute(null);
    A.CallTo(() => repository.Remove(A<Customer>._)).MustNotHaveHappened();
}

[TestMethod]
public void RemoveCommand_SelectedCustomerNotNull_ShouldRemoveInRepository()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SelectedCustomer = new Customer();
    vm.RemoveCommand.Execute(null);
    A.CallTo(() => repository.Remove(A<Customer>._)).MustHaveHappened();
}

[TestMethod]
public void RemoveCommand_SelectedCustomer_ShouldBeNull()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SelectedCustomer = new Customer();
    vm.RemoveCommand.Execute(null);
    vm.SelectedCustomer.Should().BeNull();
}

[TestMethod]
public void RemoveCommand_ShouldNotifyCustomers()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SelectedCustomer = new Customer(); 
    var wasNotified = false;
    vm.PropertyChanged += (s, e) =>
    {
        if (e.PropertyName == "Customers")
            wasNotified = true;
    };
    vm.RemoveCommand.Execute(null);
    wasNotified.Should().BeTrue();
}

[TestMethod]
public void SaveCommand_ShouldCommitInRepository()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SaveCommand.Execute(null);
    A.CallTo(() => repository.Commit()).MustHaveHappened();
}

[TestMethod]
public void SearchCommand_WithText_ShouldSetFilter()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SearchCommand.Execute("text");
    var coll = CollectionViewSource.GetDefaultView(vm.Customers);
    coll.Filter.Should().NotBeNull();
}

[TestMethod]
public void SearchCommand_WithoutText_ShouldSetFilter()
{
    var repository = A.Fake<ICustomerRepository>();
    var vm = new MainViewModel(repository);
    vm.SearchCommand.Execute("");
    var coll = CollectionViewSource.GetDefaultView(vm.Customers);
    coll.Filter.Should().BeNull();
}

Now we have all the tests for the ViewModel and have our project ready for the future. We went step by step and finished with a .NET 5.0 project that uses the MVVM pattern and have unit tests. This project is ready to be updated to WinUI3, or even to be ported to UWP or Xamarin. The separation between the code and the UI makes it easy to port it to other platforms, the ViewModel became testable and you can test all logic in it, without bothering with the UI. Nice, no ?

The full source code for the project is at https://github.com/bsonnino/MvvmApp

 

Recently I bought a Fitbit Sense watch, it has tons of sensors to measure your health and fitness, I really enjoy its features.

One extra thing that I like is that Fitbit provides an SDK to create new apps and clock faces for your watch, you can use it for the Ionic, Versa and Sense devices and customize your watch, adding the features you want. This SDK is Javascript based, so you must use Javascript to create your apps. But you are not completely tied to Javascript, you can use Typescript and have all the features the language provides to create your apps. In this article, we’ll see how to use Typescript to create a simple clock face.

Setting up your workspace

The Fitbit SDK is mostly web based. You must have a user account, have the Fitbit mobile app installed in your phone and have a device associated with your account (this is needed in order to upload the app to the store). Once you have this set up, you can open Fitbit Studio and start developing your app. To show the result of your app, you must download and install the Fitbit Simulator from here (this is the Windows version, but you can download the Mac version from here).

When you use Fitbit Studio, you must use JavaScript to develop. This is not what we want. We will use what Fitbit calls the CLI (Command Line Interface), to create an app in your local machine. For that, you must have Visual Studio Code installed (you can use another editor that supports JavaScript/TypeScript, if you want) and Node.js.

With this setup, you can start developing your app. In the command line, type:

npx create-fitbit-app fitbit-first

You will have to select if the application is an app or a clockface, its name,  if it has a companion component and the device it will run:

Once it runs, you can change to the folder created by the creator and start code with

code .

You will have something like this:

This is the basic structure of the Fitbit app:

  • In the app folder, you have the index.js file, the entry point for the app
  • In the resources folder, you have the index.view, with an svg file with the main display. You have also the styles.css with the css styles for the view
  • If you take a look at the main folder, you will see a tsconfig.json file. This is a good indication that TypeScript is already installed for the app.

So, as you can see, you can start to use TypeScript immediately, but you won’t have intellisense. For that, you will have to install the types for the SDK. you can do that by opening a terminal in VS Code and running this command:

npx fitbit-sdk-types

With that, you are ready to go. You can rename the main file to index.ts and start running the project. You must start the simulator and run the application. In the terminal, open the Fitbit shell with

npx fitbit

You will see the Fitbit prompt, fitbit$, where you can type commands. You can use the build command to build the app and the install command to install the app on the simulator. Alternatively, you can use the build-install or bi to build and install the app on the simulator. When you do that, you will see Hello World!  in the console and a white face in the simulator.

Let’s start with our clock face. We will create a simple digital clock, that will display the time and date. The first step is to create the UI, in the index.view file. We will add two text elements for the time and the date:

<svg>
  <text id="time" fill="fb-blue" x="50%" y="50%" 
    font-size="80" font-family="System-Regular" 
    text-anchor="middle" >XX:XXx</text>
  <text id="date" fill="fb-light-gray" x="50%" y="80%" 
    font-size="50" font-family="System-Regular" 
    text-anchor="middle" >xxx 99</text>
</svg>
Running this code you will see a black background with two pieces of text:

Now we can start adding the code. Add a new class and name it Clock.ts:

import clock from "clock";

export class Clock {
    public clockCallback: (text: string) => void;

    constructor() {
        clock.granularity = "minutes";
        // Update the clock every tick event
        clock.addEventListener("tick", this.updateClock);
    }

    private updateClock = () => {
        if (!this.clockCallback)
            return;
        const date = new Date();
        const hours = date.getHours();
        const minutes = date.getMinutes();
        const zeroPad = ((n: number) => (n < 10) ? "0" + n : n);
        const time = `${zeroPad(hours)}:${zeroPad(minutes)}`;
        this.clockCallback(time);
    }
}

export default Clock;

In this class, we will set the granularity of the clock to minutes, and add a listener to update the clock on every tick. It will create a string with the current time and pass it to the callback. The callback is set in the main file, after creating a new instance of the clock:

import Clock from "./Clock";
import document from "document";

const timeElement = document.getElementById("time") as TextElement;
const clock = new Clock();
clock.clockCallback = (t) => timeElement.text = t;

With this code, when you run the program, you will see the current time in the simulator:

This is only a 24 hours clock and doesn’t respect the user preferences. We must use the user preferences to set a 12 hour clock:

import clock from "clock";
import {preferences } from "user-settings";

export class Clock {
    public clockCallback: (text: string) => void;

    constructor() {
        clock.granularity = "minutes";

        // Update the clock every tick event
        clock.addEventListener("tick", this.updateClock);
    }

    private updateClock = () => {
        if (!this.clockCallback)
            return;
        const date = new Date();
        let hours = date.getHours();
        const minutes = date.getMinutes();
        let amPm = "";
        const zeroPad = ((n: number) => (n < 10) ? "0" + n : n);
        const twelveHour = preferences.clockDisplay === "12h";
        if (twelveHour)  { 
            amPm = hours > 12 ? "."  : "" ;
            hours = hours % 12 || 12;
          } 
        const time = `${zeroPad(hours)}:${zeroPad(minutes)}${amPm}`;
        this.clockCallback(time);
    }
}

export default Clock;

We will user the preferences from the user settings to retrieve the time display settings, if the user selects a 12 hours display, we will add a “.” after the time to indicate PM. The rest of the code remains the same and, when you run the code, you can see the effect of the user settings in the simulator by going to the User tab and selecting 12 or 24 hours:

Now, we just need to add the date to the display. For that, you must create a new class and name it DateDisplay.ts:

import clock from "clock";

export class DateDisplay {
    private months = ["JAN", "FEB", "MAR", "APR", "MAY", "JUN", "JUL", "AUG", "SEP", "OCT", "NOV", "DEC"];
   
    public dateCallback: (text: string) => void;

    constructor() {
        clock.addEventListener("tick", this.updateDate);
    }

    private updateDate = () => {
        if (!this.dateCallback)
            return;
        const currDate = new Date();
        const day = currDate.getDate();
        const month = currDate.getMonth();
        const date = `${this.months[month]} ${day}`;
        this.dateCallback(date);
    }
}

export default DateDisplay;

The code is very similar to the clock. It adds a listener to the tick event and generates a string composed of the month and the day of the current date, then it passes it to the callback, that is subscribed in index.ts:

import Clock from "./Clock";
import DateDisplay from "./DateDisplay"
import document from "document";

const timeElement = document.getElementById("time") as TextElement;
const dateElement = document.getElementById("date") as TextElement;

const clock = new Clock();
clock.clockCallback = (t) => timeElement.text = t;
const dateDisplay = new DateDisplay();
dateDisplay.dateCallback = (d) => dateElement.text = d;

When you run the app, you have the time and date in the display:

Our clock face is ready. It’s a very simple one and doesn’t use the sensor data available in the smartwatch, but it’s enough to get the feeling on what does it take to develop a clock face for the Fitbit using Typescript and VS Code. You can use the environment you are used to and have all the benefits of using Typescript, and still be able to use all the features available in the Fitbit SDK.

If you want to know more, you can go to https://dev.fitbit.com/getting-started/. There are a lot of resources, tutorials and samples to learn how to create your customized clock face.

The full source code for this article is at https://github.com/bsonnino/FitbitApp