.NET From India

Building for Alexa - 4 - Progressive Response

This post looks at a specific functionality that you can add to your skills called Progressive Response.

These allow you to show the user that the skill is working on something that may take some time. For example, if you are looking up an order delivery status, but you need to check with multiple logistic providers to get the final status. This whole operation may take maybe 30 seconds for you and then you need to package this information into a useful voice response. So it may make sense when you are half way thru your third-party checks to let the user that you are working thru the request and it may take you some more time to get the information.

Think of this are the voice equivalent of the hourglass in a windows application or the animated circle you find in web applications (Like the ones you see here).

The way you send a progressive dialog is to call the API endpoint /directives with a VoicePlayer.Speak command.

Currently, you do not have an out of the box way to do this via the AlexaSkillKit.NET library, but as this is a normal REST endpoint, you can directly call it like any other REST service.

Below is a function I wrote that you can use to send progressive dialogs as part of your Alexa Skill (This uses the RestSharp Library for the API calling). The apiToken,requestId and apiURL are part of the request you get from Amazon. You need to use the token you receive or the REST call will fail.

internal static void SendProgressiveResponse(string apiToken, string requestId,
string apiBaseUrl, string speechText)
        {
            if (speechText.Length > 600)
            {
                throw new ArgumentException("Progressive Text cannot be more than 600 characters", "speechText");
            }
            var restClient = new RestClient(apiBaseUrl);
            RestRequest restRequest = new RestRequest();;
            restRequest.Resource = "v1/directives";
            restRequest.Method = RestSharp.Method.POST;
            restRequest.AddHeader("Authorization", "Bearer " + apiToken);
            restRequest.AddHeader("Content-Type", "application/json");
            var body = new
            {
                header = new
                {
                    requestId = requestId
                },
                directive = new
                {
                    type = "VoicePlayer.Speak",
                    speech = speechText
                }
            };
            restRequest.AddJsonBody(body);

            var response = restClient.Execute(restRequest);
            if (response.StatusCode == System.Net.HttpStatusCode.NoContent)
            {
                //Debug.log("Progressive was successful");
            }
            else
            {
                //Debug.log($"Progressive failed with code - {response.StatusCode}, {response.StatusDescription}. Content - {response.Content}");
            }
        }

 You can get more information on Progressive Responses here..

Building for Alexa – 3 – Structuring your Code

In this post I want to discuss the way I like to structure my code when developing the Skill handling backend. As mentioned in previous posts, the handler is based on the ASP.NET WebAPI project template, which is based on MVC. So out of the box the structure is like below:

I like to structure the Intent Handling code in a way that I think makes it more maintainable and easier to debug. Again this is just my personal approach and is not in any way dictated by MVC or other patterns.

The AlexaSKillKit.net GitHub page has a get started section that would get you to the above structure and a working Skill handling backend.

So if you follow this structure you will have a basic Speechlet class that handles all of your intents and returns a SpeechletResponse to the Controller.  The Controller is very light weight and just contains code to receive a POST and returning, what your Speechlet implementation returns, after converting it to a API response(HTTPResponseMessage).

Since all the intents land up in the single speechlet implementation class, I prefer to create separate handlers for each intent. I usually declare a IHandler interface that contains something a single method like below:

public virtual SpeechletResponse handleIntent(IntentRequest request, Session session, Context context, Dictionary<string, string> options=null)

So for each category of intents (not always each intent, but in a lot of cases it ends up being that), there is a handler.  The Speechlet implementation then hands off to the handler based on the intent received.

So our structure changes to something like this:

This has some advantages. Since handler logic is separated out, it is easier to maintain and debug  this code, instead of everything being in a big class. Also when later you want to do something like IOC, this structure will help make that possible.

I also prefer to have a set of service gateway classes that links up our handlers to the actual business services your may end up calling.  These could be SOAP or REST  based on your business, but most skills end up calling some kind of business services. So having a gateway pattern implementation would make future expansion and management of  these interfaces easier.

CallerMemberName and Interfaces

[CallerMemberName] is a new attribute that is available in C# 5.0. It is a very useful functionality especially when doing common app framework components like logging, auditing etc.

One of the things that you usually try to do in your common logging infrastructure is to log the function that generated that log.

So before C# 5.0 it was most common to interrogate the stack to find the caller, which can be an expensive operation. Or the caller information is passed from the source function, which hard codes its name in the call.

Something like

_logger.Debug("Message", "myFunctionName");

This becomes hard to manage and so in most cases the logger actually interrogates the stack to get this information and as mentioned before that comes with a performance penalty.

Using this new attribute, you can write something like the following for your log function:

public void Debug(string Message, [CallerMemberName] string callerFunction = "NoCaller")

The second parameter here is not passed by the caller, but is rather filled up by the infrastructure. The parameter needs to be an optional parameter for you to be able to use this attribute. And this does not have a performance penalty as the compiler will fill in this parameter at compile time and so no run-time overheads.

This works fine, except when you use Interfaces. Again interfaces are generally a pattern that is often used in framework components like logging infrastructure. So you will have a ILogger interface that will have a definition like about which is implemented by your actual Logger class and the caller use the Interface to talk to your implementation.

If you do something like this, the gotcha is that the interface also needs to have the attribute defined, or the value will not be auto filled. This is something that is easy to miss and is not well documented from whatever I could find. So your interface definition for this function will have the attribute defined like below:

void Debug(string Message, [CallerMemberName] string callerFunction = "")

for the caller value to flow through to your implementation.

When you are designing your interfaces this is so easy to miss and can take a long time to figure out why.

Building for Alexa – 2 - Creating a Skill

This is the second post on the series about developing Alexa Skills using .NET.

 

Coding A Skill

Developing a skill is made up of two major parts. You setup what is know as the intent, utterances configuration. This is done in the amazon skill website console. The new iterations of this console is vastly improved and is easier to use from a newbie perspective. (It used for be a text box into which you typed in a huge JSON).You need to have a registered Amazon developer account to develop a skill and it is free.

The other part is the building of the backend that would actually process the request from  Amazon. So Amazon parses the actual voice input and then maps into the intents and structures we had configured and then calls our backend service with that data.

All data from and to the Amazon Alexa Skill system is via JSON and our backend is just a REST endpoint that is called by Amazon. So if you are a .NET  developer at this point you are thinking Web API, right? Well you are correct.

So we need to start with a Web API project. It does not matter what framework and Core vs. Non-Core, the Alexa service just needs a REST Endpoint that supports https (Important, if you are planning to host your endpoint). There are some other rules that are documented in the Amazon site for custom hosted endpoints, but if you starting off with a skill, the this is the only major rule.

Handling Alexa Requests

I am going to assume you read thru the developer documentations and setup the intent mappings and utterances for your Alexa skill. This is quite well documented in the Amazon site.

Though the interface between the Alexa service and your backend is purely REST/JSON based, it is helpful if you use a library that helps parse those requests and also create response requests. This will greatly simplify the amount of plumbing code you will have to write for everything to work.

The most common library I found is the AlexaSkillsKit.NET kit. This supports most of the commonly used Skill kit requests and would be quite easy to start with.

It is not very well documented, but you should be able to get started with the documentation available and then you will have to jump into the code or use GitHub forums to get around using more complex features.

You can get this from GitHub or as a NuGet package

Once you have a basic WebAPI project with this library integrated correctly, you should be able to host a backend that can handle requests coming in from the Alexa service.

If you are just checking it out, I would recommend you use Azure App service hosting to get a sample working. It allows you to easily have a HTTPS endpoint available which will work with the Alexa service.

The future articles in this series will be focused of specific scenarios and how you can handle those when you are building your backend.

Building for Alexa – 1 - Anatomy of a Skill

This is the first of a series of posts that talk about building an Alexa Skill using .NET.

 

I have been playing around with building Alexa skills for the last couple of years, but never wrote about it. The idea for this set of blog posts came as I was building one recently and found that is was very difficult to find information around this. The idea here is to have enough information to get started, rather than in-depth information of different aspects. I may also post about some problems I had, as .NET is not a supported platform from Amazon Skill Kit perspective.

Anatomy of a Skill

An Alexa Skill flow has four major entities involved in each flow:

1. The user who speaks a command to the echo or Alexa enabled device (I will use echo for simplicity in my posts, but it mean any Alexa enabled device should work).
2. The echo device, which receives the initial voice input and also plays back the final voice output.
3. The Amazon Alexa service, which receives the data from the devices, decodes it and then forwards it for processing to the backend. It also converts what it receives from the backend and sends it over to the actual echo device.
4. The actual back end services which contains the business logic for the skill itself. This can be hosted either in Amazon or outside. Amazon provides hosting in its cloud based Lambda service if you want to build these backends in Node.

So, basically when you are talking about building a skill one you are actually building of the back end service and configuring the Amazon Alexa service. A simple diagrammatic representation is a below.

 

The Amazon Alexa Service maps the incoming request to what are called Intents (that the developer defines) and then creates a JSON with data that you need and then sends it over to the back end services. The backend service does whatever process it is supposed to do and then returns the data back as a JSON. The JSON formats that are part of this interchange is defined by Amazon.

In this series, I am going to talk about building this Back End service using .NET, c# and  then hosting it as a Azure API service. So this will be built using the default Web API MVC project type. You can deploy this to any publicly accessible endpoint and Azure just happens to be something that was easily accessible for me.

In the next post, I will talk about the different ways you can create the backend service and what you will need to do this in .NET

Open SSH on Windows

I had a scenario a week or two ago where I wanted to connect from a Mac to a Windows machine and run some PS scripts. (Well the discussion on why not use PowerShell Core is for a different post).

So the easiest way is to do SSH. The windows machine I needed to connect to was a Win 7 machine and so did not ship with SSH built in. So I just downloaded Open SSH that the powershell team had ported to Windows and deployed it. And then connected from the Mac, no problem, everything worked great.

Now I wanted to integrate this into a script and did not want to enter a user name/password combo and SSH has a way to support this using public/private key pairs. If you search the internet, there are hundreds of articles on how to generate a key pair and copy the public key over to the server to allow the client to connect. The problem is I could not find what to do on the Windows Server side of SSH as the directory structure it uses differs slightly from what the UNIX version.

So some debugging and log analysis later found out the Windows version uses the <User Dir>\.ssh (i.e. C:\Users\xxx\) directory to find the pub key file (file should be named authorized_keys). The only gotcha is that the user directory is the user who logs in via the SSH. I had setup a local user that the SSH script will use to log in, and I was initially trying to put the file into my user dir, when it needs to be in that trying to log in users user dir. The authorized keys file is just a text file containing all public keys. So if you have more than one public key, just append to your authorized keys file.

Once the file was in the right place, you still need to run the powershell scripts the team provides to fix the Host and User File permissions(These are available in the directory where you installed OpenSSH). These are both very easy to run and should get you started quickly.    

Pitfalls with GetFileFromPathAsync

If you are working with WinRT to develop store apps (for Win8 or Win8.1) currently you most probably have been using the new Storage APIs.

One of the APIs that allows you to get access to a file given a path is GetFileFromPathAsync.

This function accepts a full path and returns a StorageFile that you can use to work with that file. If you look at the MSDN article on this you will note that there is a comment in the parameter section that says

"If your path uses slashes, make sure you use backslashes (\). Forward slashes (/) are not accepted by this method."

If you do have the wrong slashes you would get a "Path is invalid" exception. This fine if you are generating the path and you can put in the right slashes in when building the path.

But today, I came up against a scenario I did not expect. Since I don't like working with string paths, I usually work with System.Uri. In this case I was passing around an Uri and passed uriVariable.AbsolutePath and was surprised when I got the Path Invalid exception. Looks like the Uri object stores paths with Forward slashes ("/"), which the GetFileFromPathAsync does not like. Well two part of the framework that work with two different representation of paths..

Well the solution is simple, but I did spend a good 30 minutes trying to sort this one out as I initially thought the path was really wrong and this error is not very well documented elsewhere and so google searches did not turn up much.

GetFileFromPathAsync(uriVariable.AbsolutePath.Replace("/", "\"))

Wearable Devices can change insurance underwriting

I was reading an interesting article How Wearable Devices Are Disrupting the Insurance Industry. It does talk about a number of scenarios on how data collected by wearable can have an impact on how the insurance industry works and also the privacy concerns around this.

Insurance industry has been toying around with devices for some time now. I remember some auto insurance companies in US, providing devices that are placed in cars that can give you a discount if you are a safe driver. But this has not been widespread and still is in very small segment /experimental phase.

But now consumers already have devices that track most of their personal life. One thing that the new wearable like FitBit, Apple Watch, Microsoft Band etc. provide is access to health related data like heart beat and exercise routines. This can be a gold mine for health insurance companies as it gives them personalized data for each insured.

Again this opens up a can of privacy issues. Do you want your insurance company to have access to your heat beat, number of hours you sleep or exercise records?

I cannot predict the future, but if consumers agree to share data (lower premiums may push people to override privacy concerns??), this can turn the way health policies are underwritten upside down.

Obviously this requires adoption of a lot of new technologies like IOT, Big Data, machine learning etc, so that the insurance companies can make sense of the huge amounts of data that is going to be coming in and then take appropriate underwriting decisions.

Hololens

Build has some amazing demos of the new Microsoft Hololens.. Take a look.

 

 

 

 

Windows Hello

Windows hello is a new authentication system coming in Win 10. It helps you log in to windows using facial recognition or fingerprint. Fingerprint based log ins are already part of existing systems, but face recognition is new. Before we go overboard, it does not use the current webcams, but needs new camera hardware which has infrared scanning and 3D imaging support (atleast from what I can get from the articles I have been able to find.

There is also a new technology called "Microsoft Passport", which does not seem to be related to the old passport, but seems to replace password with a hardware based token exchange for authentication to apps or websites.

The following video shows an introduction to some features of Windows Hello.

 

 

Also Intel has a video where their new Realsense camera integrated into Hello