AI Builder + Lobe

During the recent Arctic Cloud Developer Challenge Hackathon I was playing around with AI Builder for the first time. The scenario we were going to build on there was the detection of Good guy / Bad guy.

The idea was that the citizens would be able to take pictures of suspicious behavior. Once the picture was taken, the classification of the picture would let them know if it was safe or not.

For this exercise I would use my trusted BadBoy/GoodBoy nephew toys:)

Lobe

To start it off I downloaded a free tool called Lobe. www.lobe.ai . Microsoft acquired this tool recently, and it’s a great tool to learn more about object recognition in pictures. The really cool thing about the software is that calculations for the AI model are done on your local computer, so you don’t need to setup any Azure services to try out a model for recognition.

Another great feature is that it integrates seamlessly with Power Platform. Once you train you model with the correct data, you just export it to Power Platform!👏

The first thing you need to do is have enough pictures of the object. Just do at least 15 pictures from different angles to make it understand the object you want to detect.

Tagg all of the images with the correct tags.

Next step is to train the model. This will be done using your local PC resources. When the training is complete you can export to Power Platform.

It’s actually that simple!!! This was really surprising to me:)

Power App

Next up was the Power App the citizens were going to use for the pictures. The idea of course that everyone had this app on their phones and licensing wasn’t an issue 😂

I just added a camera control, and used a button to call a Power Automate Cloud Flow, but this is also where the tricky parts began.

An image is not just an image!!!!! 😤🤦‍♂️🤦‍♀️

How on earth anyone is supposed to understand that you need to convert a picture you take, so that you can send it to Flow, only there to convert it to something else that then would make sense???!??!

Image64 and Power Automate – What a shit show

After asking a few friends, and googling tons of different tips/trics I was able to make this line here work. I am sure there are other ways of doing this, but it’s not blatantly obvious to me.

Set(WebcamPhoto, Camera1.Photo);

Set(PictureFormat,Substitute(Substitute(JSON(WebcamPhoto,JSONFormat.IncludeBinaryData),"data:image/png;base64,",""),"""",""));

'PowerAppV2->Compose'.Run(PictureFormat);

The receiving Power Automate Cloud Flow looked like this:

I tried receiving the image as a type image, but I couldn’t figure it out. Therefore I converted it to a Base64 I believe when sending to Flow. In the Flow I again converted this to a binary representation of the image before sending it to the prediction.

The prediction on the other hand worked out really nice!! I found my model that I had imported from Lobe, and used the ouputs from the Compose 3 action (base64 to binary). I have no idea what the differences are, but I just acknowledge that this is how it has to be done.. I am sure there are other ways, but that someone will have to teach me in the future.

All in all it actually worked really well. As you can see here I added all types of outputs to learn from the data, but it was exactly as expected when taking a picture of Winnie the Poo 😊 The bear was categorized as good, and my model was working.

Why Lobe?

One can wonder why I chose to use Lobe for this, when the AI Builder has the training functionality included within the Power Platform. For my simple case it wouldn’t really matter what I chose to use, I just wanted to test out the newest tech.

When working with larger scale (numbers) of images, Lobe seems to be a lot easier for the importing/tagging/training of the model. Everything runs locally, so the speed of training and testing is a lot faster also. It’s also simple to retrain the model an upload again. This being a hackathon it was important to try new things out 😊

More about AI builder

I talked to Joe Fernandez from the AI builder team, and he pointed me to some resources that are nice to checkout regarding this topic.

https://myignite.microsoft.com/sessions/a5da5404-6a25-4428-b4d0-9aba67076a08 <- forward to 11:50 for info regarding the AI Builder

https://youtube.com/watch?v=MQQmDUCufS8 <- Lobe

Power Automate – finding Dataverse environment URL

There are many ways of getting the current environment URL, but this is the quick and dirty version of doing just this 😏 Next week I will post about Environment variables as I know this is a possible approach that is better.

The intention about this blog is to show how to use Json to parse just a small part of an action body to get what you want exposed.

Power Automate

So this could be a typical flow in Power Automate for Dataverse. We have a trigger on top and an action below to get more data from the element that triggered.

The BODY of the trigger doesn’t contain environment information, but only Opportunity data:

But the Action contains a lot more interesting data for this.

In order to get this data we need to parse the Json returned here to retrieve the “Odata.ID” that includes the URL for our environment.

{
    "type": "object",
    "properties": {
        "@@odata.id": {
            "type": "string"
        }
    }
}

Now you can store the string returned in a String Variable

Running the Power Automate

When you run the Power Automate the the variable Environment will now include the URL of the system running. From here you can use string variables in a formula like:

IF “Environment contains org.crm4” etc.

In my next post I will show how the environmental variables functionality in Dataverse / Dynamics, and how we can use this for the same purpose 😀

PowerPlatform Saturday Oslo 2019 – Speaker

So Power Platform Oslo was a unique experience for me, since it was the first time I was on stage at a community event. Marius Agur challenged me to create a presentation that would showcase my view of Dynamics from a business perspective. I am not the most technical person, so I try to create solutions based on simplicity for the end user within a “do it yourself approach”.

3 ways the Power Platform can increase user adoption

I was using our own company as example for this demo, and demonstrating the issues we have had when migrating from HubSpot to Dynamics 365. A journey with lots of bumps in the road, and long nights to figure out the best solution.

1. Connecting Flow to API

Part One showed how we can connect flow to an API and receive real time data from sources like Bisnode, Proff.no or in my case BRREG. I will blog about how to do this soon.

2. Creating a PowerApp for simplicity

Part 2 was showing how you can create a powerapp to simplify not only the user input, but also create a bridge between those who don’t like Dynamics CRM, but accept using a PowerApp, because it is new and fun. Blog about this will also come soon.

3. Integrating to onpremise data source

The last step was showing how we could integrate our CDS/CRM solutions to an onpremise ERP solution without a single line of code. Read more about hit HERE

People to thank

Chris Huntingford: @TattooedCRMGuy
I would never have even thought of doing community or personal exposure if it were not for your exceptional outgoing personality motivating me when you where in Oslo. I know it sounds odd, but I wanted to be like you, and spread excitement and joy to others. Somehow you made something in my brain say “why not” create a blog and speak my mind! MASSIVE thanks..

Megan Walker: @MeganVWalker
Getting to know you and understanding how you use the tools in regards of blog, wordpress, youtube has been very important. You were also so kind to be my first interview subject 🙂

Mark Smith: @nz365guy
Thank you for hosting the 90 day Mentoring Challenge. Even though you were speaking to the masses, you make it personal to participants. I have also shared many conversations with you that mean a lot to me. Thank you for guidance in som many different levels in regards to creating a profile. I might not be the fastest at getting everything done, but I will get there.

Malin Donoso Martnes: @MalinMartnes
Thank you for being persistent in your work. It inspires others to try:) You gave me the final push to submit my talk, and I don’t regret it!

CDS / Dataverse List Records Filter Query using Flow

This week I needed to use the List Records function, and I realized that I had no idea how to use the filters. Thank you Jonas Rapp for creating the FetchXML Builder!! The function “Flow List parameters” saved my day:)

Simple filters

Let’s begin with the simple filters where I get a contact with the last name of Sandsør

Test your search result with the Execute button so see that anything is actually returned. Then open the Flow List Parameters

The tool converts the Fetch XML, and magically gives the correct filter to add in our FLOW query. It can’t get much simpler than that!!

Lookups

Lookups act a little bit different with the syntax, as lookups always to. This got me quite confused before finding this tool, because I was not getting match to my result.

I am searching for contacts with a given GUID. In my case I didn’t know what the GUID was, so I randomly generated a GUID for the formula. In FLOW I substituted the GUID part of query with a dynamic variable.

Filter Query with lookups, you need to add “_” as seen above. When working with lookup you won’t get at match without the “_LookupField_Value”.

Filter linked entity

The last filter is a little more complex, and might not get used due to some limitations of Odata (Must match on unique ID for related).

In this scenario I wanted to locate all contacts with last name “Sandsør” where the regarding accountID = GUID.

Choose the main entity on top, and add “link-entity”

Make sure you have the correct relationship here. Some Lookups support more than one entity, and therefore you make sure you have the correct one.

Again we find the magic with the “Flow List Parameters“.

In this scenario we also get Expand Query result that we need to copy/paste.

Apply to Each

Once you have figured out what filter to use, you can select the “Apply to Each” function, and add custom logic in here.

Flow – Create, Update or Delete trigger error

At the time of writing this post Microsoft Flow will throw an error when you create this flow with the trigger Create, Update or Delete, and the flow is standalone.

Error

This is the following error you will see when you try to create the flow (Create, Update, Delete) without any connection to a solution.

Sollution

All you have to do, is open this in a solution (for the time being).

Using Flow to copy Document Location

One confusing obstacle with a new CRM system, is the connection between Dynamics CRM and SharePoint. As a consultant I often have to demonstrate what is really happening behind the scenes to justify why this isn’t straight forward. We have to tell the customer about different folders, different entities etc, while they simply don’t care. They just want it to work together.

One particularly interesting point is when they move between entities. Why can you work with documents in the Lead, and when you navigate over to Opportunity, you no longer see your sales documents? Technically I understand why, but from a functional perspective I understand the customer.

Recently I ran into this scenario when implementing Field Service. Sales produced documents what were important throughout the whole cycle Sales -> WorkOrder -> Assett. Navigating between the entities takes to long, so the challenged me to find a solution.

FLOW TO THE RESCUE ❤

When the opportunity was done, we needed to create 1 or more WO in Field Service. Important factor was to maintain all the work done in Opportunity with the documents, so the Field Service Technicians could see everything in one entity (WO).

Challenge: When I create a new Work Order from Opportunity, I need to copy the document location from the originating Opportunity.

Flow begins with a simple trigger. When a WO is created.

Then I check to see if the related Opportunity contains data. This is just to make sure that the WO actually is created from a Opportunity.

Then I needed to get the document location regarding the Opportunity. As you can see here I am only returning 1. There won’t be any more document locations on Opportunity the way they are using it now. If you wonder how to filter, check my other post LIST CDS RECORDS IN FLOW

This is where I create a new Document location, and set the Work Order as regarding. Flow will automatically add “FOR EACH” to loop through. In my case it doesn’t matter, because I only have 1 document location from Opportunity.

What the user sees

This is a new Opportuinty

The opportunity has documents related to it.

I then convert the opportunity to Work Order via the button in ribbon. I get a message when this is complete.

I open the document location of Work Order, and VIOLA, the same documents.

Personally I love finding quick wins like this that mean a lot for simplicity. Any CRM application will be complex for the end user, so these parts help:)