I have allready posted about the Customer Lookup, but there is a bug ATM that will prevent you from setting the customer lookup correctly.
Luckily Microsoft has provided us with a temporary workaround while they figure out how they are going to support polymorphic lookups like Customer in dataflows 👌
When trying to map customer lookup you won’t be able to see the correct lookup in the DataFlow. It may differ what you see, but it could be a different combination of fields like this:
All of the fields that have “Fieldsomething.Fieldsomethingelse” are lookups. The Customer lookup does not work as intended here, even though the field “ParentCustomerID” is the correct field to use. In a previous post I showed how to get around this issue, but Microsoft removed the projects configuration options. After talking to support, they showed me how to make it work again !!🙏💪
In our system for billing, we have the following structure. A debitor (aka Account), can potentially have lots of projects connected. The only way to connect data in DataFlow is via KEY’s in the Dynamics configuration. In our case we have a key “AccountVAT”. This ID is a unique organization number used in Norway.
Both tables know AccountID, but only the Account table holds the AccountVAT number. As you see on the image above, the Project doesn’t know what AccountVAT to connect to. Lets fix that.
So I load up my 2 tables in dataflow. If you wonder how this is done, check my other posts on DataFlow.
Select the MERGE QUERIES option
Here we match the 2 tables based on AccountID. As you see here there are lots of options for matching, but now you also get a nice visual for matching status at the bottom.
Next step is choosing what data to “join” into the projects table.
So I select the AccountVAT number as the field to join.
As you see in the picture below, the Debitor.Foretaksnr is now visible. This column is now joined from the Debitor table to the Project table.
The selected row below is a special field. You see that the Debitor.foretaksnr is a field in another table, and you see the destination field being special. The destination field is of type Lookup, and is expecting “AccountNumber” as matching.
The result is projects record linked to the correct account in CRM 👍💪
I have been posting a lot from the Data Flow in Power Platform, because I needed to use it internally. So far this tool is extremely interesting, but I am not sure it is 100% clear to me how it all works.
This is why i decided to import 94.000 contacts to 1 account, to see how fast/slow it was. I also wanted to monitor how much this affects the API or any other stuff in the CDS/Dynamcis base. This post is mostly for my own curiosity.
The run itself didn’t show much of timestamps, so i opened an advanced find
So it used 1 hour for 94883 records. Not really sure how we can measure this anymore, because of speed caps hitting a little randomly. I would say this is decent speed when importing. 94883 contacts / 60 minutes / 60 seconds = 26,35 contacts pr second
The storage grew a fair bit…. 2.72GB -> 3.44GB = 720 MB = 0,007 MB pr contact
This is the old Address entity.. Anyone even use this anymore???
How about API calls?
The first time i ran this, it was only create. Then i did a second round, and all I got was 13,544 API calls. I am not sure I trust the Analytics yet!
After running the import 2 times, i never got any indication about API usage. There must be something wrong with the statistics here? I did almost 100k upserts, and there is nothing to prove that I have accept for the Creates under performing operations. This alone would make this an unbelievable integration platform for CDS because of the API’s we are saving. I understand that this is probably not the case, and there must be something wrong with the reporting.
That being said. No one has been able to tell me if this cost any money. If this is considered a part of Dynamics/PowerPlatform license i am impressed. A truly hidden GEM.
I still don’t know if this is local to my tennant, but I have tried in 2 different orgs, and I always get an error with Customer field for Contact. This post is almost identical to the Set Parent Lookup, but there are some differences in the end.
I have one Excel file with 6 contacts. They are linked to the account with number 311. The unique identifier I have on contact is email.
I setup a key for email on contact, so that the system will understand what to write to.
On the Account I have 2 fields for number matching, because I have different systems that integrate, but it doesn’t matter in this case. PO Account Number is what we are going to use, and I will try to explain why (based on my findings).
PS: Both Account Number and PO Account Number are Alternate Keys on Account!!
Setup Data Flow
Go to https://make.powerapps.com and start a new Data Flow project. In this case I am choosing the Excel file for simplicity. I have uploaded the Excel file to One Drive, so that it is available at all times online.
In the Excel file I had to make sure that the first row was headers before i continue
During the mapping I will see both the Alternate Keys I have for account. Normally you probably only have 1 Alternate Key for Account, but my setup requires 2 because of 2 different systems integrating against 2 different numbers. For blog i am using value 311, so it doesn’t really matter here.
When done mapping Account Number (again it doesn’t matter if I choose one of the other) continue to the next step. I choose a manual refresh, and it get stuck here.
You will also see an error in the data flow projects.
Open data integrations, and open the one that failed.
From here you can open the mapping table
This is where you most likely will see a missing mapping. For some reason it cancels out my mapping. I have tried this in 2 environments and same issues. Every time i choose the Customer Lookup i have this problem.
Click on the destination field, and navigate WAY to the bottom.
This is where I believe the bug is located. Data Flows happens to be sensitive to what Alternate Key was created last. This key is the last thing that I published on Account, and therefore it is in the list. If you only have 1 alternate key, this will show.
Again, it doesn’t matter in this demo because i have 311 in both KEY fields.
The Work Around
After you save the changes, open Data integration. Here you will setup a schedule to run from the admin.powerplatform.microsoft.com.
Click save schedule and wait for it to run. PS!! Remember to stop the schedule when done!
For the Final result you will see 6 contacts connected to the Account
Next up is how long does it take to import 94000 contacts to CDS via Data Flow!
This blog post is step 3 of the Data Flow setup in combination with On Premise Gateway. In the last post in imported all of our ERP accounts to the CDS database. Now I wanted to import all active projects, and connect them to the the Accounts in CDS. There are a few odd things you have to do in the setup to make this work, but I will try to outline this without being to detailed..
First off we need to get familiar with the Alternative Keys in Dynamcis. In Account I created alternative key for AccountNumber. I need to do the same for Project.
The field is also represented on the form. I choose to keep these as “Single Line Of Text”. Just keep it simple, because this doesn’t need to be advanced.
This is important so the import is able to match every time runs. Remember that the Data Flow is doing UPSERTS. It will create a new record if it can’t find the project number. Imports CAN’T connect to GUID’s because my ERP system does not have the CRM GUID.
The most important thing you do here, is make sure you have the correct environment!! 😉
If you don’t understand this setup, look at my first blog about the DataGateway
The important pieces of information here are: * Project Number * Project Name *Account Number
This is all i need to create a project in CRM and connect it to the Account
Connecting to Lookup
This is where you need to pay close attention. First you choose the project (or other entity) from a list of existing entities. The Alternate Key will display if you have done this correctly. If nothing is there you have to go back and start again with alternate keys
There are 2 lookups on project. Account and Contact. Account has 2 alternate keys, and Contact has 1 alternate key.
This is why you will see 3 fields in the beginning of the mapping 1. ncg_Account.AccountNumber – Account looup identifier 2. ncg_Account.ncg_POAccountNumber – Accound lookup identifier 3. ncg_CustomerContact.EMailAddress1 – Contact lookup identifier
I am using the AccountNumber for a different purpose, so let’s just forget this one for now. I am using the Account Number from ERP to the ERP field on the Account entity to create a match.
After you have set a schedule, you should see the results in CRM:) In a later post I will tell you why I am connecting the project records to dynamics. A quick hint, Forms PRO and NPS!