Executing Dynamics 365 Finance Post Refresh Tasks using Custom Service and D365.Integrations Tools

Currently when copying PROD data to a lower environment like a Tier2 sandbox or a cloud hosted dev box, there is no standard way of how the data copied will be cleansed and how PROD specific endpoints will be updated to point to TEST endpoints. Some customers do it manually(yup!), some using a SQL script(requires JIT access SQL if doing on a tier2 environment) and some using a custom service.

In this post, I will solely be talking about how I was able to accomplish this with a custom service which can be called from an Azure Devops pipeline using a PowerShell script using D365.Integrations tools. This will be a two part process:

Develop a Custom Service in D365

Note that here I will not talk about how to create a customer service in Dynamics 365 Finance. For that Peter has done a splendid job explaining how to accomplish that in his blog post here Create a Custom Service in D365

When creating this service and before running the code to take request object to perform actions, a validation should be added so that the code in this service is never accidently run in a PROD environment.

Below is an example of the service class showing how I did it.

using Microsoft.Dynamics.ApplicationPlatform.Environment;
public class ABCDataCleanupServiceClass
{
public ABCDataCleanupResponse runCleanUpTasks(ABCDataCleanupRequest _request)
{
    CustParameters  custParametersTable;
    var response = new ABCDataCleanupResponse();
    const str tier2sandbox      = "sandbox.operations.dynamics.com";
    const str axcloudmachine = "axcloud.dynamics.com";
    try
    {
        //Start with validating that code only runs in a non-PROD environment
        IApplicationEnvironment env = EnvironmentFactory::GetApplicationEnvironment();
        str currentUrl = env.Infrastructure.HostUrl;
        if (strContains(currentUrl, tier2sandbox) || strContains(currentUrl, axcloudmachine))
        {
            //Code goes here
              changecompany(_request.parmDataAreaId())
            {
                try
                {
                    select forupdate custParametersTable;
                    custParametersTable.MandatoryTaxGroup = _request.parmMandatoryTaxGroup();
                    ttsbegin;
                    custParametersTable.update();
                    ttscommit;
                    response.parmDebugMessage("Param updated"); 
                    response.parmSuccess(true);
                }
                catch (Exception::CLRError)
                {
                    System.Exception interopException = CLRInterop::getLastException();
                    response.parmSuccess(false);
                    response.parmErrorMessage(interopException.ToString());
                }
            }
        }
    }
    catch (Exception::CLRError)
    {
        throw error('This a production environment');
    }
    return response;
}
}

Create an Azure DevOps Pipeline

In an Azure devops pipeline, create a new task of type PowerShell to install required tools. See script below:

Install-PackageProvider nuget -Scope CurrentUser -Force -Confirm:$false
Write-host "NUGET INSTALLED"
 
Install-Module -Name AZ -AllowClobber -Scope CurrentUser -Force -Confirm:$False -SkipPublisherCheck
Write-host "AZ INSTALLED"
 
Install-Module -Name d365fo.integrations -AllowClobber -Scope CurrentUser -Force -Confirm:$false
Write-host "D365.INTEGRATIONS INSTALLED"

Create a separate PowerShell task or use the same as per your need to call the custom service using Invoke-D365RestEndpoint. Note that before this custom service can be accessed by the script, enable the Azure App ClientID in D365 by going to Azure Active Directory Applications and assign it to a service account with system administrator role. To register the app in Azure, this blog has step by step instructions on how to do it. App Registration in Azure

Finally, below is an example for the script that has the request parameters in form of a JSON and passed as Payload to the function Invoke-D365RestEndpoint

$Payload = '{
    "_request" :
    {
        "DataAreaId": "USRT",
        "MandatoryTaxGroup": "true"
    }
}'
$oDataToken = Get-D365ODataToken -Tenant $Tenant -Url $D365EnvironmentURL  -ClientId $ClientId -ClientSecret $ClientSecret #-RawOutput -EnableException
Invoke-D365RestEndpoint  -ServiceName "ABCDataCleanupServiceGroup/ABCDataCleanupService/runCleanUpTasks" -Payload $Payload -Tenant $Tenant -Url $D365EnvironmentURL  -ClientId $ClientId -ClientSecret $ClientSecret  -Token $oDataToken

Payload variable here contains parameters that will be passed to the custom service in D365 as a JSON

D365EnvironmentURL here will be the D365 environment where these tasks are to be performed

oDataToken here is being generated for D365 URL using the function Get-D365ODataToken included in d365.integations tools

Once the script is run in this example, it would have updated this parameter

Parameter updated in D365

Are you doing something similar or have any suggestions to improve this further? Let me know in comments.

MacOS + Dynamics 365/Power Platform…whats missing?

Without going into the debate of which OS is better for a desktop computer, you may find yourself in a situation where you are using MacOS and involved in work that is related to Microsoft technologies like Dynamics 365 and/or Power Platform and wonder if everything will work in similar way as it would on a Windows. To answer that, in my experience below are few things that I found which had an Windows OS requirement and I had to switch either to a spare Windows that I had or use a VM.

Workflow Editor for Dynamics 365 Finance & SCM

When dealing with D365 Finance and Operation(or new names Finance and Supply Chain Management), you may have to build workflows for certain business processes. To be able to do that Workflow Editor is used which actually is a one-click application which won’t work on a MacOS.

Management Reporter Designer

Recently while working on a request from finance team to modify one of their financial report I stumbled upon a similar to workflow editor limitation where the management reporter designer won’t open on a MacOS computer and required Windows

Regression Suite Automation Tool (RSAT)

With all the ever growing need to customize D365 Finance, SCM and MPOS/CPOS to handle specific business scenarios comes a challenge where regression testing may become hectic and super time consuming. Thankfully Microsoft has a tool to automate such testing using the RSAT tool (which I personally love) but if you plan to use it on MacOS, you’re out of luck. RSAT requires Windows to run.

Power Automate Desktop

Isn’t it fun to see how Power Platform is evolving? And how easily several of your legacy repetitive tasks can so easily be automated so you can focus on other stuff? If you’re like me, you would have come across a task or two where you may have had to automate a process on a computer instead of running it on Power Automate as a cloud flow. If yes, Power Automate Desktop(it’s pretty slick and worth checking if you haven’t so far) is your tool however to use this, Windows OS will be required.

For me, thankfully it was just the development part where I ran into this requirement so I just used a Windows computer instead to develop and then my actual computer where the tool had to run was already a Windows.

Visio

Though a general tool, worth mentioning here as while working with D365 one may need to have a visual representation of business processes and architecture. For this purpose what better tool then Visio! Though some alternate tools are available for MacOS and even Power Point to some extent, I still felt the gap.

Have you come across anything that did not work for you on MacOS? Let me know in comments.

Convert Date To Julian In D365 Electronic Reporting Formula Editor

Often when talking to third-party systems/softwares, there is a requirement to transmit a Julian date in the file. A similar requirement here I had to pass today’s date formatted in Julian in the EFT file which was being generated using Electronic Reporting in Dynamics 365 Finance. To accomplish this, I used below formula in formula editor of electronic reporting. I am using today’s date here however as per your need, you can change that to a date from a data source/field as needed. 

1000*(INTVALUE(DATEFORMAT(TODAY(),”yyyy”))-2000)+DAYOFYEAR(TODAY())

So today’s date, 5/12/2020 gets converted to Juian date 20133

Accessing Website as an App using Edge

Recently a user reached out asking if it is possible to pin specific Dynamics 365 Finance – Customer Service page to task bar so that instead of going to the browser every time and then clicking on the specific URL, that specific form is readily available with a single click.

 I recalled I’ve done something similar in Chrome browser to add twitter but in case of D365, the option wasn’t available. However the new Edge browser from Microsoft, which comes pre-installed in Windows 10 gives us an option to install as app option for pretty much any website. Follow along below quick steps to accomplish this.

1. Go to the website you want to access as an app. Click three dots on the top left of the browser > Apps > Install This Site as an App

2. Give it a name and click Install

3. You should at this point see a new app window pop-up with your desired website. You can at this point pin it to taskbar by right clicking on the icon in the taskbar.

Are you a macOS user? No worries!

Same steps apply there as well and all you’ll need is Edge browser installed. Once installed, go to right click > Options > Keep in Dock to add this app to your dock.

Mobile Number Portability (MNP) Experience

Since I had enough with Warid’s customer support I finally planned to port my number to some other network. After researching call rates and services (especially if the network has EDGE or not) by all other I finally shortlisted Ufone  as my next network. Though I knew that I will still not get Mobilink like customer support but still Ufone suited my needs and pocket.

The whole procedure for porting my number was easy (the service center being very close to my place) and  cost me Rs. 55/- (Rs. 50 for Ufone port in charges and Rs 5 for photocopies of my SIM card and CNIC). Due to holidays, it took around 8 days for my number to be transfered to the Ufone network and I threw away my 64k(useless) Warid SIM.

Since I switched to Ufone, I have not had any deal-breaker issues and if I do find any, will have to wait for 60 days before I can switch back or to some other network.  The only issue is that I receive text messages multiple times from Warid subscribers but will see with time if this resolves itself.

I tried all available services for Internet

Starting with Telenor’s GPRS which they offered for no activation charges and then later on offered EDGE as well but they weren’t able to provide much constant availability and quality. The connection drops all the time and server timeouts are too much. So I will say they are pretty useless when it comes to internet.

Ufone, huh! they can’t even give a good voice quality and service availability let alone GPRS.

Mobilink, their GPRS service is good. When I used it, I didn’t faced any unusual disconnects.

Warid, the service provider I am loyal to since the day they entered Pakistan’s telecom market are the best in the service. GPRS, all time available with no issues , no timeouts (congestions and lower coverage area are exceptions). Also for my Zahi package was very economical, Rs. 500 + Tax for unlimited GPRS access. So it comes out around 21/day and I take full advantage of this service being unlimited :p . Emails, IMs, Voice Chat and tether internet for my PC as well. Voice chat does get choppy and lags at times but at this price, i’ll take it.

Installing Urdu Keyboard Layout in Ubuntu

Now I needed Urdu Keyboard setup for my Ubuntu. I knew that installing them on Windows system was too simple as I just downloaded an executable and installed the keyboard and then selected that in the keyboard layout. Searching for same to work for my Ubuntu I stopped my search at Center for Research in Urdu Language Processing was where I found everything I need.The link below had the fonts as well as the layout for keyboard. I have a good hand on Phonetic and same was available here..yay!!http://www.crulp.org/software/localization.htm

Downloaded the zipped fonts as well as the keyboard layout and started with copying the fonts to the fonts directory.

sudo nautilus fonts:

After this the keyboard layout needed to be edit with the new one which was obtained from below link.

http://www.crulp.org/Downloads/ur.zip

Using this command I opened the layout file and copied the contents of ur.txt file present in ur.zip, obtained above.

sudo gedit /etc/X11/xkb/symbols/pk

After saving the changes adding Pakistan keyboard layout at System > Preferences > Layout did the trick. To place a shortcut with which I can easily switch between the layouts, I added Keyboard Indicator to the panel.

Voice chat problem solved for my Linux

Had been using Yahoo or MSN Messenger on Windows for voice chat but being a newbie on Linux the first thing I was able to get hold of for voice chat were Ekiga and Skype. Ediga came with the Ubuntu and Skype I installed from the website. Feisty Fawn (7.04) version worked good on my Gusty 7.10 as well. It just needed to download 3 more packages which didn’t take much time. So Skype with which I has been using on Windows as well is good to go; only problem here is not many of my contacts use Skype right now.

I am still looking for something that allows me to chat with my MSN or Yahoo contacts when on Ubuntu.

Connecting pc to internet via cell phone

Now after my pc is up and running once again(and yes, I formatted it again!), I did some configurations so that I am able to connect it to the internet using my cell phone. I have Nokia N72 and connecting internet on Windows was not an issue. Just installed the Nokia PC Suite rest it did by itself. I just had to  select my service provider which was Warid Telecom.

Now comes Linux’s turn. The procedure was similar to installing a CDMA phone to connect internet but this time I was able to get hold of an easier way below.

Plugged in my phone into my computer using the USB cable and disconnected the established gprs connection on my phone. Went to the Terminal and wrote a command which almost did everything by itself.

Below is the command and the output I got with it.

======================================================
rao@rao-desktop:~$ sudo wvdialconf /etc/wvdial.conf
[sudo] password for rao:
Editing `/etc/wvdial.conf’.

Scanning your serial ports for a modem.

ttyS0: ATQ0 V1 E1 — failed with 2400 baud, next try: 9600 baud
ttyS0: ATQ0 V1 E1 — failed with 9600 baud, next try: 115200 baud
ttyS0: ATQ0 V1 E1 — and failed too at 115200, giving up.
ttyS1: ATQ0 V1 E1 — failed with 2400 baud, next try: 9600 baud
ttyS1: ATQ0 V1 E1 — failed with 9600 baud, next try: 115200 baud
ttyS1: ATQ0 V1 E1 — and failed too at 115200, giving up.
Modem Port Scan: S2 S3
WvModem: Cannot get information for serial port.
ttyACM0: ATQ0 V1 E1 — OK
ttyACM0: ATQ0 V1 E1 Z — OK
ttyACM0: ATQ0 V1 E1 S0=0 — OK
ttyACM0: ATQ0 V1 E1 S0=0 &C1 — OK
ttyACM0: ATQ0 V1 E1 S0=0 &C1 &D2 — OK
ttyACM0: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0 — OK
ttyACM0: Modem Identifier: ATI — Nokia
ttyACM0: Speed 4800: AT — OK
ttyACM0: Speed 9600: AT — OK
ttyACM0: Speed 19200: AT — OK
ttyACM0: Speed 38400: AT — OK
ttyACM0: Speed 57600: AT — OK
ttyACM0: Speed 115200: AT — OK
ttyACM0: Speed 230400: AT — OK
ttyACM0: Speed 460800: AT — OK
ttyACM0: Max speed is 460800; that should be safe.
ttyACM0: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0 — OK

Found an USB modem on /dev/ttyACM0.
Modem configuration written to /etc/wvdial.conf.
ttyACM0: Speed 460800; init “ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0”
======================================================

After this I needed to edit the wvdial which I did like this

rao@rao-desktop:~$ sudo gedit /etc/wvdial

[Dialer Defaults]
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = USB Modem
;Phone =

ISDN = 0
;Username =

Init1 = ATZ
;Password =

Modem = /dev/ttyACM0
Baud = 460800
Stupid Mode = 1

I then edited the commented lines by entering the phone no (*99#) and removing the semi-colon and comments. I left the username and password fields blank as there is none required to connect to internet through Warid.

Entered sudo wvdial to dial to the connection which was just configured and gave the below output.

======================================================
rao@rao-desktop:~$ sudo wvdial
[sudo] password for rao:
WvDial: WvDial: Internet dialer version 1.56
WvModem: Cannot get information for serial port.
WvDial: Initializing modem.
WvDial: Sending: ATZ
WvDial Modem: ATZ
WvDial Modem: OK
WvDial: Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
WvDial Modem: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
WvDial Modem: OK
WvDial: Modem initialized.
WvDial: Configuration does not specify a valid login name.
WvDial: Configuration does not specify a valid password.
======================================================

Last two lines show that a username and password is required for wvdial to work. I entered dummy username and password and again tried to connect.

======================================================
WvDial: WvDial: Internet dialer version 1.56
WvModem: Cannot get information for serial port.
WvDial: Initializing modem.
WvDial: Sending: ATZ
WvDial Modem: ATZ
WvDial Modem: OK
WvDial: Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
WvDial Modem: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
WvDial Modem: OK
WvDial: Modem initialized.
WvDial: Sending: ATDT*99#
WvDial: Waiting for carrier.
WvDial Modem: ATDT*99#
WvDial Modem: CONNECT
WvDial Modem: ~[7f]}#@!}!} } }2}#}$@#}!}$}%\}”}&} }*} } g}%~
WvDial: Carrier detected. Starting PPP immediately.
WvDial: Starting pppd at Mon Jan 28 12:29:48 2008
WvDial: Pid of pppd: 5625
WvDial: Using interface ppp0
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: local IP address 10.20.71.30
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: remote IP address 10.6.6.6
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: primary DNS address 10.2.3.4
WvDial: pppd: (�[06][08]H�[06][08]
WvDial: secondary DNS address 10.2.3.5
WvDial: pppd: (�[06][08]H�[06][08]
======================================================

To see if the connection has been established or not try pinging any ip. I did the following

rao@rao-desktop:~$ ping http://www.ssuet.edu.pk
PING sirsyed.ssuet.edu.pk (221.120.196.129) 56(84) bytes of data.
64 bytes from khi77.pie.net.pk (221.120.196.129): icmp_seq=1 ttl=33 time=658 ms

Ping reply was there… so yay! I’m connected.

To disconnect I will use Ctrl +C. Will I said because i still haven’t disconnected after I first logged in. 😀

Installing CDMA Phone (Qualcomm Chipset) for Internet Usage

Now after Linux been installed the first thing I needed to setup was my internet. I had a PTCL provided ZTE phone which uses CDMA technology. I started my search and soon got know this is possible.

First I tried to see if my phone is being detected by Ubuntu by using command “lsusb” on the terminal. This showed me that my phone was there.

Then I searched dmesg to see where my phone was being placed in /dev. The results of this command showed me something like “USB ACM device” and “/dev/ttyACM0” which meant that my phone was ready to be used for the internet.

In Linux wvdial is used as a connection dial just like the way dial up connection application in windows. Just because there is some command line kinda thing need to be done before you can use, it looks such a trouble. Anyways what need to be done was this code to be placed in the wvdial.conf file in /etc.

First the system didn’t allowed me to add this thing and save it in the file but since I was determined to do it searched in the book I bought (Linux for Dummies), to see how I can get root access and found command “sudo” which gave me access to edit the file.
Complete command which I used to do that was

sudoedit /etc/wvdial.conf

[Dialer ptcl]
Modem = /dev/ttyACM0
Baud = 460800
Init1 = ATZ
Modem Name = CDMA
Modem Type = USB Modem
Phone = #777
Username = vwireless@ptcl.com
Password = ptcl
Stupid Mode = 1
PPPD Options = crtcts multilink usepeerdns lock defaultroute

After this I had to make a link for this so this is the command i used

“sudo ln -s /dev/ttyACM0 /dev/ptcl”

When done I opened terminal and entered “wvdial ptcl” and saw internet connection being made an DHCP assigning my computer an IP. When I did this thing on Fedora I wasn’t able to browse, use my IMs etc. but was able to ping any IP or server which confirmed me that I was connected to the internet but there is something else which I am missing so I looked for the services which were running at that particular time and found out that the TCP/IP service (name i don’t remember) was disabled. I just enabled that and there it go, I was successful in overcoming this problem. 

Just for an example this was the output I got. Though I still have questions as to what few things on this output mean like pap-secrets and chap-secrets but for now, as long as I’m connected, I’ll leave them for later.

 

/hrao@rao:~$ wvdial ptcl
–> WvDial: Internet dialer version 1.56
–> Cannot get information for serial port.
–> Initializing modem.
–> Sending: ATZ
ATZ
OK
–> Modem initialized.
–> Sending: ATDT#777
–> Waiting for carrier.
ATDT#777
CONNECT
–> Carrier detected. Waiting for prompt.
–> Don’t know what to do! Starting pppd and hoping for the best.
–> Starting pppd at Wed Oct 3 14:21:35 2007
–> Warning: Could not modify /etc/ppp/pap-secrets: Permission denied
–> –> PAP (Password Authentication Protocol) may be flaky.
–> Warning: Could not modify /etc/ppp/chap-secrets: Permission denied
–> –> CHAP (Challenge Handshake) may be flaky.
–> Pid of pppd: 5054
–> Using interface ppp0
–> pppd: [10] [06][08]
–> pppd: [10] [06][08]
–> pppd: [10] [06][08]
–> pppd: [10] [06][08]
–> pppd: [10] [06][08]
–> local IP address 10.0.68.72
–> pppd: [10] [06][08]
–> remote IP address 2.2.2.2
–> pppd: [10] [06][08]
–> primary DNS address 202.125.148.204
–> pppd: [10] [06][08]
–> secondary DNS address 203.135.0.70
–> pppd: [10] [06][08]

 

Disconnecting
At this point if Ctrl + C is pressed in the same terminal the connection will be disconnected and something like this will be written on the terminal window and when the prompt appears again, that means the connection has been successfully closed.

Caught signal 2: Attempting to exit gracefully…
–> Terminating on signal 15
–> pppd: [10] [06][08]
–> Connect time 75.7 minutes.
–> pppd: [10] [06][08]
–> pppd: [10] [06][08]
–> Disconnecting at Wed Oct 3 15:37:21 2007

I am still searching for more on this one, like if I could have some shortcut made on my desktop like i have in windows system and let it run in background.