How to debug Arc-enabled Logic App deployment issues where no workflow shows up in Azure after deployment

I was trying to deploy an Arc-enabled Logic App from both my local machine & from an Azure DevOps build agent. Arc-enabled Logic Apps allow you to run your Logic App workflows from any CNCF-certified Kubernetes cluster. This is useful if you need to run your Logic Apps in an environment that is not Azure.

The code I am working on can be found on my GitHub repo.

Arc-enabled Logic Apps run Logic Apps Standard, which follow a similar local project structure to Azure Functions. These can be created using the Azure CLI with the Logic Apps extension. The key file is the src/ProcessDataAndSendEmail/workflow.json file, which contains the Logic App workflow definition.

tree /F src
Folder PATH listing for volume Local Disk
Volume serial number is 7AD2-1F10
C:\USERS\DWIGHTSCHRUTE\SOURCE\REPOS\ARC-LOGIC-APP\SRC
│   .funcignore
│   .gitignore
│   connections.json
│   host.json
│   local.settings.json
│   
├───Artifacts
│   ├───Maps
│   └───Schemas
├───ProcessDataAndSendEmail
│       workflow.json
│
└───workflow-designtime
        host.json
        local.settings.json

Locally, I was able to run all the of the commands using PowerShell 7 (this will be important later). I first need to compress the src directory so I can do a zip deploy to my Arc-enabled Logic App.

Compress-Archive .\src\* ./app.zip -Update

This produces a valid zip file that can be deployed via the Azure CLI (with the Logic Apps extension).

az logicapp deployment source config-zip --name logic-arcLogicAppCli-eus-dev --resource-group rg-arcLogicApp-eus-dev --src .\app.zip

This produces a valid Azure Logic App and runs correctly.

However, if I try to run these same commands using the default PowerShell tasks in Azure DevOps, the deployment “succeeds”, but no workflow shows up in the portal.

- task: PowerShell@2
    displayName: Create Build Artifact
    inputs:
      targetType: inline
      workingDirectory: ${{ parameters.pathToSrcDirectory }}
      script: |
        Compress-Archive ./* $(Build.ArtifactStagingDirectory)/${{ parameters.artifactName }}.zip -Update

I couldn’t find anything in the Azure activity logs about the deployment issue nor in the deployment logs.

In order to debug this, I need to login to the container running the logic app and debug how the deployment went wrong. I can login to the container running my Logic App using the az connectedk8s proxy command (follow the commands here to get the TOKEN value).

az connectedk8s proxy -n $CLUSTER_NAME -g $RESOURCE_GROUP --token $TOKEN

Now we can issue commands via kubectl. All of the Logic Apps pods will show up under the appservice-ns namespace (in my case, yours may vary).

> kubectl get pods -n appservice-ns | grep logic

logic-arclogicappcli-eus-dev-99f4d8f65-8rhvp                 2/2     Running   0             4m34s

We can now exec into that pod & investigate the file system (there are technically 2 containers in the pod, the http one is the default and the one with the Logic App code).

> kubectl exec -it logic-arclogicappcli-eus-dev-99f4d8f65-8rhvp -n appservice-ns -- /bin/sh

Defaulted container "http" out of: http, functions-http-leader, app-init (init)
#

The Logic App definition is located at /home/site/wwwroot directory. The folder & file structure should match your local project directory.

# ls -al /home/site/wwwroot
total 32
drwxrwxrwx 2 root root 4096 Aug  9 13:28  .
drwxrwxrwx 3 root root 4096 Aug  9 13:28  ..
-rwxrwxrwx 1 root root   71 Aug  9 13:24  .funcignore
-rwxrwxrwx 1 root root   78 Aug  9 13:24  .gitignore
-rwxrwxrwx 1 root root 3562 Aug  9 13:24 'ProcessDataAndSendEmail\workflow.json'
-rwxrwxrwx 1 root root 1222 Aug  9 13:24  connections.json
-rwxrwxrwx 1 root root  153 Aug  9 13:24  host.json
-rwxrwxrwx 1 root root  296 Aug  9 13:24 'workflow-designtime\host.json'

You will note that it should have 2 directories (ProcessDataAndSendEmail & workflow-designtime). However, there is something wrong with how the build artifact got unzipped. It created files with the name ProcessDataAndSendEmail\workflow.json instead of a directory named ProcessDataAndSendEmail and a file in it named workflow.json.

But if I did the deployment from my local machine, everything looks correct.

# ls -al /home/site/wwwroot
total 32
drwxrwxrwx 4 root root 4096 Aug  9 21:46 .
drwxrwxrwx 3 root root 4096 Aug  9 21:46 ..
-rwxrwxrwx 1 root root   66 Aug  9 21:03 .funcignore
-rwxrwxrwx 1 root root   73 Aug  9 21:03 .gitignore
drwxrwxrwx 2 root root 4096 Aug  9 21:46 ProcessDataAndSendEmail
-rwxrwxrwx 1 root root 1190 Aug  9 21:03 connections.json
-rwxrwxrwx 1 root root  146 Aug  9 21:03 host.json
drwxrwxrwx 2 root root 4096 Aug  9 21:46 workflow-designtime

# ls -al /home/site/wwwroot/ProcessDataAndSendEmail
total 12
drwxrwxrwx 2 root root 4096 Aug  9 21:46 .
drwxrwxrwx 4 root root 4096 Aug  9 21:46 ..
-rwxrwxrwx 1 root root 3467 Aug  9 21:03 workflow.json

How can this be? I am running the same commands on both my local machine & the build machine. Both machines are running Windows (even though the Arc-enabled Logic App is running on a Linux container).

If I dump the app.zip file generated locally & compare it to the app.zip file generated by the build agent, I can see the differences. You can dump the contents in vim.

The version I built locally uses UNIX path separators (forward slash), whereas the version created by the build agent uses Windows path separators (back slash).

The key turns out to be the version of PowerShell I was using. My machine is using PowerShell 7, which is cross-platform and will create a zip file that uses to UNIX path separators, even when run on Windows.

The version of PowerShell that was running on the build agent is not PowerShell 7, so it used the backslashes that are default on Windows.

When the file gets unzipped on the Linux pod, it doesn’t create the correct directory structure.

I changed the build agent to use Linux instead of Windows and uses the zip command instead of the PowerShell Compress-Archive command and everything is resolved.

cd ${{ parameters.pathToSrcDirectory }}
zip -r $(Build.ArtifactStagingDirectory)/${{ parameters.artifactName }}.zip ./