A simple Logic App (consumption) using a on-premise data gateway to pick up files on-prem and put them in azure using a claim check pattern. This is a common need from our customers, and here I will share my solution to it.
This integration is built for reuse and easy configuration using parameters set in your deploy script.
The purpose is to duplicate it for each new folder to get files from.
This is the LA we are going to look into
The only thing I can’t parameterize is the trigger header, this must be set manually to INT1001.
That INT-number is something that can be used later on in you logs.
If you are using a Logic App standard this is equally important!
These are my parameters, each of these will be used throughout the logic app.
The value of these parameters are set during deployment of the the Logic App.
List files in folder, here I want to check if there are any on-prem files to process.
Using parameters
environmentFolders
workingFolder
Filter listfiles array by mediatype
Using parameter
mediaType
Get file content has no parameters in that way
Create blob, remember I said we are using a claim/check pattern here? Here is a part of that pattern.
Using parameters
StorageAccount
StorageAccountContainer
Name is a combination of a datetimeStamp and the name of the on-prem file. Content is set from action “Get File content”
Claim/Check
Due to storage limitations (265kb) in the Servicebus we can store the content in a blob container och leave a message on the Servicebus pointing to this content.
If you use a higher SKU you will be able to store up to 1Mb messages.
Send SB message
Using parameters
topic
StorageAccountContainer
workingFolder (hidden in the OrgSource expression)
Note! The messageId will contain the orginial filename, this will be used and logged by subscribers of this message.
Always Managed identity!
Copy file to archive / delete from workingfolder
Finally we use a combination of parameters to copy the file to the archive
Using parameters
environmentFolder
workingFolder
archiveFolder
Deployment
First we need a parameter file
Deployment
Then we deploy the Logic App using bicep to provide the ARM with parameters.
Each of these parameters are then used in the Logic App Arm.
Deployment
Its important that you change these values to read from your ARM input parameters.
Deployment
Same for your trigger
Deployment
For logging I always set the basics INTId, and EntityId as tracked properties.
Summarize
This Logic App is duplicated and deployed with ease.
By using expressions and parameters I have managed to create a simple yet powerful Logic App that does a fantastic job.
It is easy to reuse for any other kind of similar job, read from on-prem database, sFTP and more. There are of course alternativ solutions for all of that. But the concept is still the same regardless if you choose Azure Function or Logic App standard.
The design and architecture is the same!
