LOGIC APPS

Logic Apps is a part of Azure App Service, and it is basically used to automate the access and use of data across clouds without writing code. Azure Logic Apps enables customers to quickly build powerful integration solutions that connect applications and services on-premises and in the cloud. It provides a visual designer to model and automates your process as a series of steps known as a workflow. Logic Apps allow developers to design workflows. Logic Apps is a fully managed iPaaS (integration Platform as a Service) allowing developers not to have to worry about building hosting, scalability, availability, and management. Logic Apps will scale up automatically to meet demand. 

We have Created Demo Logic App to Demonstrate how to pull the data from API with pagination and store in Data Lake Storage. 

1.

We have Initialize Variable increment with value 1 which will be further use for pagination

Figure 1 Increment Variable 

2. We have use open API gorest for above logic app API end point:
https://gorest.co.in/public/v1/comments

3.

We have use http logic app object to call data from API

4. We have used parse json to parse data returned from API.

5. Sample Json  
   { 
   “meta”: { 
   ”pagination”: {
   “total”: 1840, 
   ”pages”: 92, 
   “page”: 1, 
   “limit”: 20, 
   ”links”: { 
   ”previous”: null,
   “current”: ”https://gorest.co.in/public/v1/comments?page=1″, 
    “next”: ”https://gorest.co.in/public/v1/comments?page=2″ 
   } 
   } 
   }, 
   ”data”: [ 
   { 
   ”id”: 27, 
   “post_id”: 21, 
   “name”: ”Indira Desai”, 
   ”email”: ”desai_indira@pacocha.name”, 
   ”body”: ”Cumque aliquid consequuntur. Et et voluptas.” 
   } 
   ] 

   } 

6. We use pages parse through json to know total number of pages and loop until all pages are called using until loop Activity

7.

Figure 5: Azure Blob Creation

8. We then Increment the variable by 1 until all pages are traversed.  

9. We Can Set Limit On no of times Until loop to Run If we have estimation on total number of pages. I have set limit to 100 since total number of pages returned by API are

Figure 8 Logic app execution

10.

Logic App Calls API HTTP request 92 times since number of pages.

11. 92 json file are generated named according to page number and are store storage account container test. 

Aniket Ghodinde
Data Engineer
Addend Analytics