1. Triggers and bindings
1.1. Triggers are what cause a function to run
1.2. A trigger defines how a function is invoked and a function must have exactly one trigger
1.3. Triggers have associated data, which is often provided as the payload of the function
1.4. Binding to a function is a way of declaratively connecting another resource to the function
1.4.1. bindings may be connected as input bindings, output bindings, or both
1.4.2. A trigger is a special type of input binding
1.5. Data from bindings is provided to the function as parameters
1.6. You can mix and match different bindings to suit your needs
1.6.1. Bindings are optional and a function might have one or multiple input and/or output bindings
1.7. Bindings let you avoid hardcoding access to other services
1.8. Your function receives data (for example, the content of a queue message) in function parameters
1.9. You send data (for example, to create a queue message) by using the return value of the function
1.10. See attached screenshot for some examples of how you could implement different functions
1.11. Triggers and bindings are defined differently depending on the development language
1.11.1. For Python, triggers and bindings are defined in the function.json file
1.11.1.1. All triggers and bindings have a direction property in the function.json file
1.11.1.1.1. For triggers, the direction is always in
1.11.1.1.2. Input and output bindings use in and out
1.11.1.1.3. Some bindings support a special direction inout
1.11.1.2. You can connect your function to other services by using input or output bindings
1.11.2. You can create custom input and output bindings
1.11.2.1. Bindings must be authored in .NET, but can be consumed from any supported language
2. Extending the simple HTTP Trigger function
2.1. The Hello World function (auto created by choosing the HTTP trigger template) is very simple as it is triggered by a basic HTTP Get or Post request, accepts a single input parameter (name) and returns a single output parameter (a trivial text message) as an HTTP payload
2.1.1. To have your function interact with Azure resources, we will need additional bindings beyond the trigger and HTTP return payload
2.1.1.1. It's common for a binding to need a connection string in order to connect and authenticate to a bound Azure service, so we'll explore how to bind to a storage account as one of the most common examples
2.2. The connection settings for the storage account that we deployed as part of our function app is held securely for us in Application Settings
2.2.1. In VS Code, we can access the Application Settings for the function app and the connection details for the storage account are held in an application setting named: AzureWebJobsStorage
2.2.1.1. If you click the eye icon you can toggle visibility for the application setting on and off, which uses your Azure sign-in credentials for authorization, and you can see sensitive details here like the account name and account key
2.3. As a further pre-requisite, we need to install the Azure Storage extension into VS Code
2.4. Whilst in the Azure: Functions section of VS Code, press F1 to open the command palette, then search for and run the command Azure Functions: Download Remote Settings....
2.4.1. Select subscription
2.4.2. Select the function app (i.e. the one we created in our "Hello World" deployment)
2.4.3. Select the appropriate local.settings.json file as the destination for the downloaded settings
2.4.3.1. Pay attention to the path to be sure you'll be overriding the appropriate local.settings.json file
2.4.3.2. In my case, I had two local Azure projects, so needed to choose the one for Azure Functions
2.4.4. Confirm local overwrite by clicking "Yes to all"
2.5. After downloading the remote settings into our local.settings.json file, we can see all the storage account details in our local file
2.5.1. We can see that this includes secrets, which is an obvious concern when we're going to be publishing from our local machines and committing code to Git repos that are pushed up to Azure DevOps
2.5.1.1. However, because it contains secrets, the local.settings.json file never gets published, and is excluded from source control
2.5.1.1.1. We can satisfy ourselves about this by inspecting two other project files:
2.5.1.1.2. The drawback is that when other developers clone the Git repo and need to do work on the Azure functions for that function app, they will need to repeat the running of the Azure Functions: Download Remote Settings command
2.6. As a prerequisite for adding a storage output binding to our function, we need the Storage bundles extension to be installed
2.6.1. Because our host.json project file has been configured to use extension bundles, this automatically installs a predefined set of extension packages, including the required Storage bundles extension
2.7. In VS Code, right click the function.json project file and choose the option: Add binding...
2.7.1. Select binding direction = out
2.7.2. Select binding = Azure Queue Storage
2.7.3. Set name to identify binding in code = msg
2.7.3.1. Note: this could be any valid name for a Python variable
2.7.4. Set queue name = outqueue
2.7.4.1. Note: this could be any valid name for an Azure Storage Account queue
2.7.5. Select setting from local.setting.json = AzureWebJobsStorage
2.7.6. Now our function.json project file has the new output binding for the storage queue
2.7.6.1. The extension bundles made the process of adding this binding nice and simple
2.8. Now we need to add some code to our __init__.py script file that uses the new binding
2.8.1. Here's our __init__.py script before the changes
2.8.2. Here's our __init__.py script after the changes
2.8.2.1. We added msg to the main function definition as a 2nd parameter (output), typed as a queue message
2.8.2.1.1. msg: func.Out[func.QueueMessage]
2.8.2.2. We changed the main function annotated return type from an HTTP Response object to a string (str)
2.8.2.2.1. I'm not sure if this technically makes any difference, but is probably done to reflect the fact that the primary output of the function is now a string destined for a storage queue
2.8.2.3. We add an invocation of the set method on the msg object to write the supplied name argument to the storage queue
2.8.2.3.1. msg.set(name)
2.8.2.3.2. The msg parameter is an instance of the azure.functions.InputStream class
2.8.2.3.3. Its set method writes a string message to the queue, in this case the name passed to the function in the URL query string
2.9. Test locally
2.9.1. Start the debugger by pressing F5
2.9.1.1. You can also start the debugger via Run menu
2.9.1.2. This causes botht eh function app project to start and also Core Tools
2.9.1.2.1. Remember that Core Tools is the thing that let's you run Azure project code in your local machine environment (i.e. it provides a virtual environment for running the code that is integrated with VS Code)
2.9.2. Go to Azure: Functions and under Local Project, right click your function and choose the option: Execute Function Now...
2.9.2.1. Set name when prompted
2.9.2.1.1. I set it to "Ian"
2.9.2.2. Confirmation should pop up in bottom right of screen to confirm the function executed
2.9.3. Press Ctrl+C to halt debugger (and stop Core Tools)
2.9.4. Using Storage Explorer, check that the message was written correctly to the queue
2.10. Deploy to Azure and re-test function
2.10.1. In VS Code, press F1 (to open the command palette) and run command:
2.10.1.1. Azure Functions: Deploy to function app...
2.10.1.1.1. Select the folder
2.10.1.1.2. Select subscription
2.10.1.1.3. Select function app
2.10.1.1.4. Confirm the deployment when prompted
2.10.2. Deployment should be confirmed via message in bottom right of screen
2.10.3. Repeat test in the same way as for local test, with only difference being that you don't start the VS Code debugger first and you browse to your Azure subscription under Azure: Functions, not the local project
2.10.3.1. I decided to execute my function and pass the name as "Favorita"
2.10.3.1.1. Storage Explorer confirms that the Azure function wrote the new message to the queue successfully!
3. Azure Functions development process
3.1. Development of an Azure Function typically happens on a developer's local machine using VS Code
3.2. The pre-requisites and are covered via the Hello World notes
3.3. The process to create a new project is also covered via the Hello World notes
3.3.1. The only thing to note is that whilst the HTTP trigger is the most common, others are available such as Azure Blob Storage trigger, Azure Queue Storage trigger and Timer trigger
3.4. Generated project files
3.4.1. A number of project files are generated when you create a new Azure Functions project in Python (some common to all languages, some specific to Python):
3.4.1.1. host.json
3.4.1.1.1. Common to all languages, this file lets you configure the Functions host
3.4.1.1.2. These settings apply when you're running functions locally and when you're running them in Azure
3.4.1.2. local.settings.json
3.4.1.2.1. Common to all languages, this file maintains settings used when you're running functions locally
3.4.1.2.2. These settings are used only when you're running functions locally
3.4.1.2.3. Because the local.settings.json file can contain secrets, you need to exclude it from your project source control
3.4.1.3. requirements.txt
3.4.1.3.1. Python specific, this is a project-level file that lists packages required by Functions
3.4.1.4. function.json
3.4.1.4.1. Python specific, this file is placed inside a project subfolder named after your function
3.4.1.5. __init__.py
3.4.1.5.1. Python specific, this file is placed together with the function.json file in the function subfolder of the project and is referenced by function.json
3.4.1.5.2. function.json + __init__.py together represent the Azure function code
3.4.2. Folder structure for a Python Azure Function app
3.4.2.1. As well as the project level files host.json, local.settings.json and requirements.txt, and the function subfolder, there are a few additional project files and subfolders
3.4.2.1.1. .vscode/
3.4.2.1.2. .venv/
3.4.2.1.3. .funcignore
3.4.2.1.4. .gitignore
3.4.2.1.5. proxies.json
3.5. Install bindings
3.5.1. Except for HTTP and timer triggers, bindings are implemented in extension packages
3.5.2. You must install the extension packages for the triggers and bindings that need them
3.5.3. The process for installing binding extensions depends on your project's language
3.5.3.1. For Python, the easiest way to install binding extensions is to enable extension bundles
3.5.3.1.1. When you enable bundles, a predefined set of extension packages is automatically installed
3.5.3.1.2. To enable extension bundles, open the host.json file and update its contents to match the following code:
3.6. Add a function to your project
3.6.1. You can add a new function to an existing project by using one of the predefined Functions trigger templates
3.6.2. To add a new function trigger, select F1 to open the command palette, and then search for and run the command:
3.6.2.1. Azure Functions: Create Function
3.6.3. Follow the prompts to choose your trigger type and define the required attributes of the trigger
3.6.4. If your trigger requires an access key or connection string to connect to a service, get it ready before you create the function trigger
3.6.5. In Python, the result is a new subfolder in your project
3.6.5.1. The folder contains a new function.json file and the new Python code file (__init__.py)
3.7. Connect to services
3.7.1. You can connect your function to other Azure services by adding input and output bindings
3.7.2. Bindings connect your function to other services without you having to write the connection code
3.7.3. Visual Studio Code lets you add bindings to your function.json file by following a convenient set of prompts
3.7.3.1. Remember that function.json is specific to Python
3.7.4. To add a binding, open the command pallet (F1) and type:
3.7.4.1. Azure Functions: Add Binding
3.7.5. Choose the function for the new binding, and then follow the prompts, which vary depending on the type of binding being added to the function
3.7.5.1. The type of bindings you can add are limited, and depend on the selected direction
3.7.5.1.1. in options include:
3.7.5.1.2. out options include:
3.7.5.2. Here's an example of a JSON object added to the bindings array of the function.json file by the add binding command:
3.7.5.2.1. { "type": "queue", "direction": "out", "name": "msg", "queueName": "outqueue", "connection": "AzureWebJobsStorage" }
3.7.5.3. The binding may prompt the creation of a connection string in local.settings.json
3.7.5.3.1. For example, AzureWebJobsStorage references an application setting that will be added to local.settings.json
3.7.5.3.2. Remember that the local.settings.json file is not published, so you'll need to set these values up using the Azure Portal after deploying the function app
3.7.6. Update the Main definition (in the __init__.py file) to add the input or output parameter corresponding to the binding
3.7.6.1. To continue with the out(put) queue storage binding example, our main definition would change to:
3.7.6.1.1. def main(req: func.HttpRequest, msg: func.Out[func.QueueMessage]) -> str:
3.7.6.1.2. Note that the output parameter added is:
3.7.6.1.3. Note also that we've given the output parameter the name of "msg" in this example
3.7.7. Within __init__.py, add one or more appropriate method calls to the bound input/output object
3.7.7.1. To continue with the out(put) queue storage binding example, we can add this to the function:
3.7.7.1.1. msg.set(name)
3.7.7.1.2. Note that set() is an available method for objects typed func.Out[func.QueueMessage] and name references a string parameter fetched as part of the HTTP in(put) trigger
3.7.8. Follow link for more on how to learn more in general about which bindings can be added to a function
3.8. Sign in to Azure
3.8.1. Before you can publish your app, you must sign in to Azure
3.8.2. If you aren't already signed in, choose the Azure icon in the Activity bar, then in the Azure: Functions area, choose Sign in to Azure....
3.8.2.1. When prompted in the browser, choose your Azure account and sign in using your Azure account credentials
3.8.3. The subscriptions that belong to your Azure account are displayed in the Side bar
3.9. Publish to Azure
3.9.1. Visual Studio Code lets you publish your Functions project directly to Azure
3.9.2. In the process, you create a function app and related resources in your Azure subscription
3.9.3. The function app provides an execution context for your functions
3.9.4. When you publish from Visual Studio Code to a new function app in Azure, you can choose either a quick function app create path using defaults or an advanced path
3.9.4.1. The advanced path gives you more control over the remote resources created
3.9.4.1.1. Press F1 to bring up the command pallet, and enter:
3.9.5. For republishing, you can simply repeat the process from Visual Studio Code, but for team projects it is recommended to set up a CI/CD pipeline so that any pull requests into your collaboration branch trigger and automated build and release process
3.10. Get URL for function
3.10.1. To call an HTTP-triggered function from a client, you need the URL of the function when it's deployed to your function app
3.10.1.1. For example, to call an Azure Function from Azure Data Factory, I will need the appropriate URL
3.10.1.2. This URL includes any required function keys
3.10.1.3. You can use the extension to get these URLs for your deployed functions
3.10.1.3.1. Press F1 to open the command palette, and then search for and run the command:
4. POC: Call Azure Function from Data Factory
4.1. Developing Azure Functions that do something with the Azure SDK is fine but to be really useful, we need to be able to call the function as part of some automated process; calling from a Data Factory pipeline is a perfect example
4.2. In this POC, we use the ArgentoAzureSDKTestADLS function previously developed, calling it from an ADF pipeline and record the results in an Azure SQL Database table
4.3. Step 1: Azure SQL DB setup
4.3.1. We'll create a table to hold the list of file systems (containers) returned by the function:
4.3.1.1. CREATE TABLE dbo.AzureFunctionTest ( FileSystem VARCHAR(100) NOT NULL );
4.3.2. We'll create a stored proc that truncates the table, and another proc that adds a record to it
4.3.2.1. CREATE PROC dbo.InitialiseAzureFunctionTest AS BEGIN TRUNCATE TABLE dbo.AzureFunctionTest; END;
4.3.2.2. CREATE PROC dbo.UpdateAzureFunctionTest @FileSystem VARCHAR(100) AS BEGIN INSERT INTO dbo.AzureFunctionTest(FileSystem) VALUES (@FileSystem); END;
4.4. Step 2: Develop ADF pipeline
4.4.1. Add pipeline parameter to hold the POST request body for the function
4.4.1.1. We add a placeholder for the name parameter, <pipeline_name>, which we'll substitute with the actual ADF pipeline name shortly
4.4.1.2. { "name":"<pipeline_name>" }
4.4.1.3. We will be prompted to create a linked service for the Azure Function
4.4.1.3.1. We had to copy and paste in the function token as we previously configured the function to use this authentication
4.4.1.3.2. We can retrieve the function key via the Azure Portal by navigating to the function and under Function Keys, clicking the "Show values" option, which then gives us a copy button
4.4.2. Add stored procedure activity that calls: InitialiseAzureFunctionTest
4.4.3. Add Azure function activity that invokes our function, setting the method to POST and entering an expression for the (POST) Body
4.4.3.1. The POST Body expression is:
4.4.3.1.1. @replace(string(pipeline().parameters.RequestBody),'<pipeline_name>',pipeline().Pipeline)
4.4.4. Add ForEach activity that iterates over the output from the Azure Function activity, specifically the output.file_systems
4.4.4.1. Before specifying the expression for the ForEach Items, we can run the pipeline and check the output of the ForEach
4.4.4.2. The ForEach Items expression is:
4.4.4.2.1. @activity('Get File System List').output.file_systems
4.4.4.3. Inside the ForEach activity, we add a second stored procedure activity that calls: UpdateAzureFunctionTest
4.4.4.3.1. We map item(), from the ForEach activity, to the FileSystem input parameter
4.5. Step 3: Test ADF pipeline
4.5.1. After clicking Debug, we should see all steps complete successfully
4.5.2. And when we check the Azure SQL DB table contents, we should see all the file system names retrieved from the Azure function
5. Hello World: First Azure Function in Python
5.1. Prerequisites:
5.1.1. An Azure account with an active subscription
5.1.2. Install the Azure Functions Core Tools
5.1.2.1. I installed the Core Tools v3.x Windows 64-bit installer
5.1.2.2. This provides a virtual environment for running and debugging your Azure functions on your local machine prior to deploying into an Azure Function App
5.1.3. Python 3.8, 3.7 or 3.6
5.1.3.1. Check for latest on Python versions supported by Azure Functions
5.1.3.2. To check for installed versions of Python in Windows run the following in PowerShell (or Windows cmd):
5.1.3.2.1. py -0p
5.1.4. Visual Studio Code
5.1.5. Python extension for Visual Studio Code
5.1.6. Azure Functions extension for Visual Studio Code
5.2. In VS Code, under Azure and Functions section, create a new project
5.2.1. Choose a local directory for your project
5.2.1.1. I created a new Azure DevOps Git repo named argento-azure-functions and I cloned this locally, putting the new project inside a subdirectory named hello-world
5.2.2. Set the language for the project to Python
5.2.3. Select the location for your Python interpreter, which creates an alias for a new virtual environment
5.2.4. Select HTTP trigger for the template
5.2.5. Name the function
5.2.5.1. I named mine HttpArgentoHelloWorld
5.2.6. Set authorization level to Anonymous
5.2.6.1. Anonymous allows anyone to call your function endpoint
5.2.6.2. For more private functions, we are more likely to choose the Function option, which sets up a function access key
5.2.6.2.1. However, using a function access key is a shared secret model that is not actually considered best practice, and there are multiple preferred options for securing Production HTTP endpoints, whilst still leaving your authorization level as anonymous
5.2.7. Choose option: Add to workspace
5.2.8. A new __ini__.py file should be created with boiler plate code from the template and you may see the "Waiting for Azure sign-in" prompt
5.2.8.1. After clicking on the Azure sign-it, it will launch a web browser window that prompts to authenticate to Azure and once completed, you should see your subscriptions listed
5.3. Debug project locally
5.3.1. Visual Studio Code integrates with Azure Functions Core tools to let you run this project on your local development computer before you publish to Azure
5.3.2. With your __init__.py script file the focus, press F5 to start the debugger (or start via Run menu) and check that the terminal output shows the worker process is started and initialized, and a local function endpoint URL is displayed
5.3.2.1. Note: the local function endpoint in this case is:
5.3.2.1.1. http://localhost:7071/api/HttpArgentoHelloWorld
5.3.2.2. If you have trouble running on Windows, make sure that the default terminal for Visual Studio Code isn't set to WSL Bash
5.3.3. Test the function locally
5.3.3.1. With the endpoint running locally, we can now test the function
5.3.3.2. With Core Tools running, go to the Azure: Functions area, expand Local Project | Functions, then right-click your function and choose: Execute Function Now...
5.3.3.2.1. The request body pops up in JSON format: press Enter to send it
5.3.4. Halt debugging
5.3.4.1. Click inside the Terminal window to ensure it has focus
5.3.4.1.1. Press Ctrl + C to stop Core Tools and disconnect the debugger
5.4. Publish Azure function
5.4.1. Choose the Azure icon in the Activity bar, then in the Azure: Functions area, choose the Deploy to function app... button
5.4.1.1. Note that the Deploy to function app button (the up arrow) only appears when you select the function or hover the mouse over it
5.4.1.2. Provide the following information at the prompts:
5.4.1.2.1. Select folder: Choose a folder from your workspace or browse to one that contains your function app
5.4.1.2.2. Select subscription: Choose the subscription to use
5.4.1.2.3. Select Function App in Azure: Choose + Create new Function App in Azure...
5.4.1.2.4. Enter a globally unique name for the function app: Type a name that is valid in a URL path
5.4.1.2.5. Select a runtime: Choose the version of Python you've been running on locally
5.4.1.2.6. Select a location for new resources: For better performance, choose a region near you
5.4.1.3. Progress output should appear in bottom right of screen, and after a couple of minutes you should see a successful deployment message
5.4.1.4. When completed, the following Azure resources are created in your subscription, using names based on your function app name:
5.4.1.4.1. A resource group, which is a logical container for related resources
5.4.1.4.2. A standard Azure Storage account, which maintains state and other information about your projects
5.4.1.4.3. A consumption plan, which defines the underlying host for your serverless function app
5.4.1.4.4. A function app, which provides the environment for executing your function code
5.4.1.4.5. An Application Insights instance connected to the function app, which tracks usage of your serverless function
5.4.1.4.6. By default, the Azure resources required by your function app are created based on the function app name you provide, and also by default they are also created in the same new resource group with the function app
5.4.1.5. Select View Output in this notification to view the creation and deployment results, including the Azure resources that you created
5.4.1.5.1. If you miss the notification, select the bell icon in the lower right corner to see it again
5.4.1.5.2. The output should confirm successful deployment and also show the public endpoint for your newly deployed Azure function
5.5. Run function in Azure
5.5.1. In VS Code, go to Azure: Functions area and locate your function now inside your subscription, then right right and choose: Execute Function Now...
5.5.1.1. When prompted for the "name" input parameter, try changing it from "Azure" to another name and press enter
5.5.1.2. You should see a success response returned, which pops up in the bottom right of the screen
6. POC: Using Azure Function with Azure SDK: ADLS Gen2 Storage
6.1. The idea for this POC is to use the Azure SDK to have our Azure Function enumerate a list of containers (file systems) held within an ADLS Gen2 storage account
6.1.1. The Azure SDK for Python is divided into two library types: Client libraries and Management libraries
6.1.1.1. In our case, we are interested in the following client library: Storage - Files Data Lake
6.2. Additional prerequisites
6.2.1. All the same prerequisites apply as for our Hello World function, but we have one more:
6.2.1.1. Azure CLI Tools extension for VS Code
6.2.1.1.1. In VS Code, go to Extensions and search for Azure CLI Tools, then click the Install button
6.2.1.1.2. Go to the page for installing Azure CLI locally and following the link for the Windows installation
6.2.1.1.3. Once the Azure CLI installer is complete, we can verify the installation via Windows command or Powershell
6.2.1.1.4. The reason we need Azure CLI locally is primarily for some later steps to create a user service principle in Azure that represents the developer running an application from his own machine, and this service principle will be used for the local debugging of our Azure Function
6.3. In our local hello-world project, we'll create a new function
6.3.1. Go to Azure : Functions, and click the option to Create Function...
6.3.1.1. Follow the steps to create a new HTTP trigger function but for authorization level, we'll choose Function this time instead of Anonymous
6.4. As per the SDK documentation, we need to create a DataLakeServiceClient object in order for the function to interact with one of our ADLS Gen2 storage accounts
6.4.1. To create a new DataLakeServiceClient object, we need two input parameters
6.4.1.1. URI for the storage account, passed as a string
6.4.1.2. A credential, which can be any of the following:
6.4.1.2.1. SAS token string
6.4.1.2.2. Account shared access key
6.4.1.2.3. Token credential from the azure.identity client library
6.4.2. We will create a token credential, so we need to install two Python libraries using the pip command:
6.4.2.1. pip install azure-identity
6.4.2.2. pip install azure-storage-file-datalake --pre
6.4.2.3. Let's do this from inside VS Code by adding a new terminal (via the Terminal menu)
6.4.2.3.1. When prompted, let's choose our hello-world project folder as the working directory for the new terminal
6.4.2.3.2. If we switch the terminal to PowerShell Integrated, we can check our installed version(s) of Python and the version of pip installed using the following commands:
6.4.2.3.3. We install the Azure Identity client library first
6.4.2.3.4. Then we install the Azure Datalake client library
6.5. Add an environment setting for the storage account, and replicate this in our local.settings.json file
6.5.1. In Azure portal, navigate to our Azure Function App and under Configuration | Application settings, click the button: New application setting
6.5.1.1. We add a new entry, which I chose to name: StorageAccountName
6.5.1.1.1. To get the value for the data lake service, we can go to the Properties of the storage account in Azure portal
6.5.2. We add the same setting to our local.settings.json file inside VS Code
6.6. Setup for local authentication and authorization in Azure
6.6.1. We will need a service principal for our Azure Function when running it locally in VS Code
6.6.1.1. Note: when the function runs in Azure, we'll be using managed identity, not the service principal
6.6.2. In VSCode, we start a terminal and run the following Azure CLI commands:
6.6.2.1. az login
6.6.2.2. az ad sp create-for-rbac --name ian-bradshaw-az-func-sp --skip-assignment
6.6.2.2.1. Note: every developer will need their own service principal and a common convention is to include the developer name in the name of the new sp
6.6.2.2.2. If you don't give a name to your service principal, it will get a system assigned named that begins "azure-cli" followed by a timestamp
6.6.2.3. We get some important values returned to us in the terminal window for the new service principal, which we need to copy
6.6.2.3.1. appId
6.6.2.3.2. password
6.6.2.3.3. tenant
6.6.3. In VS Code, we'll edit our local.settings.json file and add 4 entries in the values section:
6.6.3.1. AZURE_SUBSCRIPTION_ID
6.6.3.1.1. Get the value for this from Azure portal | Subscriptions
6.6.3.2. AZURE_TENANT_ID
6.6.3.2.1. Copy this from the tenant value returned by the az ad sp command
6.6.3.3. AZURE_CLIENT_ID
6.6.3.3.1. Copy this from the appId value returned by the az ad sp command
6.6.3.4. AZURE_CLIENT_SECRET
6.6.3.4.1. Copy this from the password value returned by the az ad sp command
6.6.4. Assign service principal an RBAC role for the storage account using Azure portal
6.6.4.1. In this case, we choose the built-in role: Storage Blob Data Contributor
6.7. Edit our function code in the __init__.py script file using VS Code
6.7.1. First, we'll add the import statements for creating the credential object and the data lake client object
6.7.1.1. from azure.identity import DefaultAzureCredential
6.7.1.2. from azure.storage.filedatalake import DataLakeServiceClient
6.7.2. Add credential
6.7.2.1. credential = DefaultAzureCredential()
6.7.3. Run debug (press F5 or go via the Run menu) to verify that code runs without error
6.7.3.1. Gotcha! You might get a failure: No module named 'azure.identity'
6.7.3.1.1. This is probably because the Azure package installations we did earlier went into a general Python distribution on your local machine and not into the Python 3 virtual environment (.venv) that VS Code is using for your Azure Functions
6.7.3.2. If no errors are displayed in the terminal window, you will be able to navigate to Azure : Functions and under the Local Project, right-click your function and choose: Execute Function Now...
6.7.4. Add data lake service client
6.7.4.1. We need to leverage environment variables in order to pull application setting references, so we must import the os module for this at the top of __init__.py
6.7.4.1.1. import os
6.7.4.2. Below our credential object creation line, we add the line to create the data lake object, using the following to set the account_url parameter: os.environ["<application_seting_name>"]
6.7.4.2.1. data_lake = DataLakeServiceClient(account_url=os.environ["StorageAccountName"], credential=credential)
6.7.5. Run debug (press F5 or go via the Run menu) to verify that code runs without error
6.7.5.1. All being well, we will simply execute the function from our local project and see a successful message returned (pops up temporarily in bottom right of VS Code):
6.7.5.1.1. Executed function "ArgentoAzureSDKTestADLS". Response: "Hello, Azure. This HTTP triggered function executed successfully."
6.7.6. Add code to use data lake service client to get a list of file systems and convert this into a JSON format
6.7.6.1. To prepare a response in JSON format, we will capture the file systems in a list that we'll put inside a dictionary and then convert to JSON
6.7.6.1.1. We need the json module for this, so we add it to our imports at the top of __ini__.py
6.7.6.2. Call the list_file_systems() function on the data lake service client, which returns a generator that we can enumerate
6.7.6.2.1. file_systems = data_lake.list_file_systems()
6.7.6.3. Initialise a couple of variables for preparing the result, one dictionary and one list (array)
6.7.6.3.1. result_dict = {} result_arr = []
6.7.6.4. Iterate over file_systems, capturing the name property for each object and appending that to result_arr
6.7.6.4.1. for f in file_systems: result_arr.append(f.name)
6.7.6.5. Add the name input parameter and the result_arr variable to result_dict
6.7.6.5.1. result_dict["Requested by"] = name result_dict["File systems"] = result_arr
6.7.6.6. Convert result_dict to JSON format using json.dumps()
6.7.6.6.1. result_json = json.dumps(obj=result_dict, indent=4)
6.7.6.7. For debugging purposes, we can add a print statement
6.7.6.7.1. print(result_json)
6.7.7. Change return statement to pass result_json as the body for the HttpResponse()
6.7.7.1. return func.HttpResponse(result_json)
6.7.8. Run debug and then under Azure : Functions, run the function from the local project
6.7.8.1. After entering the name input parameter at the prompt, we should see the response printed out to the debug console (courtesy of the print statement we added), and the actual HTTP response will pop up in the bottom right
6.7.8.2. Here's the response, formatted as JSON:
6.7.8.2.1. Executed function "ArgentoAzureSDKTestADLS". Response: "{ "Requested by": "Azure", "File systems": [ "archive", "avq-dw-src-files", "avq-parquet", "data-flow-no-schema-files", "delta", "internetsales", "logs", "raw", "synapse", "test-files", "wrangling-data-flows" ] }"
6.8. Deploy to Azure and re-test function
6.8.1. These are the same deployment steps we've done before, but before running the Azure function, we need to take care of some security setup via Azure Portal
6.8.2. Go to Azure Portal and find the Identity page for your function app: we need to enable the system assigned identity (managed identity) for the app
6.8.2.1. After following the prompts, we'll see the identity becomes enabled (and registered in Azure Active Directory), and the object ID is revealed
6.8.3. In Azure Portal, go to the storage account and under Access Control (IAM), assign the function app managed identity to the Storage Blob Data Contributor role
6.8.4. In VS Code, go to Azure: Functions area and locate your function now inside your subscription, then right right and choose: Execute Function Now...
6.8.4.1. Gotcha! The function did not succeed because the expected response was not returned
6.8.4.1.1. Go to Azure Functions | Functions and click on the function in question
6.8.4.1.2. Under Function | Monitor | Invocations, click on the hot-linked date entry for the failed function call
6.8.4.1.3. The solution to this was simple: add the external packages to requirements.txt in VS Code and re-publish the function app
6.8.4.2. Once the gotcha is resolved, a re-run of the Azure function produces the expected HTTP response as per the local debug