Step by step Introducing to Azure Cloud Deployment: Deploying a FastAPI ML Feature API

by Posted on

This last post is again a journal to me of an how-to. This time, the post is dedicated on how to deploy in the Cloud a Machine Learning’s features API, built with FastAPI. For the cloud, I have chosen Azure as I will probably work with it. Indeed, the deployment in the Cloud is the last step to close my Machine Learning’s API creation process investigation. Previously, I explored roughly all the required steps from ML (Machine Learning) customization to development with FastAPI, through the discovery of specific packages like Whisper, Spacy for instance.

For this post, you can find all files for each project on my GitHub account. See https://github.com/bflaven/ia_usages/tree/main/ia_deploy_api_ml_architecture

For such exploration, I know the drill by now! Based on some stakeholders needs, I came to the unsurprising conclusion that the target will be an API, developed in Python with FastAPI, hosted in the Azure Cloud, exposing value-added AI “features” based on different models such as the ones provided by Spacy or Whisper for various contents type such as Audio, Text, Image… overcoming as much as possible the multilingual issue for non-English languages.

In these last months, I just more or less consciously combine a Design Thinking Process with DevOps Lifecycle. All that remains now is to iterate on this workflow to refine the API creation segments.

In a near future, I will shed some light over the overall methodology that has inspired me, this natural combination between Design Thinking Process & DevOps Lifecycle.

1. Methodology reminder: Involuntarily mixing Design Thinking Process & DevOps Lifecycle

I did not intend to do so but involuntarily I have taken a mix of the two methods. At least, their keywords/values have been my compass during this first part of a journey through IA!
Just as reminder for myself, for each method, here are both the basic and an acronym forged like a mantra to remember these keywords/core values.

1.1 Design Thinking Process

“Design thinking should bring your ideas to life by putting users at the center of every process.”

Acronym: E.D.I.P.T

  • Empathize – Understanding people
  • Define – Figuring out the problem
  • Ideate – Generating your ideas
  • Prototype – Creation and experimentation
  • Test – Refining the product

1.2 DevOps Lifecycle

DevOps is a software development methodology that combines software development (Dev) with information technology operations (Ops), blending these two worlds in the entire service lifecycle: from the initial product design, through the whole development process, and to production support.

Acronym: P.C.B.T.P.R.O.M

  • Planning: task management, schedules.
  • Coding: code development and review, source code management tools, code merging.
  • Building: continuous integration tools, version control tools, build status.
  • Testing: continuous testing tools that provide quick and timely feedback on business risks, performance measurement.
  • Packaging: artifact repository, application pre-deployment staging.
  • Releasing: change management, release approvals, release automation.
  • Operating: infrastructure installation, configuration and management, infrastructure changes (scalability), infrastructure as code tools, capacity planning, capacity & resource management, security check, service deployment, high availability (HA), data recovery, log/backup management, database management.
  • Monitoring: service performance monitoring, log monitoring, applications performance monitoring, end-user experience, incident management.

2. Experiments to prepare deployment of POC, made with FastAPI, on Azure

Let’s get down to practice without further delay. On many subjects and particularly on artificial intelligence, far too many people pay lip service.

Here is the code produced through the different experiments that I made to understand deploy on Azure for a webapp.

  • advanced_docker_compose_fastapi: several example using docker-compose and Makefile.
  • api_fastapi_routes: routing issue and different solutions provided with the help of ChatGPT
  • fastapi-simple-app: simple app to deploy
  • fastapi_cheatsheet: cheat sheet for fastapi for documentation especially.
  • fastapi_tiangolo_advanced_settings: advanced setting examples.
  • mamamia-fastapi-azure: application, written with the help of ChatGPT, made with FastAPI deployed to the Azure Cloud.

3. Requirement: Using docker is a requirement to deploy on Azure

Well, you will quickly discover that the must-have to deploy on Azure is to use Docker.
Docker enable you to create en development environment for your app and then deploy it to Azure. Moreover, Docker makes it easy to get started and enables easier switching between projects, operating systems, and machines.

According to me, here are the two pitfalls that I learn from Docker usage:

  1. Clean often Images and Containers: Docker takes huge amount of space disk on your local disk as you are going to create images, containers and so on. So, you must learn quickly how to undo. Undo means to clean up your local disk and get rid of the images and containers.
  2. Leverage on docker-compose.yaml & Makefile: Docker require a to know/learn a bunch of commands and concepts that will slow down your learning curb. The best idea to optimize this task is by adding to your project a docker-compose.yaml and Makefile to manage the Docker for FastAPI. The Makefile is the ‘entrypoint’ for the tools in this structure, such that you can easily run different commands without remembering the exact arguments.

Source: https://github.com/BiteStreams/fastapi-template

4. How-to: Deploying a containerized FastAPI app to Azure Container Apps

Below here is the best post that I have found made by Pamela Fox, Principal Cloud Advocate at Microsoft. I am truly focusing only on the step by step to deploy a very simple fastapi on Azure services.

Source: https://blog.pamelafox.org/2023/03/deploying-containerized-fastapi-app-to.html

I have described each step as much detail as possible because I have an annoying tendency to forget steps or to not remember commands. I also hate typing these very commands. It is the reason I added so many Makefiles or scripts to all the projects. Let do the computers do the work for me. This is the great benefit of IT, namely automation, even if sometimes it is as said Peter Drucker “There is surely nothing quite so useless as doing with great efficiency what should not be done at all.”

4.1. Installing and Using Azure Cli


The first thing to do is to install Azure Cli


# SOME COMMANDS FOR AZURE CLI

# Install the azure-cli
brew update && brew install azure-cli

# Check the install
az --version

# Log in to Azure
az login

# By default, this command logs in with a user account. CLI will try to launch a web browser to log in interactively. If a web browser is not available, CLI will fall back to device code login. To login with a service principal, specify --service-principal.

4.2. Create a simple FastAPI App to deploy to Azure


You need to have an app to deploy! Right? Here is a simple one below. You can find some more on my github account in “simple-app-fastapi-azure” and in “mamamia-fastapi-azure”

For this post, you can find all files for each project on my GitHub account. See https://github.com/bflaven/ia_usages/tree/main/ia_deploy_api_ml_architecture

# go to the dir
cd /Users/brunoflaven/Documents/03_git/ia_usages/ia_deploy_api_ml_architecture

# make a directory named "simple-app-fastapi-azure"
mkdir simple-app-fastapi-azure

# get into the directory named "simple-app-fastapi-azure"
cd simple-app-fastapi-azure

# command to delete the all directory
rm -R simple-app-fastapi-azure

# create the FastAPI files
# you can create all the files at once
touch main.py requirements.txt Dockerfile .dockerignore
# and the cut and paste the code for each file.

For the code to put in each file. See below. It reduces to the minimum.

1. main.py

#!/usr/bin/python
# -*- coding: utf-8 -*-
#
from fastapi import FastAPI, File, status
from fastapi.responses import RedirectResponse
from fastapi.responses import StreamingResponse
from fastapi.responses import FileResponse
from fastapi.middleware.cors import CORSMiddleware
from fastapi.exceptions import HTTPException
 
app = FastAPI()

# home
@app.get("/")
async def home():
    # return RedirectResponse("/docs")
    return {'Azure FastAPI test api': 'It is running'}

2. requirements.txt

fastapi
uvicorn

3. Dockerfile

# Choose our version of Python
FROM python:3.9

# Set up a working directory
WORKDIR /code

# Copy just the requirements into the working directory so it gets cached by itself
COPY requirements.txt .

# Install the dependencies from the requirements file
RUN pip install --no-cache-dir --upgrade -r requirements.txt

# Copy the code
COPY . .

EXPOSE 80

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80", "--proxy-headers"]

4. .dockerignore
I have picked an extended model for .dockerignore on this site: https://shisho.dev/blog/posts/how-to-use-dockerignore/

__pycache__
*.pyc
*.pyo
*.pyd
.Python
env
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis
Makefile
README.md

If you have a development environment, you can test your API. A development environment is made with tools like Anaconda, Poetry, Venv… etc. I am using Anaconda.


# V1
# launch the app
uvicorn main:app --reload

# in a browser you can check type URL
http://127.0.0.1:8000
http://localhost:8000
# I have declared in my hosts file cypress.mydomain.priv pointed to 127.0.0.1
http://cypress.mydomain.priv:8000

# ctrl+c to stop the server

# V2
# launch the app
uvicorn main:app --host 0.0.0.0 --port 80 --reload

# in a browser you can check type URL
http://127.0.0.1
http://localhost
# I have declared in my hosts file cypress.mydomain.priv pointed to 127.0.0.1
http://cypress.mydomain.priv


# See more at https://fastapi.tiangolo.com/deployment/manually/

4.3. Using Docker

Before the deployment in itself on Azure, it is better to run locally this new API you just build with the help of Docker.


# get into the dir
cd /Users/brunoflaven/Documents/03_git/ia_usages/ia_deploy_api_ml_architecture/simple-app-fastapi-azure

# START with increment the tags e.g test1, test2, test3... etc

# build the Docker image named "bf-fastapi-demo:test2"
docker build --tag bf-fastapi-demo:test2 .
# The application Docker has to be up and running

# run a container named "test2-bf-fastapi-container". It is an instance of an image named "bf-fastapi-demo:test1"
docker run -d --name test2-bf-fastapi-container -p 80:80 bf-fastapi-demo:test2

# The Dockerfile tells FastAPI to use a port of 80, so the run command publishes the container's port 80 as port 80 on the local computer. I visit localhost:80 to confirm that my API is up and running. 🏃🏽‍♀️

# check the app locally
http://localhost:80

# list the images
docker ps

# stop the container test2-bf-fastapi-container
docker stop test2-bf-fastapi-container

# remove the container test2-bf-fastapi-container
docker rm -f test2-bf-fastapi-container


# remove the image with bf-fastapi-demo:test2
docker rmi --force bf-fastapi-demo:test2


# END If you change the code, you can restart from START above and redo the commands.

# NUKE DOCKER IMAGES AND CONTAINERS
# remove all containers
docker rm -f $(docker ps -aq)



# remove everything
docker system prune

4.4. Deploying to Azure

Then it is time to play in the big leagues and deploy on Azure your application. One According to me, Deployment has to be treated as non-event for a POC or a real application. It has to occur as much as possible as its is the best way to learn form your errors.

Here is a list of REQUIRED VALUES that you are going to manipulate. Behind these values you have element that interact with the azure platform. I found handy to make a list to figure out before stating the deployment process all the values required.

# Example of model project
# See https://blog.pamelafox.org/2023/03/deploying-containerized-fastapi-app-to.html

docker-image-name = fastapi-demo
registry-name = pamelascontainerregistry
registry-server-login-server = pamelascontainerregistry.azurecr.io
resource-group = fastapi-aca-rg
container-registry = application-name = pamelascontainerregistry
container-env = fastapi-aca-env

# Example for a personal project
docker-image-name = fastapi-try-demo
registry-name = maltryappkdo001
login-server = maltryappkdo001.azurecr.io
resource-group = fastapi-try-rg
container-registry = application-name = maltryappkdo001
container-env = fastapi-try-env

4.1 Push image to registry

This step is required before publishing any app on Azure. When this step is completed, you do not have to redo this step, you are just updating an existing app, you can jump directly to the point 3.4


# Go to the dir
cd /Users/brunoflaven/Documents/01_work/blog_articles/ia_deploy_api_ml_architecture/simple-app-fastapi-azure/


# Deploying Option #2: Step-by-step az commands
# STEP_1. check the install
az --version

# STEP_2. Requirement, you must be logged
az login

# By default, this command logs in with a user account. CLI will try to launch a web browser to log in interactively. If a web browser is not available, CLI will fall back to device code login. To login with a service principal, specify --service-principal.

# STEP_3. Create a resource group
# Careful you can reuse an existing one but in this example I created from scratch
# COMMAND MODEL :: az group create --location eastus --name fastapi-aca-rg

# COMMAND GOOD
az group create --location eastus --name fastapi-try-rg

# if you need to delete
az group delete --name fastapi-try-rg


# STEP_4. Create a container registry wannatrycontainerregistry for the resource group "fastapi-try-rg"

# COMMAND MODEL :: az acr create --resource-group fastapi-aca-rg \ --name pamelascontainerregistry --sku Basic

# COMMAND GOOD
az acr create --resource-group fastapi-try-rg --name wannatrycontainerregistry --sku Basic


# STEP_5. Log into the registry so that later commands can push images to it:
# COMMAND MODEL :: az acr login --name pamelascontainerregistry

# COMMAND GOOD
az acr login --name wannatrycontainerregistry



# STEP_6. uploads the code to cloud and builds it there:
# COMMAND MODEL :: az acr build --platform linux/amd64 -t pamelascontainerregistry.azurecr.io/fastapi-try:latest -r pamelascontainerregistry .

# must be in a directory with a Dockerfile
# cd /Users/brunoflaven/Documents/03_git/ia_usages/ia_deploy_api_ml_architecture/simple-app-fastapi-azure

# COMMAND GOOD
az acr build --platform linux/amd64 -t wannatrycontainerregistry.azurecr.io/fastapi-try:latest -r wannatrycontainerregistry .

4.2 Deploy to Azure Container App


# You must be in the directory of your app 
# cd /Users/brunoflaven/Documents/03_git/ia_usages/ia_deploy_api_ml_architecture/simple-app-fastapi-azure


# STEP_1: (OPTIONNAL) Upgrade the extension and register the necessary providers:
az extension add --name containerapp --upgrade
az provider register --namespace Microsoft.App
az provider register -n Microsoft.OperationalInsights --wait

# STEP_2: Create an environment for the container app:
# COMMAND MODEL :: az containerapp env create --name fastapi-aca-env \
    --resource-group fastapi-aca-rg --location eastus

# COMMAND GOOD
az containerapp env create --name fastapi-try-env --resource-group fastapi-try-rg --location eastus



# This will output the URL aka defaultDomain e.g : 
# orangesmoke-cceb35b3.eastus.azurecontainerapps.io
# purpleground-2c63d040.eastus.azurecontainerapps.io


# STEP_3: You must enable the admin:
# Run 'az acr update -n pamelascontainerregistry --admin-enabled true' to enable admin first.

# COMMAND GOOD
az acr update -n wannatrycontainerregistry --admin-enabled true

# STEP_4: Generate credentials to use for the next step:
# COMMAND MODEL :: az acr credential show --name pamelascontainerregistry

# output where you get the password

# COMMAND GOOD
az acr credential show --name wannatrycontainerregistry

# OUPUT where you can find the password.
{
  "passwords": [
    {
      "name": "password",
      "value": "FAKE+Ubb8iqWTtnDVDZg1ylHcCtHTTogZDt6iULcbKC+XXXXXXXXX"
    },
    {
      "name": "password2",
      "value": "FAKE+lwCMl71GlToW8YyeAsGaskVE4X8oLE2S8HipB+XXXXXXXXX"
    }
  ],
  "username": "wannatrycontainerregistry"
}


# STEP_5: Create the container app, passing in the username and password from the credentials:

# COMMAND MODEL 
az containerapp create --name fmm-fastapi-app \
    --resource-group fmm-fastapi-rg \
    --image pamelascontainerregistry.azurecr.io/fastapi-aca:latest \
    --environment fastapi-aca-env \
    --registry-server pamelascontainerregistry.azurecr.io \
    --registry-username pamelascontainerregistry \
    --registry-password PASSWORD_HERE \
    --ingress external \
    --target-port 80

# COMMAND GOOD
# Do not forget to replace the password with one show earlier

az containerapp create --name try-fastapi-app \
    --resource-group try-fastapi-rg \
    --image wannatrycontainerregistry.azurecr.io/try-fastapi:latest \
    --environment try-fastapi-env \
    --registry-server wannatrycontainerregistry.azurecr.io \
    --registry-username wannatrycontainerregistry \
    --registry-password PASSWORD_HERE \
    --ingress external \
    --target-port 80



# Output you have your f... latestRevisionFqdn where it is the url where the app live ! Bingo

4.3 If you need to clean up everything on Azure

CAUTION: I like to undo thing or I should even say nuke things. This command is the perfect one, you “kill” all your azure environment from group to app deployed. With great power comes great responsibility.


# CAUTION: with this command, you will remove and clean up everything
az group delete --name try-fastapi-rg

4.4 If you need to update an existing application on Azure


Here the most useful 2 commands as you may change some stuff in your application, every time you want to deploy just perform these 2 commands. You must be in the root application directory and ensure that the application is running locally with Docker.

# You must be in the directory of your app 
# cd /Users/brunoflaven/Documents/03_git/ia_usages/ia_deploy_api_ml_architecture/simple-app-fastapi-azure/

# Do not forget to login if needed. Requirement, you must be logged
# az login


# Make any code updates just re-build the image (step_1) and tell the container app to update (step_2):
# COMMAND MODEL :: STEP_1 
az acr build --platform linux/amd64 \
    -t pamelascontainerregistry.azurecr.io/fastapi-aca:latest \
    -r pamelascontainerregistry .

# COMMAND MODEL :: STEP_2
az containerapp update --name fastapi-aca-app \
  --resource-group fastapi-aca-rg \
  --image pamelascontainerregistry.azurecr.io/fastapi-aca:latest


# COMMAND_GOOD_1
az acr build --platform linux/amd64 \
    -t wannatrycontainerregistry.azurecr.io/try-fastapi:latest \
    -r wannatrycontainerregistry .

# COMMAND_GOOD_2
az containerapp update --name try-fastapi-app \
  --resource-group try-fastapi-rg \
  --image wannatrycontainerregistry.azurecr.io/try-fastapi:latest 

# show the log
az webapp log tail --name wannatrycontainerregistry --resource-group try-fastapi-rg

# show the group
az group show --name try-fastapi-rg

4.5 The magic command “all-in-one”


A unique command that “Create or update a container app as well as any associated resources (ACR, resource group, container apps environment, GitHub Actions, etc.).”

# COMMAND UNIQUE MODEL 
az containerapp up \
  -g fastapi-aca-rg \
  -n fastapi-aca-app \
  --registry-server pamelascontainerregistry.azurecr.io \
  --ingress external \
  --target-port 80 \
  --source .

# CAUTION the registry-server must exist

# COMMAND_GOOD UNIQUE
az containerapp up \
  -g try-fastapi-rg \
  -n try-fastapi-app \
  --registry-server wannatrycontainerregistry.azurecr.io \
  --ingress external \
  --target-port 80 \
  --source .

5. Other infos

I just note down few elements on solutions to the obstacles encountered or essential commands. FOMO syndrom?

5.1 Fixed issue on path on Poetry

I had to solve an issue on path for Poetry, here is what works for me. I always forget the command to edit the bash profile on a Mac.

# how-to to get started you need Poetry's bin directory (/Users/brunoflaven/.local/bin) in your `PATH` as an environment variable.
Add `export PATH="/Users/brunoflaven/.local/bin:$PATH"` to your shell configuration file. Alternatively, you can call Poetry explicitly with `/Users/brunoflaven/.local/bin/poetry`.

# check the path
echo $PATH

# Added path to ~/.zshrc
# edit 
sudo vi ~/.zshrc

# Save and Update ~/.zshrc
source ~/.zshrc

# Check PATH
echo $PATH

# You can test that everything is set up by executing:
poetry --version

5.1 Some Docker commands


Always, keep in mind some docker commands that can be useful.


# Enter the following command for details on the containers currently running:
docker ps

# Remove all containers
docker rm -f $(docker ps -aq)

# Remove docker images
docker system prune

# List the containers
docker ps
docker container ls
docker ps –a

# Stop container #container-id
docker stop e53c84c9f9d2

# Below it requires a docker-compose.yml

# to stop the stack
docker-compose stop 

# to start the stack
docker-compose start

Videos to tackle this post

You can find my WALKTHROUGHS for these 5 videos walkthrough_ia_deploy_api_ml_architecture.diff (Video #1, Video #2, Video #3, Video #4, Video #5)

Video #1 seamless #fastapi #api #development: From Local Setup with #anaconda to #azure #deployment

A quick and basic API building FastAPI to test it locally with anaconda.

Video #2 Simplified #azure #deployment: #containerizing #fastapi #app with #docker & Integration with #azure

Optimizing Azure Deployment: Containerize Your FastAPI Application with Docker for Effortless Integration into the Azure Ecosystem

Video #3 mastering #azure #cloud #deployment: Deploy a Simple FastAPI Application in 10 Minutes in Azure!

A comprehensive step-by-step tutorial takes you on a journey through the entire process of deploying a FastAPI application on the Azure Cloud. From creating the Azure resource group to publishing your app, you’ll learn how to do it all using Azure Command Line Interface (AZ CLI). Get ready to streamline your FastAPI deployment on Azure in no time!”

Video #4 #streamline #azure #deployment: Simplify #docker Operations with a #makefile for Your #application

In this video, we’ll guide you through the essential steps of preparing your application for deployment on Azure. We’ll also show you how to save precious time by using a Makefile to automate and simplify the process of running Docker commands. Say goodbye to tedious typing and hello to efficient Azure deployment!

Video #5 #development: #streamlit, #fastapi, simplify the deployment of #frontend & #backend with a #makefile

Explore the advanced world of application development as we guide you through the creation of a dynamic web application using Streamlit for the frontend and FastAPI for the backend. But that’s not all – we’ll also show you how to streamline your workflow with a Makefile, eliminating the need to type out lengthy Docker commands.

More infos

Published by Bruno Flaven

By now, Bruno Flaven works as AI Coordinator. He has been a project manager in a wide variety of Internet business applications both in Mobile and in Desktop for 20 years now. You can find more information about his professional life on his personal website (www.flaven.fr) or his linkedin profile: https://www.linkedin.com/in/brunoflaven. He is currently working In France Media Monde (FMM) mostly on mobile applications (iOS and Android). He has also worked as P.O for a Backoffice project made with Symfony. He also made few trainings to facilitate the handling of the tools that he helps to make.

Related Posts