Simple Usecases on AI Prototyping and few lessons from Developing a WebApp + API + LLM Combination for NLP Tasks

by Posted on

It’s not easy to document a process in progress, I am always late! Since what I am talking about has already become obsolete.

For this post, you can find all files for each project on my GitHub account. See https://github.com/bflaven/ia_usages/tree/main/ia_llms_usecases

Indeed, even though with the help of ChatGPT, I am in the process of making a prototype for an IA combination: “WebApp + API + LLM” to meet the AI needs that I have identified such as transcription, translation, generation using prompt and a bunch of various NLP tasks: Summary, NER, even Spelling Correction which has nothing to do with IA!

This Prototype’s project I have already mentioned in small touches throughout the last posts on this blog and as evidenced by the GitHub directory: https://github.com/bflaven/ia_usages

In this post, I will present two classic cases: News Topic Classification and Sentiment Analysis. As is often the case when we act, we find ourselves confronted with the reality of what needs to be done and the idea we have of it. What are the takeaways from these two POCs applying IA on Data.

What I learn from this experience?

I had already made such experiments but this time, somehow, I needed to present results and drew some conclusion. So, here they are:

  1. In a data-centric approach, data is key (dataset). The dataset goes through three stages: Bronze, Silver, Gold.
  2. 80% of the work focuses on Evaluation Strategy, 20% Model Implementation
  3. The Data-centric approach can be summarized like so :
    AI = Model + Data where you mostly focus on data.
  4. DATA IS A FUEL, as an input as an output

It is quite peculiar because you can choose a model without breaking a sweat especially for classical issues such as News Topic Classification, Sentiment Analysis but you are going to bust your ass gathering the data and structuring it! In an almost caricatured manner, the workload is often distributed as follows:

  • 80% of the work is to consolidate the data and making change management.
  • The resting 20% of the work is choosing Model and developing a combination Webapp + API + LLM. By the way, development that can be made by ChatGPT, Mistral, Codelama, llama2…. Choose your weapon.

Methodology: Some other concerns and few valuable lessons (MLOps).

There are also two major concerns to deal with: Confidentiality, Free.

In News Topic Classification issue, USECASE_2, the fact of having replaced Mistral with ChatGPT, this was therefore done free of charge and securely.

Finally, we followed what we call an ML-OPS process. Process which is defined as follows:

1. Ideation
2. Framing
3. Data Collection
4. Data Preparation
5. Feature Engineering
6. Modelling
7. Deployment
8. Change
9. Run

Here an illustration of the MLOps.
The Al Spectrum: Unveiling Layers of Intelligent Systems
Source: https://ml-ops.org/content/mlops-principles

MLOps has much in common with Design Thinking Process
Design Thinking Process

Build a release, a must-have.

After, this framing and thinking, let do the acting. The time-to-market require to release a product as quick as possible by the end of few sprints even for a POC. Thanks to the LLM such as Mistral of ChatGTP, writing a rough combination rough for a “WebApp + API + LLM” is becoming more and more a piece of cake.

You can check the directory “fastapi_usecase_1_sentiment_analysis”

https://github.com/bflaven/ia_usages/tree/main/ia_llms_usecases/fastapi_usecase_1_sentiment_analysis

With a series of ten or twenty prompts, you can jot a serious candidate. I beat recently my personal record with 50 prompts for a “WebApp + API + LLM” combination. The only reason why the number prompts inflates is that the number API’s endpoints was higher and that induced an increase complexity for the Streamlit or Vite.js UX Webapp.

Find Dataset

Due to some confidentiality, I have substituted my datasets by some sample grabbed on Kaggle. Anyway, the principles are the same and the code is strictly identical.

I found one by searching “News Topic Classification” and “Sentiment Analysis” on Kaggle

Let’s dig the usecases

The first case is called usecase_1_sentiment_analysis, a very classical one. You got text, most of the time comments and you want to classify if the text is negative or positive.

My only issue was that the text was in French but for educational, i have swap the model and give now an English sample and a model trained on English. The French Model is still in the code, commented, is based on “distilcamembert-base-sentiment” found on https://huggingface.co/cmarkea/distilcamembert-base-sentiment

The English model is https://huggingface.co/lxyuan/distilbert-base-multilingual-cased-sentiments-student

I have also drastically reduced the sample to 1000 rows for the Sentiment Analysis because it works easily on my Mac.

See the sample at https://www.kaggle.com/code/robikscube/sentiment-analysis-python-youtube-tutorial/input

The second case is more sophisticated than the first one: usecase_2_text_classification. It is an issue on text classification e.g. Posts made on Social Networks have to re-categorized. I have made 2 user stories on this issue that I have turned to prompts sent to Mistral.

What is amazing is that with LMM is that as far you have created an API that is able to query with a prompt a LLM such as Mistral then you are good to go to implement whatever user story you want! See below, two prompts slightly different but the mechanic remains the same.

     V1 choice in a limited number of predefined categories
    # first_prompt = ChatPromptTemplate.from_template(
    #     "Act as a highly intelligent news chatbot and classify the given news text into one of the following categories only: France, Europe Africa, America, Asia-Pacific, Middle East, Sports, Economy, Technology, Culture, Environment. Do not code. Return only one word answer with only the category name that the given news text belongs to. In the output, return the result in a field named \"category_predicted:\" and return the comment in the field \"category_decision:\" in a python Dictionary.\t News text: {news}?"
    # ) 

    # V2 no choice let the LLM defined the category
    first_prompt = ChatPromptTemplate.from_template(
        "Act as a highly intelligent news chatbot and classify the given news text into the most adequate single category. Do not code. Return only one word answer with only the category name that the given news text belongs to. In the output, return the result in a field named \"category_predicted:\" and return the comment in the field \"category_decision:\" in a python Dictionary.\t News text: {news}?"
    )

Illustrations

I was looking for illustrations for “Data Science Process Lifecycle” and also to have a big picture on the connection between machine learning, AI…etc

I found these 2 images that I found quite interesting mostly because it is not ugly and incidentally because it is straightforward.

Data Science Lifecycle: This is pretty much all the stages of transformation that you will go through you and your data before learning anything from it.
Data Science Lifecycle

A classic but effective image to locate the different areas when we talk about AI.
The Al Spectrum: Unveiling Layers of Intelligent Systems

I grab these pictures from these sources:

  1. Understanding the Data Science Lifecycle: https://www.sudeep.co/data-science/2018/02/09/Understanding-the-Data-Science-Lifecycle.html
  2. Machine Learning: A Quick Introduction and Five Core Steps: https://centricconsulting.com/blog/machine-learning-a-quick-introduction-and-five-core-steps/
  3. Natural Language Processing and Machine Learning: https://www.encora.com/insights/natural-language-processing-and-machine-learning
  4. The ai spectrum: unveilling layers of intelligent system: https://www.scribbr.com/ai-tools/deep-learning/

Presenting the result

In addition, I have made a jupyter notebook to easily present the ML process especially the data exploitation.

You can check the file usecase_1_sentiment_analysis.ipynb. Below, you have the command to leverage on the anaconda environment created for the usecase.

The jupyter notebook is available at https://github.com/bflaven/ia_usages/tree/main/ia_llms_usecases/usecase_1_sentiment_analysis

How to launch a jupyter notebook on a specific environment?

# In base launch the specific sentiment_analysis with this command
python -m ipykernel install --user --name=sentiment_analysis

# Activate the env sentiment_analysis
source activate sentiment_analysis

# Launch the jupyter notebook in this env sentiment_analysis
jupyter notebook

# Create the virtual environment
conda create -n 'environment_name'

# Activate the virtual environment
conda activate 'environment_name'

# Make sure that ipykernel is installed
pip install --user ipykernel

# Add the new virtual environment to Jupyter
python -m ipykernel install --user --name='environment_name'

# To list existing Jupyter virtual environments
jupyter kernelspec list

# To list existing conda environments
conda env list

## To remove conda environment
conda env remove -n 'environment_name'

## To remove the environment from Jupyter
jupyter kernelspec uninstall 'environment_name'

Create an anaconda environment

[env]
# Conda Environment
conda create --name sentiment_analysis python=3.9.13
conda info --envs
source activate sentiment_analysis
conda deactivate

# if needed to remove
conda env remove -n [NAME_OF_THE_CONDA_ENVIRONMENT]
conda env remove -n sentiment_analysis
conda env remove -n faststream_kafka



# update conda 
conda update -n base -c defaults conda

# to export requirements
pip freeze > requirements.txt

# to install
pip install -r requirements.txt

More infos

Published by Bruno Flaven

Bruno Flaven has been a Project Manager in a wide variety of Internet business applications both in Mobile and in Desktop for 20 years now. You can find more information about his professional life on his personal website (www.flaven.fr) or his linkedin profile: https://fr.linkedin.com/in/brunoflaven. He is currently working In France Media Monde (FMM) mostly on mobile applications (iOS and Android). He is currently P.O for a Backoffice project made with Symfony. He also made few trainings to facilitate the handling of the tools that he helps to make.

Related Posts