Empower Your Workflow: Harnessing the Power of LM Studio and Ollama for Seamless Local LLM Execution

Exploring Hugging Face, finding LLMs and packaging the LLMs into a “quick and dirty” API with FastAPI was my last sprint goal. It just ended on Monday this week.

Again the title was made with AI, check the word “Harnessing

For this post, you can find all files for each project on my GitHub account. See https://github.com/bflaven/ia_usages/tree/main/ia_using_other_llm

Indeed, newly appointed AI coordinator, I am working to structure my work as I would on any project.

I am now starting the second Sprint with a hybrid objective:

I want to find a way to locally test other LLMs in a more confidential and secure way than on “classical” websites such as ChatGPT (https://chat.openai.com/) or the Microsoft’s copilot (https://copilot.microsoft.com). Especially, if I start to work with more personal data.
I had to prepare training documents for teenagers on the discovery of the Python language, but the emergence of generative AI completely demonetizes reading a book on programming or even attending training.

By the way, you can find a set of resources from Books, Exercises, Notebooks to discover and learn Python. It exists both in French and in English on my GitHub at https://github.com/bflaven/learning_python_for_all

Consequently, it seemed much more opportunistic to me to provide a Jupyter Notebook or a Google Collab. Indeed, the learner can learn and practice at the same time. Even better, coupled with a tool like Ollama or Lmstudio, the learner can obtain the information via prompts to complete his/her instruction. So, it’s two birds with one stone, learning Python self-guided by the art of the prompt.

I made a french version only for kids: https://github.com/bflaven/learning_python_for_all/blob/main/003_code_for_family_lesson/001_notebook_python_code_for_family_lessons.ipynb

Without even mentioning the fact that the provision of different LMMs makes it possible to:

Compare the performance of each of them.
To practice the art of prompting either personally or professionally using more “sensitive” data such as statistical data, generating real Jira tickets if you are a PO or producing Cypress Code for UAT testing if you are QA, produce python code if you are a devlopper… etc.

But, in addition to the immediate and unconstrained use of LLMs, a question remains: should we rejoice in these facilities offered by AI? All of this, at the same time, totally devalues work and therefore will inevitably “kill” an astronomical number of jobs. The hunt, like the question, remains open.

Loading LLM with ollama.ai or lmstudio.ai

I found 2 great tools to query LLMs confidentially and securely. The only remark the only remark that this calls for is that lmstudio is much more UX friendly than ollama.

Baseline for lmstudio.ai
It says what it does…

With LM Studio, you can ...
🤖 - Run LLMs on your laptop, entirely offline
👾 - Use models through the in-app Chat UI or an OpenAI compatible local server
📂 - Download any compatible model files from HuggingFace 🤗 repositories
🔭 - Discover new & noteworthy LLMs in the app's home page

Using ollama commands

You can find other info at https://github.com/jmorganca/ollama

# COMMANDS

# To run and chat with Llama 2
ollama run llama2
ollama run orca-mini
ollama run mistral:text
ollama run falcon:7b


# To run and chat with orca-mini
ollama pull llama2
ollama pull orca-mini

# remove a model
ollama rm llama2
ollama rm orca-mini
ollama rm mistral
ollama rm falcon:7b
ollama rm mistral:text


# list the model
ollama list


# when you are in the model you can use
>>> /?
>>> /list
>>> /set verbose

# to get out from the model
/exit

# create a custom prompt
# you need to create a file for your model named "hotwater". It look like a file
ollama create hotwater - ./hotwater

# ollama create [modelFileName] - ./[modelFileName]

Conclusion:
With such tools, you can load any LLM at no cost, then you have unlimited access to LLMs, a great sandbox for experimentation and a pretty good Privacy. You join the band wagon of the IA revolution for better and for worse. As you may discover that your job has changed or even worse does not exist anymore. So, you can consider retraining and start a new career of “Amazon-Book-Faker”, using IA to write fake books on any subject 🙂 2024 will be the Post-Anything (Truth, Work, Books, Training…) Era or will not be.