{"id":12707,"date":"2023-12-19T08:32:19","date_gmt":"2023-12-19T07:32:19","guid":{"rendered":"https:\/\/flaven.fr\/?p=12707"},"modified":"2026-09-16T11:45:22","modified_gmt":"2026-09-16T09:45:22","slug":"empower-your-workflow-harnessing-the-power-of-lm-studio-and-ollama-for-seamless-local-llm-execution","status":"publish","type":"post","link":"https:\/\/flaven.fr\/2023\/12\/empower-your-workflow-harnessing-the-power-of-lm-studio-and-ollama-for-seamless-local-llm-execution\/","title":{"rendered":"Empower Your Workflow: Harnessing the Power of LM Studio and Ollama for Seamless Local LLM Execution"},"content":{"rendered":"

Exploring Hugging Face, finding LLMs and packaging the LLMs into a “quick and dirty” API with FastAPI was my last sprint goal. It just ended on Monday this week.<\/p>\n

Again the title was made with AI, check the word “Harnessing<\/i><\/p>\n

For this post, you can find all files for each project on my GitHub account. See https:\/\/github.com\/bflaven\/ia_usages\/tree\/main\/ia_using_other_llm<\/a><\/b><\/p>\n

Indeed, newly appointed AI coordinator, I am working to structure my work as I would on any project.<\/p>\n

I am now starting the second Sprint with a hybrid objective:<\/p>\n

    \n
  1. I want to find a way to locally test other LLMs in a more confidential and secure way than on “classical” websites such as ChatGPT (https:\/\/chat.openai.com\/<\/a>) or the Microsoft’s copilot (https:\/\/copilot.microsoft.com<\/a>). Especially, if I start to work with more personal data. <\/li>\n
  2. I had to prepare training documents for teenagers on the discovery of the Python language, but the emergence of generative AI completely demonetizes reading a book on programming or even attending training. <\/li>\n

    By the way, you can find a set of resources from Books, Exercises, Notebooks to discover and learn Python. It exists both in French and in English on my GitHub at https:\/\/github.com\/bflaven\/learning_python_for_all<\/a><\/i><\/p>\n<\/ol>\n

    Consequently, it seemed much more opportunistic to me to provide a Jupyter Notebook or a Google Collab. Indeed, the learner can learn and practice at the same time. Even better, coupled with a tool like Ollama or Lmstudio, the learner can obtain the information via prompts to complete his\/her instruction. So, it’s two birds with one stone, learning Python self-guided by the art of the prompt.<\/b><\/p>\n

    I made a french version only for kids: https:\/\/github.com\/bflaven\/learning_python_for_all\/blob\/main\/003_code_for_family_lesson\/001_notebook_python_code_for_family_lessons.ipynb<\/a><\/i><\/p>\n

    Without even mentioning the fact that the provision of different LMMs makes it possible to:<\/p>\n

      \n
    1. Compare the performance of each of them.<\/li>\n
    2. To practice the art of prompting either personally or professionally using more “sensitive” data such as statistical data, generating real Jira tickets if you are a PO or producing Cypress Code for UAT testing if you are QA, produce python code if you are a devlopper… etc.<\/li>\n<\/ol>\n

      But, in addition to the immediate and unconstrained use of LLMs, a question remains: should we rejoice in these facilities offered by AI? All of this, at the same time, totally devalues work and therefore will inevitably \u201ckill\u201d an astronomical number of jobs. The hunt, like the question, remains open.<\/p>\n

      Loading LLM with ollama.ai or lmstudio.ai<\/h2>\n

      I found 2 great tools to query LLMs confidentially and securely. The only remark the only remark that this calls for is that lmstudio is much more UX friendly than ollama.<\/p>\n

        \n
      1. https:\/\/ollama.ai\/<\/a><\/li>\n
      2. https:\/\/lmstudio.ai\/<\/a><\/li>\n<\/ol>\n

        Baseline for lmstudio.ai<\/b>
        \nIt says what it does…<\/p>\n

        \r\nWith LM Studio, you can ...\r\n🤖 - Run LLMs on your laptop, entirely offline\r\n👾 - Use models through the in-app Chat UI or an OpenAI compatible local server\r\n📂 - Download any compatible model files from HuggingFace 🤗 repositories\r\n🔭 - Discover new & noteworthy LLMs in the app's home page\r\n<\/pre>\n
        Using ollama commands<\/b><\/p>\n
        You can find other info at https:\/\/github.com\/jmorganca\/ollama<\/a><\/p>\n
        \r\n# COMMANDS\r\n\r\n# To run and chat with Llama 2\r\nollama run llama2\r\nollama run orca-mini\r\nollama run mistral:text\r\nollama run falcon:7b\r\n\r\n\r\n# To run and chat with orca-mini\r\nollama pull llama2\r\nollama pull orca-mini\r\n\r\n# remove a model\r\nollama rm llama2\r\nollama rm orca-mini\r\nollama rm mistral\r\nollama rm falcon:7b\r\nollama rm mistral:text\r\n\r\n\r\n# list the model\r\nollama list\r\n\r\n\r\n# when you are in the model you can use\r\n>>> \/?\r\n>>> \/list\r\n>>> \/set verbose\r\n\r\n# to get out from the model\r\n\/exit\r\n\r\n# create a custom prompt\r\n# you need to create a file for your model named \"hotwater\". It look like a file\r\nollama create hotwater - .\/hotwater\r\n\r\n# ollama create [modelFileName] - .\/[modelFileName]\r\n\r\n<\/pre>\n
        Conclusion:<\/b>
        \nWith such tools, you can load any LLM at no cost, then you have unlimited access to LLMs, a great sandbox for experimentation and a pretty good Privacy. You join the band wagon of the IA revolution for better and for worse. As you may discover that your job has changed or even worse does not exist anymore. So, you can consider retraining and start a new career of “Amazon-Book-Faker”, using IA to write fake books on any subject \ud83d\ude42 2024 will be the Post-Anything (Truth, Work, Books, Training…) Era or will not be.<\/b><\/p>\n
        Videos to tackle this post<\/h2>\n
        Video #1 Using LLM in Ollama or LM Studio: Discover LLMs potential (orca-mini) help through Ollama for daily tasks in a safe and confidential way.
        \n