I was wondering: How to automatically test LLMs output to ensure that the quality\/relevance should be always present? So, you can ship LLM application online safely.<\/b><\/p>\n
This question is partly answered by “promptfoo” as its purpose is to “Test & secure your LLM apps” <\/p>\n
Source : https:\/\/www.promptfoo.dev\/<\/a><\/p>\n For this post also, you can find all files and prompts, on my GitHub account. See https:\/\/github.com\/bflaven\/ia_usages\/tree\/main\/ia_testing_llm<\/a><\/b><\/p>\n The answer is simple: “promptfoo” has the ability in the same test, to simultaneously ensure the validity of the result produced by the LLM both in form and content.<\/p>\n So, what does “promptfoo” bring to the table? My testing scenario was the following:<\/p>\n Obviously, these tests, at the LLM level, would be a useful complement to functional tests carried out with Cypress on the API, therefore on the application side.<\/p>\n The combination of the two test suites would therefore offer a guarantee of quality on the results generated by the AI, within a CI\/CD.<\/p>\n Technically, my POC is based on an LLM (Mistral) operated by Ollama. So I modified the `promptfooconfig.yaml` file to configure the `providers` so that everything is compatible with my development environment.<\/p>\n Here some good ressources that gave good insights on how to configure and use promptfoo:<\/p>\n “How to Use Promptfoo for LLM Testing”, a good post as an introdution to promptfoo is this one: https:\/\/medium.com\/thedeephub\/how-to-use-promptfoo-for-llm-testing-13e96a9a9773<\/a><\/p>\n A bunch of examples are provided by promptfoo so you can see the extent of possibilities offered by the framework. Check https:\/\/github.com\/promptfoo\/promptfoo\/tree\/main\/examples<\/a><\/p>\n For me, these 8 examples in particular have been a good source of inspiration.<\/b><\/p>\n After reading this post and browse the examples, my “shopping” list was the following. I am operating the LLMs locally with the help of Ollama and the model I am using is Mistral.<\/b><\/p>\n Below an extract on how to set the provider in the file promptfooconfig.yaml<\/b><\/p>\n The walkthrough<\/b><\/p>\n Conclusion:<\/b><\/p>\n Integrate Promptfoo into your development workflow or in your CI\/CD, this is probably the only way to enhance quality, and reliability of the LLM output.<\/p>\n Here is below the praises sung by the promptfoo’s creators themselves :<\/p>\n![\"Enhance](\"https:\/\/flaven.fr\/wp-content\/uploads\/2024\/10\/promtfoo_cycle_600x129.png\")
\nSource: promptfoo<\/i><\/p>\n\n
Good ressources on how-to use “promptfoo”<\/h2>\n
\n
\n
\r\nproviders:\r\n - id: openrouter:mistralai\/mistral-7b-instruct\r\n config:\r\n temperature: 0.5\r\n - id: openrouter:mistralai\/mixtral-8x7b-instruct\r\n config:\r\n temperature: 0.5\r\n - id: openrouter:meta-llama\/llama-3.1-8b-instruct\r\n config:\r\n temperature: 0.5\r\n<\/pre>\n
\r\n\r\n#create the dir\r\nmkdir 001_promptfoo_running\r\n\r\n# path\r\ncd \/Users\/brunoflaven\/Documents\/01_work\/blog_articles\/ia_testing_llm\/001_promptfoo_running\/\r\n\r\n# install promptfoo\r\nnpm install -g promptfoo@latest\r\n\r\n# change providers in promptfooconfig.yaml\r\n- ollama:mistral:latest\r\n\r\n# eval\r\nnpx promptfoo eval\r\n\r\n# launch commands\r\nLOG_LEVEL=debug npx promptfoo eval\r\n\r\n# launch eval\r\nnpx promptfoo eval\r\n\r\n# view result\r\nnpx promptfoo view\r\n\r\n#uninstall\r\nnpm uninstall -g promptfoo\r\n# gain space disk\r\nnpm cache clean --force\r\n<\/pre>\n
\n