Extending Streamlit usage to create data science applications and other type of applications
Writing a post or a book, it is to set a milestone to your thoughts. By the way, that is also one of the main advantage of the agile method with the Sprint concept, setting a time limit. As a consequence, empiricism and action has always my favor on theory. For sure, in digital, remaining theorical is sometimes to stay in a comfort zone with neither validation nor invalidation from the battlefield. That is a way to grow the virus of inaction that lead to impotence! Wow… sounds angry 🙂 No, let’s say, I am more weary, weary of chilly people exhausting you with formal prerequisites that will never be satisfied. I believe that these so-called prerequisites mostly reveal fear for uncertainty! But, once for all, let’s take it for granted: the place of the unforeseen cannot be eliminated. Human actions and thought are always stricken with incompleteness! That being said, it is also wised not to act without thinking, even though your thoughts are confused.
You can find the code on my github account:
- Extending Streamlit usage: https://github.com/bflaven/BlogArticlesExamples/tree/master/extending_streamlit_usage
You can watch the videos related to this article:
- Part 1 Extending Streamlit Usage – Discovering and using Python with SQlite through the console and with sqlitebrowser
- Part 2 Extending Streamlit Usage – Create a CRUD countries manager with Streamlit
- Part 3 Extending Streamlit Usage – Using Streamlit as a Wireframing Tool
That is exactly my situation, by learning python, I do not know what exactly I am looking for in the first place but discovering Streamlit step by step has become a win-win game where:
- I deepen my knowledge and practices of Python for real purpose.
- I explore possibilities of Datascience and ML.
- And create a whole series of web applications sometimes far beyond the data science perimeter!
Indeed, for my own work, as an autodidact in computer science, I do not respect any constraints except my imagination! Streamlit is announced as being the fastest way to build custom ML tool but it has also the potential to become much more than that.
It has definitely the potential to change the way you can create, build and share tools because it’s so plain simple to use.
Some take-aways from this exploration
From this multipurpose Streamlit exploration, I can withdraw several assets :
- Keep Exploring NLP and Streamlit: Still making NLP experience with Spacy. I have made some attempts with NLP usage on theater play written by my sister in a 99 project experiment.
Source : http://www.the99project.net/2021/04/09/99-ambitieuses-2/ - Best way to Leverage, Optimize and Automate any kind of P.O Work into an webapp: This approach fits perfectly with Streamlit. Anything that streamline my tasks both dull or absorbing work by processing it with a creative turn-to-key tools build on top of Streamlit is welcome! With Streamlit, I may have found the “Graal” and the Sky seems to be the limit!
- Streamlit as a Wireframing Tool: I am definitly a Wireframing fanatic! As a PO, I always try create a “user experience cinematic” to explain the feature that I want to be developed. To do this, I use numerous software such as Balsamiq, OmniGraffle, Axure… and release sometime movies out of it to show and explain as much as possible what I want. In a way, reading my tickets in Jira is more like going to the movie. In the future, I may straighly leverage on Streamlit to expose what I want in terms of UX. Building directly the app replace a real sketch. This might help me to remove all the unnecessary, annoying or confusing elements that detract developpers. Streamlit will provide me “UX fat-free” app…
- Introducing Submit button and Forms 📃
https://blog.streamlit.io/introducing-submit-button-and-forms/ - Designing Streamlit Apps for the User
https://blog.streamlit.io/designing-streamlit-apps/ - Using Streamlit and SQLite for bigger amount of Data: Replacing CSV and using the most used database engine in the world can be practical. This free database is also a great alternative to MySQL to build a webapp on the top of it if you required to store more data than in single CSV file for instance.
Some command to install some librairies via pip or anaconda.
# go to any dir if you are using anaconda cd /path/ # install spacy for efficiency python -m spacy download fr_core_news_sm # spacy.load('fr_core_news_sm'); # install spacy for accuracy python -m spacy download fr_dep_news_trf # spacy.load('fr_dep_news_trf'); |
pip install -U pip setuptools wheel pip install -U spacy python -m spacy download en_core_web_sm python -m spacy download fr_core_news_sm |
python -m spacy validate |
pip install pysqlite3 pip install db-sqlite3 |
- explosion – spacy-streamlit
https://github.com/explosion/spacy-streamlit - Turn Python Scripts into Beautiful ML Tools
https://towardsdatascience.com/coding-ml-tools-like-you-code-ml-models-ddba3357eace - Awesome Streamlit
https://awesome-streamlit.readthedocs.io/en/latest/_copy_of_project_root/AWESOME-STREAMLIT.html - TextBlob: Simplified Text Processing
https://textblob.readthedocs.io/en/dev/ - Text Analysis in Python 3 by geeksforgeeks.org
https://www.geeksforgeeks.org/text-analysis-in-python-3/ - Tutorial: Text Classification in Python Using spaCy
https://www.dataquest.io/blog/tutorial-text-classification-in-python-using-spacy/ - Excellent Intermediate Python Tutorials from realpython.com
https://realpython.com/tutorials/intermediate/ - Excellent Advanced Python Tutorials from realpython.com
https://realpython.com/tutorials/advanced/ - One of the best and educational tutorials on NLP – Natural Language Processing With spaCy in Python
https://realpython.com/natural-language-processing-spacy-python/ - A french text, source of some work – 99 ambitieuses – Récit collectif
http://www.the99project.net/2021/04/09/99-ambitieuses-2/ - Text Mining in Python: Steps and Examples
https://towardsai.net/p/data-mining/text-mining-in-python-steps-and-examples-78b3f8fd913b - Text Analytics for Beginners using NLTK
https://www.datacamp.com/community/tutorials/text-analytics-beginners-nltk - Text Analysis post by monkeylearn.com
https://monkeylearn.com/text-analysis/ - Ultimate guide to deal with Text Data (using Python) – for Data Scientists and Engineers
https://www.analyticsvidhya.com/blog/2018/02/the-different-methods-deal-text-data-predictive-python/ - A Beginner’s Guide to Exploratory Data Analysis (EDA) on Text Data (Amazon Case Study)
https://www.analyticsvidhya.com/blog/2020/04/beginners-guide-exploratory-data-analysis-text-data/ - Text Analysis in Python for Social Scientists by Dirk Hovy
https://www.cambridge.org/core/elements/
text-analysis-in-python-for-social-scientists/BFAB0A3604C7E29F6198EA2F7941DFF3 - Why You Should Do Text Analysis in Python (Even if You Don’t Want to) – Bhargav Srinivasa Desikan
https://www.youtube.com/watch?v=eKR2X2oKlFM - Text Preprocessing in Python: Steps, Tools, and Examples
https://medium.com/@datamonsters/text-preprocessing-in-python-steps-tools-and-examples-bf025f872908 - How to Clean Text for Machine Learning with Python (working with Metamorphosis by Franz Kafka)
https://machinelearningmastery.com/clean-text-machine-learning-python/ - PhantomInsights – mexican-government-report
https://github.com/PhantomInsights/mexican-government-report - TextBlob is a Python (2 and 3) library for processing textual data. https://textblob.readthedocs.io/en/dev/
- texthttps://sqlite.org/index.html
- texthttps://sqlitebrowser.org
- Github Results search on “Spacy&type=Repositories”
https://github.com/search?p=2&q=Spacy&type=Repositories - Github Results search on “Spacy+Jupyter&type=Repositories”
https://github.com/search?q=Spacy+Jupyter&type=Repositories - Github Results search on “spacy+extracting&type=Repositories”
https://github.com/search?p=2&q=spacy+extracting&type=Repositories - Github Results search on “Jupyter+Notebook&q=spacy&type=Repositoriess”
https://github.com/search?l=Jupyter+Notebook&q=spacy&type=Repositories - Rishabbh-Sahu – extractive_summarization
https://github.com/Rishabbh-Sahu/extractive_summarization/blob/main/extractive_summarization.ipynb - spacy-streamlit: spaCy building blocks for Streamlit apps
https://github.com/explosion/spacy-streamlit - demo-spacy-text-processing
https://github.com/wshuyi/demo-spacy-text-processing - Introduction to spaCy for NLP and Machine Learning
https://github.com/NSchrading/intro-spacy-nlp/blob/master/Intro_spaCy_NLP.ipynb - spaCy tuTorial
https://github.com/DerwenAI/spaCy_tuTorial - Text Data Analysis for CoMeDiAnS- NLP Project
https://github.com/NishthaChaudhary/Text-Data-Analysis-for-CoMeDiAnS–NLP-Project - jbnunn – spaCy-Notebooks
https://github.com/jbnunn/spaCy-Notebooks/blob/master/00_spacy_intro.ipynb - Hacking Buddhist Texts with NLP
https://github.com/tyler-seymour/jupyter-portfolio/blob/master/Computational-Linguistics_Chinese-NLP-Pandas-SpaCy-ZH.ipynb - DerwenAI – spaCy_tuTorial
https://github.com/DerwenAI/spaCy_tuTorial/blob/master/Extract_Text_from_PDF.ipynb - giancarlotorres22 – nlp-projeto-2
https://github.com/giancarlotorres22/nlp-projeto-2 - Applied Language Technology (University of Helsinki)
https://github.com/Applied-Language-Technology/notebooks - FinTech_WordCloud_spaCy
https://github.com/peichenli3/FinTech_WordCloud_spaCy/blob/main/WordCloud.py - Natural Language Recipe Ingredient Parser using SpaCy
https://github.com/vishwapardeshi/NL_Parser_using_Spacy - Resume-Parser
https://github.com/Kowsi/Resume-Parser/blob/master/ResumeParser.py - dewadkar – LegalNER
https://github.com/dewadkar/LegalNER - srstevenson – keyword-extractor
https://github.com/srstevenson/keyword-extractor - Chunking text and comparing similarities to extract insights from free text.
https://github.com/jzhangab/SpaCy_NLP_example - Adding a custom tokenizer to spaCy and extracting keywords
https://howard-haowen.github.io/blog.ai/keyword-extraction/spacy/textacy/ckip-transformers/jieba/textrank/rake/2021/02/16/Adding-a-custom-tokenizer-to-spaCy-and-extracting-keywords.html - explosion – spacy-transformers
https://github.com/explosion/spacy-transformers - How to Build a UI for your Model using Streamlit
https://www.section.io/engineering-education/streamlit-ui-tutorial/ - JCharis Jesse
https://github.com/Jcharis/ - How to Add Layout to Streamlit Apps
https://blog.jcharistech.com/2020/10/10/how-to-add-layout-to-streamlit-apps/ - Awesome Streamlit Awesome
https://awesome-streamlit.org/ - This gallery contains a selection of examples of the plots Altair can create.
https://altair-viz.github.io/gallery/index.html - Reference Gallery – Bokeh
https://holoviews.org/reference/index.html - pentoai / streamlit-terran-timeline
https://github.com/pentoai/streamlit-terran-timeline - Voilà Dashboards – Dashboarding with Jupyter
https://github.com/voila-dashboards/ - Datacamp – Argument Parsing in Python
https://www.datacamp.com/community/tutorials/argument-parsing-in-python - How to Execute a Python File with Arguments in Python?
https://blog.finxter.com/how-to-execute-a-python-file-with-arguments-in-python/ - 10 tips for passing arguments to Python script
https://www.codeforests.com/2020/10/18/passing-arguments-to-python-script/ - 🏆 A ranked gallery of awesome streamlit apps built by the community
https://reposhub.com/python/deep-learning/jrieke-best-of-streamlit.html - “how to install sqlite3 in python” Code Answer’s
https://www.codegrepper.com/code-examples/sql/how+to+install+sqlite3+in+python - Power Thesaurus
https://www.powerthesaurus.org/ - Masanobu Fukuoka
https://en.wikipedia.org/wiki/Masanobu_Fukuoka - Bill Mollison
https://en.wikipedia.org/wiki/Bill_Mollison - David Holmgren
https://en.wikipedia.org/wiki/David_Holmgren - Mu (zen)
https://en.wikipedia.org/wiki/Mu_(negative) - Qu’est-ce que la contingence ? (French)
https://www.franceculture.fr/emissions/repliques/quest-ce-que-la-contingence - Transidentité (French)
https://fr.wikipedia.org/wiki/Transidentité