As a PO, I always try to find out some way to increase productivity but above all is to avoid doing repeatedly boring stuff! So, it often happens that I am writing code for my own use to avoid tedious work or even the work itself!<\/p>\n
I am currently working on Python and try to sneak out practical results quickly with the help of ML, NLP or IA. As I read a lot of stuff about IA, I incidentally discovered Ludwig! So, this post is my personal tchotchkes’ collection on Ludwig. <\/p>\n
My feeling was that the more I was reading about Ludwig, the more it maybe the right tool for me! Curiosity for IA but laziness and reluctancy for theoretical explanations and coding. Anyway, enough pointless personal considerations, let’s introduce Ludwig and then use it to train, predict and visualize!<\/p>\n
Ludwig is a code free deep-learning tool box<\/b>. It is supported by a Uber research scientist named Piero Molino. Ludwig lets people without a machine learning background train prediction models without the need to write code. <\/p>\n
Ludwig can help you make deep learning easier to understand for non-experts and enable faster model improvement iteration cycles for experienced machine learning developers and researchers alike. <\/p>\n
Ludwig is a good starting to cover many ML use cases. It has:<\/p>\n
Ludwig make the model understandable at least in terms of what is predicting and what is the quality of the predictions.<\/p>\n
It streamlines the ML process and assists you in all the workflow: splits the data in training, validation, and test sets, trains the model on the training set, validates on the validation set (early stopping), predicts on the test set… and so on.<\/p>\n
You really have only set the output feature and the input feature and define the data type abstraction or a model definition without coding because it is all in .yml file.<\/p>\n
The real issue that you may encounter is really to understand what you are currently doing if you are not familiar with ML. Like me, in the past, I have mostly used pretrained models so I am not aware of all the model\u2019s subtleties and the way to fine tune a model! What a schmock!<\/b><\/p>\n
There are plenty of examples on the official documentation at https:\/\/ludwig-ai.github.io\/ludwig-docs\/examples\/<\/a><\/p>\n Ludwig requires you to use Python 3.6+. If you don’t have Python 3 installed, install it by running:<\/p>\n You may want to use a virtual environment to maintain an isolated Python environment.<\/p>\n In order to install Ludwig just run:<\/p>\n Check LUDWIG Installation<\/b><\/p>\n If it is all done! You are good, you are ready to go and be part of the IA revolution and add some wow to your professional life at least.<\/b><\/p>\n Here is a quick memo on important notions that you may often encounter if you get to grip with AI or Deep learning.<\/p>\n 3.1 Few definitions: epoch, batch, dataset…<\/b> A simple example to understand the linking between all these notions.<\/b> 3.2 Must-knows on Ludwig<\/b><\/p>\n 3.2.1 requirements for ludwig, batches<\/b> 3.2.2 Under the hood: where are the files that matters \ud83d\ude42<\/b><\/p>\n After the first training in the main folder<\/b> After each training in the results folder<\/b> – In – If you want to use model for a prediction, you just have to point to the model directory e.g – You can also visualize to compare models among them with the – There is also a Programmatic API that can be used directly in python script like any python library. Check https:\/\/ludwig-ai.github.io\/ludwig-docs\/api\/LudwigModel\/<\/a><\/p>\n 3 videos with some little mistakes but that gave a good sense of what is Ludwig and how to use if your are not an expert!<\/b><\/p>\n Well, the very first Ludwig\u2019s benefit is psychologic! Ludwig lowers the supposed high entry barriers to concepts such as Deep learning, Machine Learning, Artificial Intelligence or Visualization. So, like me, if you are a P.O, a project manager, a marketing guy, you may feel, by discovering Ludwig, a little bit the meaning of what is the Artificial Intelligence revolution is about… <\/p>\n <\/p>\n As a PO, I always try to find out some way to increase productivity but above all is to avoid doing repeatedly boring stuff! So,… <\/p>\n2. Install LUDWIG<\/h3>\n
\r\nsudo apt install python3 # on ubuntu\r\nbrew install python3 # on mac\r\n<\/pre>\n
\r\nvirtualenv -p python3 venv\r\n<\/pre>\n
\r\npip install ludwig\r\n<\/pre>\n
\r\nludwig -h\r\n<\/pre>\n
\r\n# OUTPUT\r\n\r\n-h, --help show this help message and exit\r\nludwig cli runner\r\nludwig --help or ludwig -h\r\n\r\nAvailable sub-commands:\r\n train Trains a model\r\n predict Predicts using a pretrained model\r\n evaluate Evaluate a pretrained model's performance\r\n experiment Runs a full experiment training a model and evaluating it\r\n hyperopt Perform hyperparameter optimization\r\n serve Serves a pretrained model\r\n visualize Visualizes experimental results\r\n collect_summary Prints names of weights and layers activations to use with other collect commands\r\n collect_weights Collects tensors containing a pretrained model weights\r\n collect_activations Collects tensors for each datapoint using a pretrained model\r\n export_savedmodel Exports Ludwig models to SavedModel\r\n export_neuropod Exports Ludwig models to Neuropod\r\n preprocess Preprocess data and saves it into HDF5 and JSON format\r\n synthesize_dataset Creates synthetic data for tesing purposes\r\n<\/pre>\n
3. Using LUDWIG<\/h3>\n
\nThat an extract of the most simple answer about “What is an epoch?” extracted from https:\/\/www.quora.com\/What-is-an-epoch-in-deep-learning<\/a><\/b><\/p>\n\n
\nSuppose we have a dataset of 42,000 training examples and we divide it into batches of 600. To complete 1 epoch, it would have taken 70 (42,000 divided by 600) iterations.<\/p>\n
\nYou do not need to code to test training models. What is required by Ludwig is input and output? Input: You need to have dataset in .csv format as an input plus a declarative model definition (YAML).<\/p>\n
\nRunning Ludwig will create 2 files in the main folder: <\/p>\n\n
\n– In \/results\/experiment_run\/description.json<\/code>: Like it is indicated by the name, you have all the meta information about the command so the file description.json that contains all the information on how the command was run, what was the input feature and the output feature ….<\/p>\n\/results\/experiment_run\/training_statistics.json<\/code>: the file contains basic stats on what is accuracy, loss… etc. of your model<\/p>\n\/results\/experiment_run_3\/model<\/code><\/p>\n4. EXTRAS STUFF<\/h3>\n
visualize<\/code> command.<\/p>\n5. Videos tutorials on Ludwig<\/h3>\n
\n
Conclusion<\/h3>\n
\nI must agree that it does not turn you magically into a data scientist in one night! The proof is that you may find explanations in my videos a bit shaky or even ludicrous.
\nNevertheless, that is a great opportunity to leave pre-trained models to experiment your own models. “Don’t think you’re so small, you’re not so tall”! like the Talmud says.<\/p>\nMore infos<\/h2>\n
\n
https:\/\/ludwig-ai.github.io\/ludwig-docs\/examples\/<\/a><\/li>\n
https:\/\/github.com\/ludwig-ai\/ludwig\/<\/a><\/li>\n
https:\/\/eng.uber.com\/introducing-ludwig\/<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=Ns_6Ep7GAIM<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=BTkl_qc0Plc<\/a><\/li>\n
https:\/\/www.kaggle.com\/ronitf\/heart-disease-uci<\/a><\/li>\n
https:\/\/github.com\/RPI-DATA\/course-intro-ml-app<\/a><\/li>\n
https:\/\/github.com\/srbh24\/NLP\/<\/a><\/li>\n
https:\/\/www.adaltas.com\/en\/2020\/03\/02\/ludwig-deep-learning-flask\/<\/a><\/li>\n
https:\/\/colab.research.google.com\/drive\/1J-WmhxCdwvlRDJyvcYd1TDsXhqGOMCRa?usp=sharing<\/a><\/li>\n
https:\/\/medium.com\/ludwig-ai\/the-complete-guide-to-sentiment-analysis-with-ludwig-part-i-65a9e6bc054e<\/a><\/li>\n
https:\/\/medium.com\/ludwig-ai\/the-complete-guide-to-sentiment-analysis-with-ludwig-part-ii-d9f3952a06c6<\/a><\/li>\n
https:\/\/medium.com\/ludwig-ai\/hyperparameter-optimization-with-ludwig-6e31272e43fb<\/a><\/li>\n
https:\/\/www.jcchouinard.com\/python-for-seo\/<\/a><\/li>\n
https:\/\/www.holisticseo.digital\/python-seo\/<\/a><\/li>\n
https:\/\/www.holisticseo.digital\/python-seo\/website-categorization\/<\/a><\/li>\n
https:\/\/www.searchenginejournal.com\/
\nautomated-title-tag-optimization-using-deep-learning\/390207\/<\/a><\/li>\n
https:\/\/github.com\/ranksense\/Twittorials<\/a><\/li>\n
https:\/\/medium.com\/gowombat\/first-impressions-about-ubers-ludwig-a-simple-machine-learning-tool-or-not-714962bbbedc<\/a><\/li>\n
https:\/\/medium.com\/@mrunal68\/text-sentiments-classification-with-cnn-and-lstm-f92652bc29fd<\/a><\/li>\n
https:\/\/gunicorn.org\/<\/a><\/li>\n
https:\/\/www.youtube.com\/channel\/UCBOKpYBjPe2kD8FSvGRhJwA<\/a><\/li>\n
https:\/\/www.searchenginejournal.com\/titles-meta-descriptions-automatically-python-javascript\/360108\/<\/a><\/li>\n
https:\/\/www.searchenginejournal.com\/automated-intent-classification-using-deep-learning-google-sheets\/353910\/<\/a><\/li>\n
https:\/\/www.searchenginejournal.com\/automated-intent-classification-using-deep-learning\/311309\/<\/a><\/li>\n
https:\/\/dev.to\/kojikanao\/ludwig-text-classification-training-7ii<\/a><\/li>\n
https:\/\/paiml.github.io\/python_for_datascience\/intro<\/a><\/li>\n
https:\/\/github.com\/uchidalab<\/a><\/li>\n
https:\/\/github.com\/noahgift<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=6dqG2B0XkFw<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=ukErNtrBn_s<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=2ztr7k7hTZE<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=uSOsos2eKHI<\/a><\/li>\n
https:\/\/gilberttanner.com\/blog\/introduction-to-ubers-ludwig<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=uSOsos2eKHI<\/a><\/li>\n
https:\/\/www.youtube.com\/watch?v=NB2aiRKZIok<\/a><\/li>\n
https:\/\/www.politico.com\/news\/magazine\/2021\/04\/10\/lil-nas-x-montero-satan-devil-politics-controversy-religion-480655<\/a><\/li>\n
https:\/\/medium.com\/curious\/an-introduction-to-ludwig-wittgenstein-e866ec78ed06<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"