ml5js / ml5-friendly-intro-to-ml-2019f

A friendly introduction to machine learning and ml5.js - workshop

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

A Friendly Introduction to Machine Learning with ml5.js

Prepared for Golan Levin's Interactivity and Computation at CMU by Joey Lee
Forked from The intro to ML with ml5.js originally prepared for ITP Camp by Yining Shi & Joey Lee

keywords: machine learning, ml5.js, p5.js, friendly, introduction, workshop
Support generously provided by: The Frank-Ratchye STUDIO for Creative Inquiry at Carnegie Mellon University and NYU's Interactive Telecommunications Program.

TLDR

  • This is an introductory workshop to machine learning with ml5.js
  • Context/Audience: prepared for Golan Levin's Interactivity and Computation at CMU. Students in this course are are creative tech folks or creative tech adjacent and are considered "intromediates" -- "students develop the skills and confidence to produce interactive artworks, discuss their work in relation to current and historic praxes of digital art, and engage new technologies critically." -- but we hope this workshop can be used with audiences of all shapes and types.
  • General overview of machine learning concepts and terms with emphasis on learning through making.
  • We use ml5.js, a javascript library to do machine learning for the web.

About

Welcome to this friendly introduction to machine learning with ml5.js. We're happy you're here.

Code of Conduct

ml5.js is about machine learning and people. A major part of the ml5.js project is about creating broader engagement with machine learning and its effects on you, me, and the people and environment. As a user of ml5.js you are agreeing to abide by the ml5.js code of conduct. It's easy and fun to be nice and respectful to those around you. Let's have a wonderful workshop together. If you're looking for more details on what constitutes being "not nice" please see Mozilla's Community Participation Guidelines which are applicable to the space that ml5.js wants to create.

If you have any questions or need to report any issues please contact: info@ml5js.org

Overview:

Agenda

~3 hours

  • Hello! (5 mins)
  • Intro to Machine Learning (30 mins)
  • Intro to ml5.js (15 mins)
  • 10 Min Break
  • Coding Session Part 1 (~30 mins)
  • Coding Session Part 2 (~45 mins)
  • Project Shareouts (20 mins)
  • Q&A, Closing notes (10 mins)

Key Questions

This is an introductory workshop to machine learning. We are focused on helping you to not only learn through making but also start to reflect through making. The following questions are here to help focus the learning and reflecting.

  • "Machine learning is programming with __________ rather than ____________." - Kyle McDonald
  • Why are data central to machine learning?
  • What do we mean by "good" data vs. "bad" data? Why does this matter for machine learning?
  • What are the implications of all the advancements and increasing availability in machine learning tools and capabilities?
  • What is the role of art and design in machine learning today?

Slides

  • Link to slides: Slide presentation with introductions, overview of terms, project highlights and more.

Introduction to Machine Learning


The AI/ML universe

AI/ML isn't just about advancements in technology, it is about a broader universe of things that are coming together (or diverging). These include changes (or lackthereof) in governance, cultural and social interactions with, awareness of, and attitudes, and feelings towards these new technologies, the development of new tools and communities around and against AI/ML, expansion of AI/ML education, and use of AI/ML in art and design or other domains outside "traditional AI/ML" spaces (e.g. engineering and computer science).

We increasingly interact with machine learning systems every day.

Our lives -- e.g. how we access content, who gets access to services, and how we get from A to B -- are being shaped by the data we produce and the people using our data.

These increasingly algorithmically and data driven interactions reflect an assemblage of social and technical forces that almost always benefit some and harm others.

It is worth thinking about how the following factors affect or are affected by changes in AI/ML.

  • Research
  • Policies
  • Environment
  • Tools
  • Communities
  • Education
  • Art/Design

Terms

Collected from Yining's Week 1 of Machine Learning for the Web

  • Artificial Intelligence:
    • AI is the science of making things smart:
    • Human Intelligence Exhibited by Machines
    • A broad term for getting computers to perform human tasks
    • General AI: fabulous machines that have all our senses (maybe even more), all our reason, and think just like we do.
    • Most of AI today is Narrow AI systems that can only do one (or a few) defined things as well or better than humans.
  • Machine Learning:
    • One of the Approaches to Achieve Artificial Intelligence
    • A field of computer science that uses statistical techniques to give computer systems the ability to "learn" with data, without being explicitly programmed.
    • LOTS OF DATA
    • Types of machine learning learning styles:
      • Supervised Learning:
        • Training data is labeled
      • Unsupervised Learning:
        • The machine learn from an unlabelled data set.
        • Unsupervised learning can be grouped into clustering and dimensionality reduction.
      • Reinforcement Learning:
        • The machine learn by trial-and-error through reward or punishment.
    • Types of machine learning outputs:
      • Regression: Predict continuous values (price of a stock)
      • Classification: Predict a class/label (cat or dog?)
      • Clustering: Groupings based on potential relationships in the data - Most similar other examples (Amazon related products recommendation)
      • Sequence Prediction: Predict the next value/number/word/sentence in a sequence - e.g. the next chord in a melody
    • Types of machine learning use cases:
      • Object Recognition
      • Speech and Sound Detection
      • Natural Language Processing
      • Prediction
      • Translation
  • Deep Learning:
    • A technique for implementing Machine Learning
    • Deep learning - Uses multi-layered artificial neural networks to deliver state-of-the-art accuracy.
    • Can automatically learn representations/features from data such as images, video or text.

The Machine Learning Process:

The process of machine learning is messy. Like most data related projects, 80% of your time or more will be spent cleaning and preparing data for use. The rest of the time will be the actual "making" of your project. The machine learning process might be broadly (over)simplified into a handful of steps which includes: Data collection, data preparation, Training a machine learning model, and using the ML model in context.

  • Data collection
    • E.g. an artist writes a script to automatically download all images of cats, dogs, and badgers from the internet.
  • Data preparation
    • E.g. the artist uses a crowd tasking service to 1. confirm that the image is either a cat, dog or badger, 2. Tag the image with the animal type, 3. Crop the image of the animal to a square aspect ratio. The artist then resizes all the images to 64x64px
  • Training a machine learning model
    • E.g. the artist: 1. Uses a popular machine learning framework called Tensorflow in python to create a neural network. To create a neural network the artist, writes code that specifies image processing and data crunching methods that aim to pull out unique features for each animal type in the given images. The artist then 2. Iteratively refines and tests the model to achieve the highest accuracy.
  • Using the ML model in context...and awesome art projects
    • E.g. the artist opensources this new pre-trained dog/cat/badger model. This comes just in time since the ASPCA in NYC has been in great need of a method to differentiate between the animals in their shelters. The artist is hired to create a web application that uses this model to label their shelter animals “cat”, or “dog” or “badger”.

Introduction to ml5.js


ml5 Resources

ml5 in 3 steps

ml5 in 3 steps


Coding Session


Coding Session Part 1: Code-a-long Session

During this code-a-long session, we will walk through several features of ml5.js. Specifically we will walk through the following features: We will start with a blank ml5 boilerplate on the p5 web editor for these examples.

  • Pretrained models:
    • Example: image classifier -- classifying an image
    • Example: PoseNet -- detecting and estimating features of a human pose
    • Example: FaceApi -- detecting and estimating features of a human face
    • Example: Sentiment -- estimating the sentiment (0: negative, 1: positive) of text content
  • Training your own model:
    • Example: Neural Network - using existing data to train a classifier from a JSON file
    • Example: Neural Network - using existing data to train a regression from a CSV file
    • Example: Neural Network - training & interacting with a simple classifier with data produced in real-time

Coding Session Part 2: Build an interactive ML application

In this short 45 min session, select a pre-trained model (e.g. PoseNet) or build your own custom neural network to make a simple interactive web application you can share with your classmates. Team up in groups of 2 or more.

→ Use this ml5 boilerplate on the p5 web editor to get started. OR feel free use your favorite text editor and development environment.





References


Online Learning Materials, Courses, and Workshops

Presentation References (see slides)

Project References

Physical computing examples with ml5

Transfer Learning


Tools


Friendlier ML tools

  • ml5.js
  • RunwayMl - Machine learning for creators. "Bring the power of artificial intelligence to your creative projects with an intuitive and simple visual interface. Start exploring new ways of creating today."
  • Google's teachable machine - do transfer learning in the browser with google's teachable machine.

ML APIs for Python and javascript (advanced)

  • Tensorflow for python
  • Tensorflowjs - TensorFlow.js is a library for developing and training ML models in JavaScript, and deploying in browser or on Node.js
  • Keras - Keras: The Python Deep Learning library

Cloud computing (advanced)

  • Google co-lab -
  • Spell.run - "The most flexible and powerful end-to-end platform for ML and deep learning engineering."
  • Paperspace - "The first cloud built for the future. Powering next-generation applications and cloud ML/AI pipelines."



Additional Acknowledgements

  • Openmoji - "Open-source emojis for designers, developers and everyone else!"

About

A friendly introduction to machine learning and ml5.js - workshop