June learnings – Audio classification

Week of 6/21/21 – 6/27/21

• API
• Webservices
• REST / RESTful
• REST/RESTful API
• API Endpoints
• HTTP verbs
• SDK vs API
• Using Python SDK for Misty II robot

References

1. https://www.mulesoft.com/resources/api/what-is-an-api – What is API?
2. https://dev.socrata.com/docs/endpoints.html – API Endpoints
3. https://rapidapi.com/blog/url-vs-url/ – URI/URL
4. https://rapidapi.com/blog/api-vs-web-service/ – API vs web services
5. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods – HTTP verbs
6. https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html – HTTP verbs
7. https://www.c-sharpcorner.com/UploadFile/47fc0a/working-with-http-verbs/ – HTTP verbs
8. https://dev.socrata.com/docs/verbs.html – HTTP verbs
9. https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm – REST explained – original thesis
10. https://blog.ndepend.com/rest-vs-restful/ – RESTful
11. https://nordicapis.com/what-is-the-difference-between-an-api-and-an-sdk/ – SDK vs API
12. Using Python with Anaconda and Visual Studio Code (Windows)

Week of 6/21/21 – 6/27/21

• Tensorflow Lite for Audio Classification

https://www.tensorflow.org/lite/api_docs/python/tflite_model_maker/audio_classifier
https://codelabs.developers.google.com/codelabs/tflite-audio-classification-custom-model-android#0
https://blog.tensorflow.org/2021/03/transfer-learning-for-audio-data-with-yamnet.html
https://github.com/tensorflow/models/tree/master/research/audioset/yamnet
https://www.tensorflow.org/tutorials/audio/transfer_learning_audio
https://tfhub.dev/google/lite-model/yamnet/classification/tflite/1
https://github.com/tensorflow/examples/blob/master/tensorflow_examples/lite/model_maker/demo/audio_classification_demo.py
https://github.com/tensorflow/examples/tree/master/lite/examples/sound_classification/android
https://www.tensorflow.org/tutorials/audio/simple_audio
https://colab.research.google.com/github/tensorflow/tfjs-models/blob/master/speech-commands/training/browser-fft/training_custom_audio_model_in_python.ipynb

• What is MFCC?
  • https://www.kaggle.com/yassinealouini/what-is-mfcc

Week of 6/14/21 – 6/20/21

• PANNs: Pretrained Audio Neural Networks
• How to load Machine Learning models on Misty
• Misty Robot
  • Can be controlled through Skill Runner, Command Center, API Explorer
  • Misty JavaScript for Visual Studio Code
  • Misty II Project Directory, Part 1: Community Skills
    • moveToSound (*)
    • Hourly chime
    • twilio SMS/MMS
    • translate
  • Misty II Project Directory, Part 2: Sample Code
    • takePicture
    • audioLocalization (*)
    • turnToSound (*)
    • REST API examples
    • Exploring Computer Vision
  • Misty II Project Directory, Part 3: Tools, Sandboxes, & Other Inventions
    • Moving Map
    • misty-interact : Misty REST API
    • rerobots Misty II Sandbox
    • Teleop
  • Audio Localization – more details on Misty
  • Misty Hears you! What is next? – this explains moveToSound further
  • Difference between a skill and application
  • Python SDK

References

1. Taking a picture using Misty
2.  Misty saves audio recordings to her local storage as .wav files.
3. Sending External Requests

Week of 5/31/21 – 6/6/21

• Using TensorflowLite for Audio Classification: References 1-5
• (paper) Tiny Transformers for Environmental Sound Classification at the Edge

References

1. Build sound classification models for mobile apps with Teachable Machine and TFLite
2. Simple audio recognition: Recognizing keywords
3. Audio classification
4. Transfer Learning for the Audio Domain with TensorFlow Lite Model Maker
5. Module: tflite_model_maker.audio_classifier
6. Tiny Transformers for Environmental Sound Classification at the Edge
7. PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
8. Converting the YAMNet audio detection model to TensorFlow Lite