Project description
Bringing together a 15+ year old technology platform with today's latest ML engine. A simple and basic flash card game developed in ABAP and Javascript powered by SAP NetWeaver and Teachable Machine by Google.
Here's the link to a YouTube video describing the game and steps to set it up - YouTube
Motivation
After reading a blog and watching a video by Dale Markowitz Build a Machine Learning Model in 10 Minutes on how easy it is to now create and train your own model with Teachable Machine powered by TensorFlow.js, I was hoping to get some hands on with it. Also, my two kids were the motivation and inspiration in creating this flash card game and as an added bonus, they seemed to like it! Can't say that I am proud of the final game UI/UX design and aesthetics but since I am not a front-end/UI/UX designer, this'll do...
Features
-
A setup transaction (ZZATMSETUP) to create new players, games and questions with lots of customizing options such as setting a profile picture, game background music, game sound effects, setting delay between questions and set up different ML models for different games.
-
Sample model, images and sound effects included in BSP application - see folder "demo_assets" in BSP application and screenshot in the "Setup" section
NOTE: The attached sample ML model was trained with only my voice, it might not work for you - in my test with my kids, it detected their response with only about 60% accuracy. It'd be best if you create and re-train a model with your own voice, it's extremely easy and sort of the main motivation to create this game.
-
Debug mode. If option is checked during player setup, the ML model class labels and scores will be displayed at the bottom of the screen during gameplay
Pre-requisites
- A running SAP NetWeaver AS ABAP 752 server with BSP configured - I used the latest currently available (Nov 2020) Developer Edition server to develop this. Might be compatible with lower ABAP versions but no guarantees.
- Clone this repository to your ABAP server. It should contain just a single local package $ZZATM with the required BSP application and all the data dictionary objects.
- Internet connection when playing the game - because of required JS/CSS libraries
- A microphone so that your response can be captured and matched with the ML model
- A modern browser - Yes, anything but IE please.
- A ML model trained in Teachable Machine (sample model found in https://vhcalnplci.dummy.nodomain:44300/sap/bc/bsp/sap/zzatm/demo_assets/model/)
- Images that matches the "answers" in the ML model (sample images found in https://vhcalnplci.dummy.nodomain:44300/sap/bc/bsp/sap/zzatm/demo_assets/images/)
Setup
- Train and test your own ML audio model on Teachable Machine - It's really FUN!
- Export the model : Either download the model (ZIP file) or get the shareable link on Teachable Machine.
- If you've downloaded the model locally, unzip your model and upload 'metadata.json', 'model.json' and 'weights.bin' to ABAP server. I have created a 'models' folder under the BSP application ZZATM.
- Run transaction ZZATMSETUP to create a new player, game and questions.
Player
- Player name - Up to 40 characters is supported but avoid using any special characters such as &,%,*,$ etc...
- Profile picture - URL to a small avatar picture. I've included four sample images in the demo_assets folder.
- Debug - If this option is checked, during gameplay, the class labels and scores will be displayed below the screen.
- Sound settings - configure different background music during gameplay and sound effects for when getting a correct answer and when the game ends.
Games
- Game name - Up to 40 characters is supported but avoid using any special characters such as &,%,*,$ etc...
- Game type - Only one option is available now but this gives the flexibility to create more games and route them to different pages.
- Game model - The full and complete URL (https) to the ML model that contains model.json, metadata.json and weights.bin
- Match probability - The minimum score match to consider that it's a correct answer. When the ML model is 100% sure, the score is 1.00. Recommended config could be something like 0.80 (easy difficulty - not really accurate), 0.90 (medium difficulty) or 1.00 (hard difficulty - highly accurate match). You'll just need to tweak this and it also depends on your ML model as well.
- Time to answer - The time in seconds to display the flash card image and listen for an answer. Rcommended config can be something like 10 seconds (easy difficulty) - 5 seconds (medium difficulty) or 3 seconds (hard difficulty).
Questions
- Array index - this is a little tricky - you need to match the wordLabels array in the metadata.json file of your model. Typically, "Background noise" is the first entry so you can ignore array[0], and you just need to note what the other class labels are and their array index.
- Image URL - The flash card image to display when showing this question and remember to match the array index correctly.
See array index 4,5,6 and look at the matching answer - 1, 3,2 - so, don't always assume that it is in order.
Sample - demo_assets
How to play?
-
Launch the BSP application on a browser - eg: https://vhcalnplci.dummy.nodomain:44300/sap/bc/bsp/sap/zzatm/index.html is the path on my ABAP developer edition server and if you've set up a player, you should see it.
-
Select a player
-
Select and start a game
-
Gameplay screen overview
Say the flash card that is displayed and if your answer matched, the question # chip at the bottom will turn green and a correct answer sound will play if you have it configured.
Credits
See 'Credits' page in application