_Lil dose of reality…_the model I trained yesterday probably won’t be very useful since it doesn’t account for how the metaballs move around in space. I’ll need to build a more complex scene where metaballs are driven through space procedurally with something like Brownian motion and each metaball has a sphere attached to its position that follows its scale in a similar way to the scene I built yesterday. Instead of reading a single pixel at a constant location’s value, I’ll determine which pixels’ values to read by doing a world-to-screen position transformations on each of the scaling spheres’ locations. To counter false positive readings when metaballs are nearby each other, I’ll either need to make each scaling sphere a different color, or use a different camera for each sphere. Although the second method probably consumes more memory, this is simply a testing scene, and by culling out all irrelevant sphere’s I’ll be able to ensure each scaling sphere’s size is accurate to the exact metaball it’s paired to.
Constructing that scene seems easy enough, but the real question is whether I’ll be able to train a model from the data I collect in it. In my previous iteration, the model used six inputs (metaball radii) to predict one output (combined metaball radius). This new setup is much more complex. We’d still have to input the radius of each metaball, but now also factor its position. I could input a Vector3 for each position, which would result in a total of 12 inputs—6 for radius and 6 (3x1) for position. Alternatively, I could input the distance of each metaball from every other one, which would constitute 6! (21) inputs plus the 6 radius inputs for a total of 27 inputs. Or, I could combine the radius and position data into a Vector4, and send that to the model for a total of only 6 (4x1) inputs. I’m not sure which method will leave me with better predictions (or furthermore, which will be faster). In the online ML course I’m taking, I’ve only trained models with a single input data type. I’m not sure how stuff works when there are multiple input types. To start off then, it looks like method #3 is the approach I’ll take. Anyways, the model will take in the data and return each metaball’s radius for a total of 6 outputs.
Once (if) I have a working model, I’ll then begin to figure out the ML Agents/Sentis framework in Unity. No need to get ahead of myself! I sure hope this works.