I’d start this with an asterisk because it’s actually still Friday but past midnight, so I guess it’s really Saturday…
I’m reassuring myself that integrating the Video-Depth-Anything-Live (VDAL) library with Unity will be just as easy as integrating Sports2D was. Truthfully, I’m a little nervous, because can vibe-coding really be that easy? As a fulltime dev, I already know the answer to that question. Vibe coding works wonders at the start of a project, but seriously loses its effectiveness as the project expands from the scope originally provided to the LLM in order to create it. Knowing that with each additional integration, the feasibility of vibe-coding diminishes, I hereby reserve the right to my apprehension. Still, a soldier must do what soldiers do and march onward.
The key to keeping the LLM primed and ready to help is to break the process into small steps and anticipate potential obstacles ahead of time. Here are some thoughts:
- Test VDAL’s python scripts in more detail. The author provided a screenshot of a point cloud in the repo. I want to be able to visualize my data inside Unity in a similar manner, so I need to learn how the author did this. I’m getting a tingly feeling that this might be a job for a compute shader once I get this into Unity.
- Start with a blank Unity scene for testing. It’s imperative I don’t add this library to my existing Sports2D Unity integration until I’ve already gotten it working in a standalone scene. That being said, I’d want to send data from VDAL to Unity in the same way I send data from Sports2D, which is via TCP/IP. so I do plan to prime my LLM’s context using my Sports2D-Unity-Integration repo to have things built in a similar fashion.
- I need to figure out how to handle both libraries at once. Each library uses the camera, but I’ve noticed that in Windows, only one app can use the camera at a time. Therefore, I may need to create a bridge script to initialize the camera and pass frame data to each library indirectly. It may be best to try this outside of Unity first, with python. On that note, I might want to have this master script send the data to Unity for both libraries together, since I think sending data on two separate ports could be problematic and also potentially induce latency issues.
- Since both libraries rely on python, I should check to see if I can unify their dependencies because it would be nice to use a single python instance and save memory.
Finally, the most important key–getting some good sleep. I’m signing off now till today afternoon.