I’m gonna be showcasing a body tracking game for the upcoming FilmGate Interactive 12 festival next week. To be clear, I haven’t built the game yet. I know that I’m cutting things close, but my confidence in Claude has never been higher, especially since Anthropic now offers 1M context window at regular pricing. I recently signed up for the Claude Max plan and it’s been refreshing no longer having to sweat token use.
Yesterday, a friend of mine came over and I helped him set up Claude Code on his machine, and in doing so, I realized that my agent scaffolding needed some updating. I had been directing Claude via my global CLAUDE.md to use MCPorter to convert any MCP servers I needed into CLI tools with corresponding skill files. I was ahead of the game thanks to a video I saw from AI Jason, but now it seems the rest of the industry has caught up. MCPs are a thing of the past and CLI integration through skills is the what the cool kids use. Creators of several of the MCPs I’d CLI’d with MCPorter, such as Context7, are now offering CLI modes for their agent integrations, and I switched over accordingly. Some other CLI skill integrations I set up include ast-grep, GitHub, and the one I’m most excited about, Unity. That last one took a bit of shimmying to get installed, but Claude was up for the task—it’s quite satisfying getting Claude to install extensions for itself.
Unity released a Claude Code plugin for its App UI package that I figured I could use to build out my game’s UI. I was desperately trying to avoid using the newest Unity 6.3 for my game because I remembered the ZED Unity plugin being a bit jittery when I last played with it in early January. I had to manually install the latest alpha version of the App UI by dropping its source code in my /Packages directory since that version wasn’t available from the Unity 2022.3 package manager.
My main goal for today, however, was getting the ZED SDK up and running and testing out its body tracking capabilities in Unity. Man, oh man, was that a ride. First of all, as I highlighted before, I’d already worked with the ZED Unity Plugin on a different computer. Therefore, I had an idea of what to install and how to get going. What I didn’t anticipate, however, was forgetting the single most important rule when debugging constant crashes when trying to run basic ZED sample scenes: TRY A DIFFERENT USB PORT!!!
…That would have saved me about 4 hours of head-scratching debugging. On the bright side, Claude managed to find an important bug within the ZED SDK that caused the ZED Diagnostic Tool to crash instead of reporting poor USB bandwidth. With its assistance, I submitted a bug report to the developers on GitHub (shoutout to the GitHub CLI). I ended up needing to use my laptop’s Thunderbolt port. Luckily, I had a 5GB speed USB A Female to USB C Male adapter handy which let me continue to use my long cable with my ZED camera. I know it’s ridiculous that I didn’t think to try another port, but there was literally an update pushed today to the ZED SDK that supposedly fixed the exact crash I was experiencing.
Because of that, I seriously thought my issue was with the ZED SDK or ZED Unity plugin, and I ended up cycling through 3 different SDK/Plugin version combinations before finally surmising my computer had to be the problem.
Unfortunately, even after solving this most basic of hiccups, Unity crashed whenever I stopped one of the ZED demo scenes. Claude diagnosed the issue within the Unity plugin SDK and submitted another thorough bug report! I’m gonna end up with a bunch of cool GitHub badges I totally don’t deserve. I’ll wear them proudly.
So, after Claude fixed bugs in both the ZED SDK and Unity plugin, I was finally able to get body tracking working. I tested tracking in a fairly low-lit room and it worked nearly flawlessly, although the oversized straw hat I was wearing messed up head tracking. ZED has a model with 38 bones that includes all fingers—this won’t be needed for the game I’m building this week, but it certainly bodes well for the Interactive Energy Ball.