TSANCHEZ'S BLOG

SAC: Post Mortem

Being my first console game, SAC posed some interesting challenges that I’d never encountered in PC development. It also was an interesting challenge to work with a team of people on an established codebase. While there are benifits to having a fully working system to jump into, it is also hard to have to spend time hunting through new code for functionality that exists in there somewhere.

The PSP is not the robust piece of hardware you have infront of you as your PC. The compiler, OS, and hardware don’t work together to tell you that you are being stupid in the same way it catches PC programmers. Someone ejects the disk, you have to handle it. Someone powers off in the middle of your level, you have to handle it. Thankfully the engine team at HIG took care of that stuff. But it still meant interesting crashes.

From that I found my first experience with the compiler’s MAP files. Something that people just don’t seem to teach in school is proper debugging. The MAP file that a compiler generates is like the regular debugging symbols your debugger uses. But, when your game goes to test, you get reports of it crashing on some remote machine that you can’t debug. The human readable MAP lets you associate your “blue screen of death” information back to your code. You get info back from your testers that indicates

“module 0x0EA9320 crashed at $PC=0x0EAABBE0 $RA=0xEAABC91 bus error reading 0x0000000C”

You know you have SOME pointer derefrence to look for, but where? Simple, you know you crashed on the instruction located at

PC – Module Base : 0x0EAABBE0 – 0x0EA93210

You look up that address in your MAP, and you can find the function in question. If you have the same disk and symbols locally, you can use your debugger to jump directly to the code at:

Current Module Base + Test PC – Test Module Base

At this point, I’m really glad I knew my MIPS assembly instructions, because tracing back information from assembly is something you just have to do when you get crashes on release disks. Expecially since testers can return cases that you won’t be able to readly reproduce in house, but the testers can easily reproduce there (mostly because they know exactly what they did, and teaching that in words isn’t always easy. Timing can be a huge factor too, thus complicating the debug process). Being able to jump into the map and local symbols to read what the assembly did can make finding the bugs alot easier. Knowing what the code should have translated as also means that it is entirely posible to find the occasional compiler bug (we ran into atleast 2 that i know of).

Developing for an embeded system adds a lot more restrictions to what you can do over a more open setup like a home PC. At home I never worried about memory, filesystem space, or data formats. On the PSP our engine packed together everything very tightly in build stages before getting onto the PSP. Files get converted to match in-memory structures so they can just be used once loaded without any parsing. Files get packed into tight archives (something I did do in my home projects). Memory gets budgeted between gameplay, code, art and sound. Files get layed out on disk for fast access while streaming in voice overs. And lots of meta-data has to be layed out on the hard disk so the build tools can find everything they need to pack together the archives for each level.

Finally, team development is way more fun than solo development. It was a really nice experience having set goals to work to, and people to help out with all the parts. Other programmers make systems and logic that you can use. Artists produce level art, props, characters, and animations. Being together with them means quick iteration of what an animation should do, and how well your AI is using that animation to brodcast its intentions to the player. Its also quite exciting to take take over a very complex system like the Giant Clank levels, and be able to turn it into something new and exciting to everyone as a credits sequence.

Copyright © 2002-2019 Travis Sanchez. All rights reserved.