About ESP (2008 - 2009)
ESP was a start-up group inside ACES studio and the ESP product grew out of the Flight Simulator platform: A training and simulation environment which modeled the world in great detail and allowed users to modify and expand it. The product was quite complex and consisted of a variety of tools and plug-ins for different users ranging from artists to developers.
I was officially a UX Designer, but after years of being a project manager, I can't just push the pixels; I look at the bigger picture of both the product and the development process (read more about my process). I primarily owned the end-to-end experience, which you can read about here, but had done some initial brainstorms on the simulation environment before taking over the end-to-end.
About the 3D Simulation Environment
The simulation environment (a.k.a. "Scenario Editor") was a way for people to create flight- and ground-based scenarios including both creating the environment and setting actions. The package included the world that you see in Flight Simulator and tools to modify that world by adding more detail to the terrain and creating paths and AI. So the user could select an area of the Nevada desert, place trees in specific locations, and add a military installation including all of the relevant buildings and roads. They could then add a variety of vehicles which would either follow a specific route or respond to the vehicles around them, and create a path that the trainee would need to follow. The product was geared toward training, so a typical scenario could be "Fly from point A to point B, lower the winch in the helicopter and pick up the supplies, then fly back to point A within a specified time frame."
The audience for the Scenario Editor was not super-3D guys; it was trainers and possibly commanders. The person who was creating the scenarios was an expert in knowing what kinds of situations a pilot or driver would need to respond to, not in creating 3D objects or navigating in 3D environments. The Scenario Editor therefore needed to reflect both their needs and their knowledge base.
The Simulator UX
Because the product revolved around the placement of objects in space (rather than the creation of those 3D objects) and because the audience was not familiar with working in typical 3D environments, my approach revolved around a map and instrument panels. Location (XY) and altitude (Z) would be addressed separately, as would bearing (direction) and pitch and yaw. The goal was to make the simulated world resemble the real world, where each of these is measured and/or recorded by different instruments.
Also, as our users might be working on laptops, I could not assume multimon (or even a mouse), so screen real estate was an issue. Therefore I created something that would work on a small screen, but would allow users to take advantage of larger screens by allowing them to move and resize the different panels.
Scenario Creation Walkthrough
The environment is designed to allow a trainer to design a simulation by using familiar commands and UI elements: All they would ever need to do is type, drag-and-drop, click, and right-click. For instance, to place an object in 3D space, they would simply need to select the object from a menu, drag and drop it to the desired location, and then select the altitude of that location in a different menu. They could select the POV view to see what the trainee would see, and modify the location, altitude, pitch and yaw, etc. from that perspective allowing them to control what the trainee would and would not see. The video below is a walkthrough of what this experience would be.
Alas, the studio was closed and my work never shipped.