[This piece is dedicated to the engineers, the idea men, and the business people behind the Nintendo 64 project, first revealed as Project Reality. I am planning to continually update the article as I find new information to paint a greater picture of Nintendo’s most interesting console. If you were involved with Project Reality, please feel free to contact me so I can add your memories to the article.]
[Editor’s note: Quotes in past tense are taken from interviews from other articles. Quotes in present tense are from my own personal interviews. This piece was originally published on Medium.com on June 23, 2016 then on The History of How We Play on June 23, 2018.]
Despite their current reputation, Nintendo were once on the bleeding edge of gaming tech.
In 1991, the corporate giant had just come off of the best years of its life and was planning to take the U.S by storm with their fresh new Super Nintendo Entertainment System. Sega was fast proving itself a worthy opponent and so both companies were taking to the road to instill brand loyalty in consumers and retailers. At trade shows like the Consumer Electronics Show (CES), the meager Sega budget would compete against Nintendo’s (then) overwhelming presence and support. Nintendo wanted to add a technological flair to their exhibits and they would find their answer in the hottest tech of the day: Virtual Reality.
To Be, Virtually
The early 90s was abuzz with the concept of Virtual Reality. VPL Research, founded by Jaron Lanier, had created the first Virtual Reality products with Thomas Zimmerman’s Data Glove which could track hand and finger positions. The exact definition of Virtual Reality as a concept varied between companies that used the term, but in VPL’s case it was a matter of transferring physical actions into a virtual environment. They created 3D representations of hands on the computer screen that would mirror the actions performed with the glove. It was the beginning of virtual body projection.
While the front end of the DataGlove programs was an Apple Macintosh computer, the back end was something much more powerful: A Silicon Graphics workstation. Silicon Graphics was founded in 1981 by Stanford Professor James Clark and Abbey Silverstone, along with a group of Stanford graduates to create high-powered, graphically-based computers. The technology was built off a processor that Clark and Mark Hannah had built called the Geometry Engine which would power their first IRIS 1000 series workstations. Workstations were a new class of computer that began to fill in a high-end gap as room-sized mainframes went out of fashion, but personal computers were unable to fulfill because of the processing involved. Through Clark’s focus solely on graphical technology, he was able to ride the wave of both pre-rendered and real-time 3D that took off at the dawn of Virtual Reality.
VPL and Silicon Graphics were in a symbiotic relationship which pushed the VR Market. “Oh yeah. VPL uses Silicon Graphics computers,” said Marc Hannah. “We’re working with them to bring the cost down.” VPL’s current full-immersion set-up was a system called Ready Built for 2 (RB2) which combined their DataGlove or DataSuit technology, two Silicon Graphics workstations, and one of the first virtual reality displays called the EyePhone. Despite costing over $350,000, by 1991 they had sold 400 RB2 sets and bolstered the Virtual Reality field into big business.
Other companies were quick to imitate VPL’s style of Virtual Reality, focusing on limb movement and looking to merely enhance the sensation of handling objects in a virtual space. Rick Lazzarini thought differently. A well-renowned animatronics and puppetry expert, Lazzarini and his company The Character Shop had provided effects seen in blockbusters like Spaceballs and Ghostbusters II. Lazzarini was very familiar with the practical uses of computers in the creative field; the Alien Queen from Aliens having been a product of computer-aided design. Therefore he relished the idea of transferring actors into the digital realm.
Lazzarini had designed a remote animatronic system called the Waldo, named after a novella by Robert Heinlein. With it, a single operator could control many complex movements for puppets while remaining off screen. He adapted this into a virtual space by creating floating digital puppets with rudimentary mouth movements. Jim Henson demonstrated this technology on The Jim Henson Hour with a character called — you guessed it — Waldo, with graphics provided by Pacific Data Images.
Lazzarini really wanted to capture an actor’s emotion with this system though. He already had the Facial Waldo for animatronics and he could achieve the same thing in digital form by using reflective surfaces glued to an actor’s face. It worked, but it needed a proper demonstration.
SimGraphics Engineering, founded by Steve Tice, had the answer. The company had provided a development workbench for VPL’s DataGlove, giving them a strong presence in the emerging Virtual Reality field. Specifically they were focused on live motion capture, having demonstrated the first ever full-body real-time capture in a creation called Silver Suzy at the 1991 SIGGRAPH conference. One day they received a call from Dr. Dave Warner at Loma Linda University Hospital who was interested in utilizing their performance animation in an educational setting.
The demo, using a head-mounted version of Lazzarini’s Face Waldo, was to be a teaching supplement for use in pediatric psychology in order help ailing children feel more comfortable about procedures like facial surgery. Steven Greenstein would pilot the demonstration with a character constructed by Mark Sorrell & Associates called Eggwardo. The digital puppet ran on a Silicon Graphics 420 VGX workstation in an environment called the Performance Animation System. Eggwardo had features like an extendable nose, the ability to fly around the screen using Simgraphics’ Flying Mouse, and be projected onto moving backgrounds.
The project was at least a modest success, with repeat performances discussed. However, commercial offers proved much more tempting. One of the first to come knocking was Nintendo of America, who were looking to bolster their trade show presence with a new way to promote their upcoming Super Nintendo games. The Eggwardo concept was very attractive, and there were immediate parallels with their company mascot, Mario. They would need an actor to pilot the Mario head though, and luckily they had hired just the guy for the job.
Charles Martinet had shown up uninvited for an audition in 1990 where Nintendo was looking for a voice for the Italian plumber to interact with people at various trade show events. His performance was so energetic and captivating that the tape of the audition was the only one sent back to NOA headquarters in Redmont, Washington. Mario now had a voice and Nintendo now had a way to portray him outside of people in suits (or Bob Hoskins).
“… [T]he original rig involved form-fitting helmets. So the first CES may have been 1991.” says Stevie Coyle, acting buddy of Martinet who is seen donning the Face Waldo headset in the majority of the above footage. “It was determined early that he’d need someone to relieve him every few hours during those long, long CES days and he very graciously asked me.”
Even if the debut had been a year earlier, MIRT (Mario in Real Time) was the talk of the ’92 Winter CES and beyond. Demonstrating a wonderful sense of comedy, Martinet captivated the masses of people who went by the Nintendo booth. In these busy environments, Martinet would develop voices for characters such as Luigi, Wario, and even a hilarious stint as Donkey Kong to promote upcoming games. SimGraphics and Nintendo became the spokespeople for performance animation, with the former making alliances with Iwerks Entertainment and PLS Staging to create the enhanced VActor system for trade shows as well as television broadcasts with characters like Ratz on the BBC. Nintendo would later utilize the Mario head in educational titles by Interplay Entertainment called Mario’s FUNdamentals.
“Home game systems are next, predicts Steve Glenn, vice president of new business development for SimGraphics. ‘It’s going to happen in two or three years.’
“The cost? Perhaps as little as $300-$400, Glenn estimates.”
He was right, but perhaps not the way anybody expected.
Chasing the Beam
Silicon Graphics (rebranded as SGI) had been making major strides in their niche (but rapidly expanding) market. Applications for Virtual Reality had stalled in the scientific field, but the use of computer graphics in major motion pictures had ramped up. Director James Cameron’s films The Abyss and Terminator 2 both made extensive use of digital actors powered by SGI workstations, utilizing new architecture designed by MIPS Technologies. SGI acquired MIPS in 1992, just as the former was about to receive an enormous publicity boost for powering the graphics seen in Steven Spielberg’s Jurassic Park.
None of this existed in a vacuum however. Jim Clark had been working to secure the company’s future by bringing them out of multi-thousand dollar hardware into parts-providing for consumer products. First on this path was Time-Warner, the media conglomerate who were planning to launch an interactive television network and wanted SGI to provide the chips with Microsoft supplying the software. The Full Service Network, as it would be called, stoked Clark’s interest in connectivity on a mass scale and sent him searching for more opportunities in the consumer realm. Unbeknownst to him, there was a project at MIPS which would provide an in to that market very soon.
Assigned to the Warner set top box project was a newly hired recruit from Sun Microsystems named Tim Van Hook. Van Hook had been involved with 3D graphics since 1977 and describes the potential of these systems as “a camera of the mind”.
“I started spec’ing [sic] out a media coprocessor (MCP) instruction set extension for MIPS similar to what I did in SPARC VIS [instruction set] at Sun with Les Kohn.” says Van Hook. In the process, he developed an architecture called the Multimedia Engine which drew upon the Reality Engine framework built to run on $100,000 SGI machines at a huge cost reduction. The Reality Engine had been providing some of SGI’s most stunning demos to date and suddenly MIPS had a wowing product at a fraction of the price.
“In the RSP,” explains Van Hook. “I combined a variant of the integer vector coprocessor instruction set I was working on for the set-top, with my own implementation of the MIPS integer instructions (no room for FP [floating point], no interrupts/cache, addressing limited to scratchpad/code explicitly loaded by DMA [direct memory address]). The point was, a MIPS (extended) ISA [industry standard architecture], fully interlocked pipeline, compatible with (super/subset) assemblers or even HLL [high level programming languages], fully customizable data/geometry/audio per title.”
The feat was such that colleague George Zachary would refer to Van Hook as a “wizard”.
Prior to this though, Clark had been in the process of shopping around the new MIPS R4000 chip for consumer use, and he stopped by the company that seemed like it would be best suited to the tech and was also working in the interactive television field: The 3DO Company. However, he was rebuffed by CEO Trip Hawkins who was already well underway with his own technology. His next stop was at Sega of America to meet with Tom Kalinske. Kalinske was very impressed by the demonstration, but didn’t manage to get approval from Sega of Japan (Brandt, Richard L. “Nintendo Battles for its Life.” Upside Magazine October 1, 1995). Either it was a matter of the Sega engineers finding the chips inadequate and costly, or Sega President Hayao Nakayama’s insistence on having the chips be exclusive to Sega. The latter Clark could definitely not comply with, so he brought his business to Nintendo.
At the end of ’92 or the beginning of ’93, Jim Clark sat down with Nintendo President Hiroshi Yamauchi to discuss using the Silicon Graphics chipset in a Nintendo game machine. Nintendo’s prior hardware partnerships had been with Japanese companies Sharp and Ricoh, so this would be a new venture and potentially a chance for Nintendo to reclaim its lost market share in the US. The deal was struck and Nintendo’s engineers flew over to SGI Headquarters to get their first look at the new 3D graphics.
“The SGI VP I worked for asked me to spend a couple days explaining 3D graphics to visiting Nintendo [engineers],” says Van Hook. “After which they apparently drafted me to architect what would become N64.”
The deal could not have come at a better time for Nintendo. They had just launched their first polygonal 3D game, Star Fox, which utilized a special chip inside the Super Famicom cartridge called the FX Chip. Nintendo provided in the home what had only been seen in arcades up until that point, with both the public and the developers eager for more. The prospect of texture mapped 3D, which looked good even at low polygon counts, was an exciting prospect. Several members from the technical programming team at Argonaut Software stayed with Nintendo to work on new 3D titles, one being Stunt Race FX and the other being Star Fox 2. Shigeru Miyamoto, chief creative mind at Nintendo, would keep an eye on both projects, subtly priming himself for new horizons ahead.
Don’t Call it a Reality
On August 23rd, 1993 at the San Franscisco Mark Hopkin’s Hotel, Nintendo and SGI held a joint press conference. Jim Clark introduced the Mario head as well as a modified version of an existing Reality Engine tech demo which superimposed 2D Nintendo characters into a 3D streetview scene. Fittingly, the collaboration was dubbed “Project Reality” by SGI with the express goal to provide high level 3D graphics in the home for under $250 by the end of 1995.
To say that people were skeptical is putting it mildly. Nobody believed that Silicon Graphics could produce a consumer product at that price range (it was speculated that part of the price reduction was Nintendo’s ability to order in bulk). Nintendo were criticized for over promising, with immediate predictions of the project’s indefinite delay. A post on the early web three days following the event satirizes the press release with the “Nintendo Exodus”, boasting stunning features such as “10 128 bit processors” and “the Super Mario Universe 3 Game Pak(tm).” He was only a few years off.
Regardless, Nintendo was resolute in its claims, what few it definitively gave.They rode upon the VR wave with the promised “Reality Immersion Technology” and a true 64-bit processor, prompting a quote by NOA’s Howard Lincoln that they were able to “skip a generation”. “Nintendo’s ‘Project Reality’ dissolves the current limits of video play,” he said. “Causing the world to challenge its notion of what a video game can be.”
Nintendo also refused to follow in the steps of 3DO by not outwardly speaking to the device’s multimedia capabilities. This combined with the format wars (which we will return to shortly) led some pundits to believe that Nintendo had no chance of sustaining itself against the likes of 3DO and the Atari Jaguar. Nintendo was doomed to fail.
A company like Nintendo doesn’t pour multi-million dollar budgets into projects for nothing though, and the proof of SGI’s worth in the gaming world was already being shown. British developers seemed to have picked up on it very quickly with large companies like US Gold and Psygnosis using the expensive workstations to help storyboard their games. The most famous use of the stations came from Twycross-based Rareware and their game Donkey Kong Country. By creating highly detailed 3D models and then digitizing them into 2D images, Rare managed to display an aptitude for graphical prowess that no one had caught up to at the time. It’s also happened to be a great game.
With these new experiences in mind, Nintendo started assembling a group of publishers and developers which would grant them exclusive games for the platform. Nintendo had come under pressure for its exclusivity contracts in the past, but they had ultimately revoked many of their restrictions. Their first exclusive would come from Rareware in partnership with Williams Entertainment, the latter of which also announced that the Project Reality hardware would be headed to arcades before it arrived in the home and that both versions would be identical.
The last console that had attempted to have literal arcade-perfect ports was the Neo Geo AES, a home version of SNK’s arcade hardware with $200 game cartridges. Speculation about Nintendo’s software prices was about as prevalent as that of the actual hardware. Given how much Nintendo was pushing the revolutionary qualities of the device, would they hop onto new formats such as the CD? They had attempted to with the Super Nintendo, but that product failed to materialize, so maybe they had been saving it for Project Reality.
Contrary to popular belief, this decision wasn’t always set in stone. “There was lots of back and forth on […] whether to use cartridges or disk-based media.” says Richard Webb, team member #5 on Project Reality. “[Nintendo] were looking at removable optical or magnetic (for recording) disks, but decided cartridges were more secure and provided faster random access (other systems used CDs and had the dreaded “Loading….” screen all the time).” Webb also notes that, “As far as media goes, SGI was pretty neutral. I would have liked the idea of supporting reading CDs, maybe including the Video CD and Super-VCD formats which were popular in Japan and China.”
When Hiroshi Yamauchi revealed this fact in February of 1994, he added in a key note that Project Reality would “initially” only play cartridges, leaving the field open for addons in the future. Nintendo’s most consistent justification for this decision was that a CD-ROM was far too slow in loading data to produce 3D images of such quality (miles away, Ken Kutaragi laughed). Software developers and press not in the Nintendo camp were quick to point out the massive margins that Nintendo had always made on cartridges, which also granted them a significant amount of control over game manufacturers (Brandt, Richard L. “Nintendo Battles for its Life.” Upside Magazine October 1, 1995). Also worth noting is that while CD-ROMs were cheap, disk drives were not, which could raise a greater barrier for the hardware cost.
Whatever the reason, Nintendo continued courting companies to collaborate. This so-called “Dream Team” was specifically made up of North American and European-based developers and publishers for the U.S market launch. DMA Design, creator of Lemmings, signed on. Game publishers Acclaim, Virgin Interactive, and Gametek got on board before long. Though it seemed like a scant line-up, Nintendo was ramping up in-house software development like never before. All they needed was the finalized hardware.
Lets All Chip In
“From what I understood, SGI got the contract to build the N64 and put out feelers throughout the company, To engineers who had spent most of their careers working on Unix machines this seemed like an interesting project.” says Chip Burwell, who was an audio contractor on Project Reality. “Consequently, the team that was put together for Project Reality was made up of some of the best engineers at SGI.”
Tim Van Hook believes that mostly new hires worked on the project and that the seasoned engineers were actually skeptical about getting involved with the Nintendo machine. Still, Richard Webb noted the great collection of minds which blossomed as the project got underway. “Within the next 3–4 months the team grew to almost 50 folks, specializing in audio software, graphics software, CPU hardware design, graphics processor design (GPU), system (tools and OS),” he says. “Lots of great folks all working together really well. Definitely a highlight of my career.”
Through 1994, SGI would be showing off many of its Reality Engine demos when speaking about the Nintendo console. All the engineers at the company knew that sort of tech wasn’t going to fit in a $250 box. “I don’t remember how specific N64 performance targets were set.” says Van Hook. “I suspect I did what I usually did, make a best guess taking all the constraints into account (competition, feature set, cost, clock speed, chip density, external components, schedule, team, etc) and hope the errors/omissions would be balanced out by later optimizations.
“[M]ost of the major choices — the CPU, the memory, the fab partner for [the] RCP (which determined the speed/size/function of std cell library etc.) — weren’t technical but business decisions.”
Underneath these constraints though, the SGI team worked diligently to create both the hardware and development software for the device. The company established partnerships with Rambus for the RAM, Alias (later purchased by SGI) and Multigen Inc. for 3D development software, as well as Software Creations for sound tools. Nintendo would source out the chip manufacturing to computer company NEC and get yet more tools from Nichimen Graphics Incorporated.
Nintendo’s R&D3 team, headed by Genyo Takeda, never called for any major revisions. However, Webb did recall that, “… Nintendo justified spending a significant amount of engineering effort to reduce the […] pin-count [on the SGI chips]. The rumor was that each pin cost about 2 cents, so over the projected 50 Million units, each pin cost $1M. So it was worth one year of engineering time (maybe worth $300K) to reduce the pin-count by 1. There might have been some room to reduce the number of power and ground pins, but most of the pins HAVE to be there.”
Game development hardware and software were a different matter. SGI’s most powerful machine at the time was the Onyx workstation, running a new version of the Reality Engine called Reality Engine^2. Onyx systems were used as “the game simulator platform” as Webb explains, but not as an accurate baseline of the final chipset. “The game system simulators are very useful for developing game software prior to actual hardware being available,” he noted.
From the very first press release, SGI had touted their brand new “low end” SGI Indigo (Indy) machines as perfect for games development. Coincidentally, the power of those workstations would be fairly accurate to the final specifications of the machine.
“I don’t know much about what the developers actually did.” Webb admits. “Some of them (including Rare) came into our office during development to give us feedback on what they wanted to do or needed for debugging.”
Left without tools, developers could only plan and estimate the capabilities of the device. The first game said to be available for the Project Reality, Killer Instinct, didn’t use any hardware from Project Reality which would have made direct porting impossible. At Nintendo, Miyamoto had just started to put some of his basic ideas into code by July 1994, splitting his attention between the internal development team and Paradigm Simulation Inc., a flight simulation company who had been building their living off of Silicon Graphics workstations already (Nintendo Power January 1996).
“[Paradigm] were referred to Nintendo as a potential developer who really knew how to get the most out of the processors of the SGI systems.” says David Krueger, artist on what would become one of the system’s launch games. “We were never really aware of the final specs (performance wise) until mid-way through the development. We were working very closely with SGI, so much so that we were helping them flush out their own system development through our tests and research. By us pushing the limits of the system, there were adaptations that took place up until they had to finalize the dev boards.”
Some of the systems quirks were not so much in the way of raw processing power, but ways to circumvent the small storage space allotted by a cartridge. “For the textures,” says Webb, “Most developers used a compressed texture format which allowed them to store more textures on their cartridge and transfer them to main memory faster. This mode made the textures look a bit blurry. Higher quality textures could be used, but they would be slower and would result in less complex or detailed geometry.”
Even more infamous than the textures though was the added layer of anti-aliasing that gave the whole picture an overly blurry appearance. SGI had emphasized pixel quality as a main asset to Project Reality and the pixel sampling was incredibly accurate. For many though, it was difficult to discern the pixels in the smudge left behind by the edge smoothing. Webb notes that the process was actually quite complicated and not a measure to cover up graphical defects. “I think each pixel kept track of how much it got “covered” by each polygon (up to a precision of a quarter of a pixel), but that was different than how textures were processed.”
The look would be divisive, but just as seemingly strange was the decision to not have any dedicated sound hardware like the Super Nintendo had. Van Hook intended sharing the RSP [Role swap protocol] between graphics and audio processing as a cost reduction method. “I was surprised that game development apparently couldn’t manage distrust between the graphics teams and audio teams, in terms of real-time scheduling of shared resources.” he says. “It seemed like console audio people preferred exclusive use of a small DSP [digital signal processor] rather than a fixed time slice of more general powerful processor, and they were allowed do that, in contrast to the history and trend of computing and application.”
Webb, however, does not believe lacking a dedicated chip was a limiting factor. “Audio is MUCH easier to process than graphics,” he correctly notes. “So it made sense to do it in software. It also made the audio processing options almost limitless. If audio processing only consumes 5% of the CPU, then why not just do it in software? The quality and capabilities were already far beyond what other chips could do at the time. I don’t ever recall discussions about any audio co-processors being needed.”
For his part, Chip Burwell was a former Leland audio engineer who came on to review some of the software being used starting in December 1994. “[SGI] flew me [in] from San Diego, put me in a hotel and had me evaluate their software.” he says. “At the end of the three days they decided to make it a couple more weeks. Then a couple more months, then before you know it 18 months had passed. For me this was a transitional time. I went from being an audio guy to being a programmer who knew about graphics, memory management, optimization and game flow.”
Ultra Changes
On January 5th, 1995, Nintendo announced that the chip set for the newly named Nintendo Ultra 64 had been completed. It was not.
The preemptive announcement was speculated to be in response to the Japanese launch of both the PlayStation and Saturn more than a month earlier, but only four months later Nintendo made another announcement that the release and public unveiling of the console would be pushed back to November at the Space World event held for the wholesale distributors in Japan, the Shoshinkai. The North American release would be delayed until April of 1996.
The reasons for the delay have never been made entirely clear. Tim Van Hook says “I think the build volume and the launch title availability were among the issues.” The latter was certainly the opinion among the general audience, who felt that the software companies involved had failed to live up to the task. Richard Webb noted though that most of the game development didn’t begin until Spring and Summer, and that was well before the final chip set was actually done.
Former SGI technician Adrian Sfarti posited in ’95 that the company missed Nintendo’s performance target and the chips had to be redesigned, but this is far from definitive (Brandt, Richard L. “Nintendo Battles for its Life.” Upside Magazine October 1, 1995). It is worth noting that SGI was undergoing major shifts in its corporate structure at the time. Jim Clark left in January of ’94 to more actively pursue connectivity enterprises, leaving Presidents Edward McCracken and Tom Jermoluk to rally for Ultra 64. Whatever the circumstances for the delay though, “[The hardware] was mostly functional in Fall, 1995.” says Webb.
More pertinent to the consumer than any of this technological mumbo jumbo though was the actual physical item they would be buying. The first appearances of the Ultra 64 were fairly set in stone from the get go. Nintendo made a contrast with both their prior consoles and Sony by adopting a curved aesthetic more similar to the 3DO than anything else. For the first time since the Atari 5200, there were four built in controller ports for future multiplayer titles.
The controller itself had quite a few aspects to its design. The four face buttons of the Super Famicom became six, but four of them had a very specific purpose. The C-buttons were named as such for their primary function to help the player control the camera. It was a necessary measure in helping the player orient themselves in the new 3D environments. They would have to take on the role of “director” as well as “actor” to insure a smooth experience.
With uncertainty of how developers would react to 3D movement, the controller was specifically designed with different handling methods in mind. Lance Barr, who often redesigned game hardware for Nintendo of America, was the one who came up with the three-pronged controller design which would allow for situation on either the center prong plus the left or right side, or the two outer edges comfortably (Nintendo Power #79, December 1995). There were supposedly over one-hundred prototypes before the final shape and size was deemed suitable. Not all developers would properly utilize it, but few could ignore the benefits of what sat in the center of this bat-a-rang shaped device.
The analog stick was designed by Genyo Takeda as a crucial extension of the player into 3D space. The very first analog joystick in the home also (coincidentally) belonged to the Atari 5200, but it was ill-suited for commercial use due to one important aspect: No self centering. Self centering is important because a player doesn’t always want to be moving in a direction; sometimes they need to stop and reorient themselves. The Neo Geo CD had perhaps the only other self centering joystick at the time (the special Sega Saturn controller would come later) which marked Nintendo’s simple refinement of the technology as a huge step forwards.
One final aspect to the controller was how it bulged out at the back. This wasn’t just a nifty formation to make it stand up triangularly, but instead a port which existed for peripherals. The main function would be inserting the removable memory cards into the device which could be transferred between controllers and consoles. The main unit itself also had an expansion port of its own which housed a vital component necessary to run the console. The mystery of these ports would linger long after the system’s release.
Let’s Make Some Ultra Games
With all the pieces coming together, actual work on games could finally begin. The SGI development kits, based on the Indy workstations, were shipped out to the hard working developers starting in mid 1995. Nintendo also sourced out development kit manufacturing to Kyoto Microcomputer Co. as well as their own internal Intelligent Systems team. Most would receive a computer board, extra tall development cartridges, as well as ports for the controller once they were available.
The development environment included a host of demonstration software to get companies started on the specifics. There being no operating system to write to, developers had to have a good handle on how their software interacted with the machine. The console had also come at a new junction in games development where programming was largely switching from assembly code to the C programming language, which lowered the barrier for software development. Both were possible, but most developed in C, says Richard Webb.
One of the earliest game announcements was that of a version of Doom by Williams Entertainment, which quickly followed into a trilogy pack of Mortal Kombat ports. Two flight games from Spectrum Holobyte and Sierra On-Line would be on their way for the launch as well, Top Gun and Red Baron respectively.
One of the most anticipated of this lot was Acclaim’s Turok: Dinosaur Hunter developed by Iguana Entertainment. Acclaim had been building a larger technology base for the company before falling into a slump at the tail end of the 16-bit era and were eager to utilize their advanced motion capture studio in a 3D game. The aim was to trounce all other competitors with their technological edge.
“Everything you see going on in videogames today [in terms of motion and performance capture], we were doing and defining back then — trying to figure out how to do it for the first time.” said Remington Scott of Acclaim. They were also taking to bat the Ultra 64’s graphical capabilities in big way.
“I was given a book,” said project manager David Dinsthier. “And in it the authors were talking about the original Doom on the PC, and that each level was composed of about 3,000 polygons, and that each level in Quake was composed of about 10,000. An average level in Turok runs between 250,000 and 300,000 polygons.”
These early developers, taking a risk on a product which would be years in the making, were given a lot of support from Nintendo. Dinsthier recalled, “We had more direct feedback from Nintendo; we got Mario Club [Nintendo’s internal testing group] input earlier on; and we spent more face-to-face time with them during development. We had a closer relationship than was normal.”
Lucasart’s Mark Haigh-Hutchington, working on Star Wars: Shadows of the Empire, had a similar experience. “We were very lucky to receive excellent support from both SGI and Nintendo during the production of the game.” he said. “By Christmas of 1994, we had the basis of the first level of the game, The Battle of Hoth, running quite nicely on the ONYX — ‘quite nicely’ being in high resolution (1280×1024), 32-bit color, and at 60 frames a second. By this point, we had also received a very early prototype of the Nintendo 64 controller. This consisted of a modified Super Nintendo controller with a primitive analogue joystick and Z trigger.”
Across the pond, Rareware was a different story. Despite being 49% owned by Nintendo, there was little oversight on their game development, aside from approval by management. Early 1995 had seen the start of development for their own game based off the James Bond movie Goldeneye which was given mostly free-reign to explore its own path. At first it was a rail shooter very similar to Virtua Cop, but eventually began encroaching upon Turok’s territory with an elaborate first-person shooter design.
All this work was building up to the Space World event in November and the reveal of the Ultra 64. However, that name was possibly floated due to one of their prior licensees, Konami, holding a trademark on the name Ultra Games. The irony was that Ultra Games had been founded as a subsidiary of Konami of America as a Nintendo sanctioned effort to get around some of their restrictive practices in the days of the NES. While this possible reason is only a rumor, they came up with a new name for the console. Shigesato Itoi, game creator and marketing personnel for Nintendo, put forward the simple name of “Nintendo 64”. “[I]t took courage to go with something so utterly usual.” said Itoi.
For many gamers though, it didn’t matter if half the world was on Nintendo’s side. They wanted to see what Nintendo themselves were ready to put out, and specifically the work of one man who had defined their gaming experience for years.
Shiggs up his Sleeve
Shigeru Miyamoto, lead mind behind the series’ Super Mario Bros., The Legend of Zelda, and F-Zero had started to come into the public eye in the early 90s. He discussed a lot about his philosophy for game development which can be easily summed up as ‘every interaction a player undertakes must be fun’. He was all about ease of experience, and bringing that into 3D was his greatest challenge yet.
Miyamoto was directly involved with three N64 games as well as the fully-3D Star Fox 2 and Stunt Race FX on the Super Nintendo (Nintendo Power #80, January 1996). First was the game with Paradigm Simulation Inc. which would become a sequel to the game Pilotwings. Paradigm were specifically led on in development to create interesting flight mechanics without any mention of it being based on the old game, and only after the project had been previewed at the 1995 E3 show did Nintendo actually start sending their designers to help shape the game into a fun experience.
Genyo Takeda was Paradigm’s main contact through the project, with lead design duties relegated to Makoto Wada. “[Nintendo] were very involved,” says David Krueger. “To the extent that once we had emulator boards on hand (about mid way through dev), they sent their director and designer to Texas to stay on-site for the remainder of the development of the game.”
The other project Miyamoto was lightly involved in was from another company who made VR simulations and SGI demos called Angel Studios. They were contracted to create a “DNA-based driving game called Buggy Boogie,” according to Clinton Keith from Angel. “You had these vehicles that would eat other vehicles and adopt their powers and morph. It was really cool. But [Nintendo] would sign three month contracts, and Miyamoto himself would say that he did not want any documents. He would just say, ‘Find the fun, and I’ll be back in three months to take a look at what you have.’”
They never quite ‘found the fun’ and that was completely unacceptable to Miyamoto. He was putting his heart and soul into one particular N64 project that would hopefully teach everybody else how it was done: The next iteration of Super Mario.
Back during the development of the original Star Fox, Miyamoto had been actively toying with the idea of Super Mario in 3D. Contrary to popular belief, he didn’t actually start building a fully fledged Mario game with the FX chip: they were simply experiments. He had a lot of different projects he was attending to at that time: Yoshi’s Island, Super Mario RPG, and the two aforementioned games by the Star Fox programming team.
Jumping was undoubtedly on his mind, and the way this interacted with the camera would be a crucial component to its success. On the early prototypes he worked with Yasunari Nishida to create a base for the game on their SGI Onyx systems. Giles Goddard, who had engineered Stunt Race FX, recalls there being quite a lot of experimentation with the camera which also dictated the pace of the levels. An isometric perspective like that seen in Super Mario RPG was a forerunner, but “That didn’t represent that much of a jump from the original 2D Mario.” said Goddard.
Eventually Miyamoto turned to the Star Fox 2 team. They had been working on free-roaming sections in the game wherein the player’s ship could run around levels, with some very light platforming elements. One of the programmers on the team, Takumi Kawagoe, was brought on to program the camera which could adjust to both outdoor and indoor areas. One of the first experiments they created was putting Mario and a rabbit called MIPS (named after the processor chip) into the world as a test to see how much fun they could make the act of moving Mario through sparse environments.
“[W]e were just playing around with Mario.” said Goddard. “He was wandering around a simple grid to start with, just picking stuff up, dropping stuff.”
“We spent a year or so developing the characters and camera angles before we went into details.” said Miyamoto. “For example, we had the rabbit follow Mario to a mountain summit, then we changed the viewpoint there, and so on.” The automation of the camera was something that he hoped he could rely on, but it became very obvious that they could not program the camera for every situation. Therefore, the C-buttons would serve a greater purpose in the moment to moment gameplay.
Even Miyamoto had to wait six months before obtaining the actual controller for the system, having controlled Mario up to that point with a digital keyboard. Once he got his hands on the analog stick though he was blown away by the possibilities. Animators Yoshiaki Koizumi and Satoru Takizawa were given the task of creating a range of movement options that had never been seen before in a 3D game. Different sorts of jumps, attacks, and movement speeds all needed to be portrayed with a sense of momentum and freedom that no other game had achieved before.
In the midst of many of these experiments, they found time to revisit Nintendo’s first 3D creation. The latest models of the SGI Indy had come with a top-mounted camera which was designed for video conferencing, but could also interface with the computer. “I was playing around with this thing with a GL [graphics library] program on the Indy,” said Goddard. “Playing around with this Mario face […] two ping-pong balls for a face. Miyamoto said: can you make a Mario face out of that for the N64?”
With Miyamoto’s persistent sense of ‘play’ in mind, he wanted players to be able to interact with the newly refined Mario head in an interesting way. “At the time, I was thinking of things to make that would show off what N64 could do.” Goddard remembered. “One of the things was real-time vertex calculations. […] We wanted to move around individual points, morph it.” The team had created something special which was ultimately superfluous to the final product, but channeled the same energy and spirit that permeated through the whole project.
However, Miyamoto started to feel a certain fatigue about the game even after his constant pushes for exciting interaction. “I realized halfway through [development] that it was getting boring.” he said. “So I went around and asked everyone, ‘This game was really fun in the beginning, but now it doesn’t feel fun anymore, does it?’ And just as I’d expected, they all said, ‘We agree.’”
His fix would demonstrate just how painstakingly mechanical ‘fun’ truly was. “[I]t was something really simple with Mario’s movement. In the beginning, we had Mario turning really slowly, so that it was really overemphasized. But at some point he’d started turning really quickly. He kind of zipped around […] So then we changed it so that he went back to turning really slowly. And well, I’m not sure if that was the right change to make, but it was really important to me.”
That small bit of instinct, a seemingly selfish attention to detail stuck at the heart of Miyamoto’s contribution to games as a whole. ‘Finding the fun’ had as much to do with color and character as it did with instinct and number crunching. He couldn’t wait to show it to the world.
Connections
The Space World ’95 show on November 24th was split into two parts. One part was for the group of retailers who sold Nintendo products, the Shoshinkai, and the other was the open show floor wherein press and public alike would get to put their grubby mitts on the Nintendo 64 for the first time. However, unlike what they originally planned, nobody could take one home. Hiroshi Yamauchi announced yet another delay for the product, saying that the Japanese release would instead be on April 21st, 1996 with the U.S release shortly thereafter. This was certainly not surprising given how Mario 64, which definitely had to be a launch title, was less than half completed at the time (Nintendo Power #80, January 1996).
Press from all over the world attended the event and were treated to a load of newly announced software at the showcase. Kirby Bowl 64 (the only other playable game aside from Mario), Wave Race 64, Zelda 64, Mario Kart R, Blast Dozer by Rareware, DMA design’s Body Harvest, and perhaps most interestingly: Star Fox 64. For the past year Nintendo had been positioning Star Fox 2 (on their still primary hardware, the Super Nintendo) as a major release. However, Miyamoto had decided — seemingly without warning — that he didn’t want the game competing with the upcoming N64. A “clean break” as programmer Dylan Cuthbert put it.
Jez San of the same team was less kind with his choice of words. “They canned Star Fox 2 even though it was finished and used much of our code in Star Fox 64 without paying us a penny.” he said. The Argonaut team had also been working on a 3D platformer starring Yoshi that supposedly gave Miyamoto the idea for Mario 64, which he later thanked them for. The cancellation of Star Fox 2 became particularly infamous because of how long the N64 would be delayed, which in hindsight could have led to the game being released without any problems.
Despite these decisions, things were looking fairly good for Nintendo. Perhaps not all of these titles would be available near the launch of the system, but surely the road ahead was positive for consumers, right? April was a long time waiting, and with SGI’s role mostly finished, it was all on the game creators to finish the job.
Nintendo was not sitting idle however. They had been making business negotiations with Jim Clark along with his soon-to-be-famous cohort Marc Andreessen. The two of them had been looking into connectivity based services and one of their first ideas had been an online gaming network. Of course, such ideas had existed for years, but nobody knew better about how to bridge the gap of usability to computer networks than Andreessen, who had developed the Mosiac web browser to access this nifty new thing called the World Wide Web.
The two wrote up twenty pages of ideas and a business proposal that they pitched to Nintendo in 1994 that could be adapted for Project Reality. Though receptive to the idea, Nintendo had an attitude to the project that Clark was very familiar with. “They wanted to own the whole thing.” he said. “[T]hey were going to fund us. But we weren’t going to get anything out of it […] So in the end we gave up on that.”
Andreessen and Clark would instead go on to found Netscape and kick off the era of the commercial internet, culminating in a legendary Initial Public Stock Offering (IPO) that put them into the big leagues. At some point Nintendo got back in contact with the company heads, and a rumor spread that the former planned to create a game download service using a modem that would run the Netscape Navigator web browser. Nintendo’s Howard Lincoln would call this claim “unfounded”.
Even with the success of the web, Nintendo would continue exploring alternatives to remote gaming networks. They had breached the subject on their two prior consoles with Japan exclusive addons: The Famicom Modem and the Sattelaview, both in partnership with cable companies. Yamauchi would (along with Microsoft and Nomura Research) continue to pursue providing gaming through cable networks long after it was clear that the web age had begun.
Modems weren’t the only addons that had been announced. Space World ’95 also brought with it the announcement of the Nintendo 64 Disk Drive, or “Bulky Drive” as it was otherwise known, manufactured by Alps Electric. This was an extension of Nintendo’s successful Famicom Disk System which had been the launch platform for games like the original Legend of Zelda, Super Mario Bros. 2 (both versions), and Metroid. It was their method to extend the format possibilities of the device without having to give up control of it, as the 64DD used a custom floppy disk-like format called Magni-Disc rather than CD-ROM. The games planned for the system included the new Legend of Zelda game (Zelda 64) as well as the seventh iterations in the Dragon Quest and Final Fantasy role-playing game series.
Bated breaths and watchful eyes were kept poised on Nintendo as April approached. The Super Nintendo had officially wound down, the Virtual Boy failed to steal any “32-bit” market share, and Sony was slowly redefining the worldwide gaming landscape at both Nintendo and Sega’s expense. Nintendo had to act in order to secure its position.
Yet another cat had to be let out of the bag though. In late March it was announced that the Nintendo 64 would be delayed once again. Smug journalists stuck out their tongues with an “I told you so” act, but the release was finally solidified. In the US, it would launch in September. In Japan, it would be June 23rd, 1996.
Lift Off
At the price of 25,000 yen ($230 equivalent), Nintendo managed to meet the target that they had set nearly 3 years ago, but at what cost would remain to be seen. Things started out well though with 350,000 of 500,000 shipped units sold in just three days. The only games available at launch were Super Mario 64, Pilotwings 64, and the obligatory Shogi game by Seta Corporation (who would go on to have an interesting relationship with the console).
Three games which each cost upwards of $60 (equivalent) was hardly anything to write home about. Mario 64 wasn’t included as a pack-in for the launch, but even if it had there was no doubt that a more diverse launch lineup could have helped propel the product. Nintendo had chosen its software partners extremely carefully to maintain a quality brand image, but even the Virtual Boy had more than three games to start with.
The truth of the matter was fairly obvious. Developers were having an incredibly difficult time adjusting to 3D games development. To their credit, Nintendo was very generous in allowing most developers to refine their product through the endless delays to bring it to a reasonable state. Rareware’s Goldeneye game became rather legendary for being released two years after the movie came out and Acclaim’s Turok was actually postponed supposedly to fill up one of the many software droughts that the system would suffer.
Miyamoto, reflecting the company’s position of quality over quantity, would later say, “You see, the Nintendo 64 weeded out weaker developers at an early stage. In the long term, I think that was necessary. Almost a rite of passage.”
Nintendo’s commitment to this method though left them in a bad position in the coming months. Not only did many of the games come out years after their initial showing, but planned exclusives drifted elsewhere. Japanese RPG companies Squaresoft and Enix Corporation both cancelled their plans for publishing on Nintendo’s console and drifted over to the PlayStation instead due to greater sales potential (Enix did publish one game on the N64). Japanese companies in general were very scant with their N64 offerings, leaving a very wide gap between releases that Nintendo were not always able to fill.
Despite the company having incredibly strong ties with retailers, the Shoshinkai group had actually disbanded by the time of the N64’s release. Nintendo instead aligned themselves with the Japanese convenient store chain Lawson, but that was hardly a strong partnership. A minor scare also hit the company in August of 1996 upon the departure of engineer Gunpei Yokoi which caused a radical drop in the Nintendo stock price. It got so bad that they were temporarily delisted from the Tokyo Stock Exchange, but were brought back before long.
Still, the launch was successful and the showing at E3 1996 in May stoked the fire for the North American launch. The system would go for $250 with, again, no pack-in title. The advertising campaign that accompanied it bore the slogan “Change the System”, and again tried to underplay the fact that the system only had two games (no Shogi for the Americans).
Many stores sold out of the console, though some people posited that Nintendo had undelivered to make that promise. It took a few months to get rolling, but within a year they had sold some 3.6 million units in the United States, outpacing both of their competitors for the year, though not in overall sales. However, as Sony refocused and began formulating the PlayStation brand, Nintendo would ultimately lose out.
Some of that was to blame on Sony’s far more aggressive tact in the vein of Sega to appeal to audiences beyond a typical game-playing demographic. Shigesato Itoi explained it as more of a failing on Nintendo’s part. “…[I]t’s probably more accurate to just call Nintendo’s [commercials] uncool, rather than say Sony’s ads are cool.”
No accolades such as “Machine of the Year” from Time magazine or the stronger European launch in March of ’97 were going to help Nintendo rise back to its unchallenged position as a gaming juggernaut. Some of that was a good thing, but it also meant that the audience missed out on a lot of fantastic games on the N64. Ports between the Sony and Nintendo consoles were prevalent enough for those without company loyalty, but many of the games exclusive to the platform took the concept of navigating 3D space to new heights.
What Now?
The major players involved with Project Reality have all carried with them the spirit from that project in some form or another. SGI did so to its determent, ceding the market to IBM PCs by focusing on low end applications which ultimately crumbled their niche. They never successfully transitioned into components like GPU devices, and so soon faded from relevance (though the name still exists today). Tim Van Hook and many of the engineers from SGI eventually left to try and fill that niche with a company called ArtX, which would be contracted by Nintendo to create graphics technology for their N64 successor before being bought by ATI (which was then acquired by AMD).
The technology studios that Nintendo had drafted into games development, Paradigm Simulation and Angel Studios, soon shifted themselves to focusing on games for the N64 and other platforms. The former would be bought by the Infogrames “Atari” brand in 2000 and the latter was purchased by DMA Design, by then known as Rockstar Games.
https://www.youtube.com/watch?v=7hSphL_xySs
Netscape never came to the N64, but the Disk Drive did (two years late). With it came an online service called RandNet which allowed for sharing of creations made in in one of the devices many creative applications. The 64DD failed to have even a fraction of the success that the Famicom Disk Drive had though, leading to the cancellation of the addon and the service in 2000.
What about the title of this article? What of the virtual reality industry? VPL Research would linger on through the 90s, never able to recapture the spark they had ignited at the beginning of the decade, being acquired by Sun Microsystems in 1998. SimGraphics fared better, still having been active up until 2020, having continued to provide the VActor system used by Charles Martinet to entertain children as Mario, now from remote locations over the web. Rick Lazzarini and The Character Shop have also remained in business, though his own methods of melding real actors and digital ones has largely fallen by the wayside.
And how about Nintendo? Well, Nintendo’s in an interesting new phase. After a return to undoubted king-of-the-hill status with the Wii, they were forced to reconsider their business tact after their first loss in 31 years occurred in 2012. Before the successful launch of the Switch there were many rumors that they were dabbling around with VR again, though any ambitions they may have seem to be on the backburner for now. They seem to have at least gotten back on track by putting a higher amount of value on third party software developers, though only time will tell if their current vision for gaming will succeed.
The N64 could not have been created if there were not total visionaries looking beyond the practical and into the future. 3D graphics technology had always been advancing, but despite its dated look the N64 was by far the most capable game dedicated machine of its time. Super Mario 64 captured the imagination of millions and led to further refinement for 3D real-time games in both control and character with games like The Legend of Zelda: Ocarina of Time.
To belabor the point is to miss the point a little bit. Nintendo’s and Silicon Graphics’ goals were driven by experience and circumstance. The developers were driven by the desire to create true 3D games on a level that neither the PlayStation nor the Saturn could really offer. For all its failings and revolutions, the N64 stands as an embodiment of a desire to feel and experience something new. For those who were able to do so in its heyday, it certainly left an impression.
[Big thanks to Shannon Perez for editing the pictures, to Micah G. Pyre for spell check, and to all the people who graciously loaned me their time and memories to help write this story.]
5 thoughts on “The Nintendo 64 Was the Culmination of 90s Virtual Reality”