HISTORY OF COMPUTER GRAPHICS

| History 1960 | History 1970 | History 1980 | History 1990 | 2000 and beyond |

The 1970s saw the introduction of computer graphics in the world of television. Computer Image Corporation (CIC) developed complex hardware and software systems such as ANIMAC, SCANIMATE and CAESAR. All of these systems worked by scanning in existing artwork, then manipulating it, making it squash, stretch, spin, fly around the screen, etc. . . Bell Telephone and CBS Sports were among the many who made use of the new computer graphics.

While flat shading can make an object look as if it's solid, the sharp edges of the polygons can detract from the realism of the image. While one can create smaller polygons (which also means more polygons), this increases the complexity of the scene, which in turn slows down the performance of the computer rendering the scene. To solve this, a Henri Gouraud in 1971 presented a method for creating the appearance of a curved surface by interpolating the color across the polygons. This method of shading a 3D object has since come to be known as Gouraud shading. One of the most impressive aspects of Gouraud shading is that it hardly takes any more computations than Flat shading, yet provides a dramatic increase in rendering quality. One thing that Gouraud shading can't fix is the visible edge of the object. The original flat polygons making up the torus are still visible along the edges of the object.

One of the most important advancements to computer graphics appeared on the scene in 1971, the microprocessor. Using Integrated Circuit technology developed in 1959, the electronics of a computer processor were miniaturized down to a single chip, the microprocessor, sometimes called a CPU (Central Processing Unit). One of the first desktop microcomputers designed for personal use was the Altair 8800 from Micro Instrumentation Telemetry Systems (MITS). Coming through mail-order in kit form, the Altair (named after a planet in the popular Star Trek series) retailed for around $400. Later personal computers would advance to the point where film-quality computer graphics could be created on them.

In that same year, Nolan Kay Bushnell along with a friend formed Atari. He would go on to create an arcade video game called Pong in 1972 and start an industry that continues even today to be one of the largest users of computer graphics technology.

In the 1970's a number of animation houses were formed. In Culver City, California, Information International Incorporated (better known as Triple I) formed a motion picture computer graphics department. In San Rafael, California, George Lucas formed Lucasfilm. In Los Angeles, Robert Abel & Associates and Digital Effects were formed. In Elmsford, New York, MAGI was formed. In London, England, Systems Simulation Ltd. was formed. Of all these companies, almost none of them would still be in business ten years later. At Abel & Associates, Robert Abel hired Richard Edlund to help with computer motion control of cameras. Edlund would later get recruited to Lucasfilm to work on Star Wars, and eventually to establish Boss Film Studios creating special effects for movies and motion pictures and winning four Academy Awards.

In 1970 Gary Demos was a senior at Caltech when we saw the work of John Whitney Sr. This immediately developed an interest in him for computer graphics. This interest was further developed when he saw work done at Evans & Sutherland, along with the animation that was coming out of the University of Utah. So in 1972 Demos went to work for E&S. At that time they used Digital PDP-11 computers along with the custom built hardware that E&S was becoming famous for. These systems included the Picture System that featured a graphics tablet and color frame buffer (originally designed by UU).

It was at E&S that Demos met John Whitney Jr., the son of the original graphics pioneer. E&S started to work on some joint projects with Triple I. Founded in 1962, Triple I was in the business of creating digital scanners and other image processing equipment. Between E&S and Triple I there was a Picture Design Group. After working on a few joint projects between E&S and Triple I, Demos and Whitney left E&S to join Triple I and form the Motion Picture Products group in late 1974. At Triple I, they used PDP-10s and a Foonley Machine (which was a custom PDP-10). They developed another frame buffer that used 1000 lines; they also built custom film recorders and scanners along with custom graphics processors, image accelerators and the software to run it. This development led to the first use of computer graphics for motion pictures in 1973 when Whitney and Demos worked on the motion picture "Westworld". They used a technique called pixellization which is a computerized mosaic created by breaking up a picture into large color blocks. This is done by dividing up the picture into square areas, and then averaging the colors into one color within that area.

In 1973 the Association of Computing Machinery's (ACM) Special Interest Group on Computer Graphics (SIGGRAPH) held its first conference. Solely devoted to computer graphics, the convention attracted about 1,200 people and was held in a small auditorium. Since the 1960's the University of Utah had been the focal point for research on 3D computer graphics and algorithms. For the research, the classes set up various 3D models such as a VW Beetle, a human face, and the most popular, a teapot. It was in 1975 that a M. Newell developed the Utah teapot, and throughout the history of 3D computer graphics it has served as a benchmark, and today it's almost an icon for 3D computer graphics. The original teapot that Newell based his computer model on can be seen at the Boston Computer Museum displayed next to a computer rendering of it.

Ed Catmull received his Ph. D. in computer science in 1974 and his thesis covered Texture Mapping, Z-Buffer and rendering curved surfaces. Texture mapping brought computer graphics to a new level of realism. Catmull had come up with the idea of texture mapping while sitting in his car in a parking lot at UU and talking to another student, Lance Williams, about creating a 3D castle. Most objects in real life have very rich and detailed surfaces, such as the stones of a castle wall, the material on a sofa, the wallpaper on a wall, the wood veneer on a kitchen table. Catmull realized that if you could apply patterns and textures to real-life objects, you could do the same for their computer counterparts. Texture mapping is the method of taking a flat 2D image of what an object's surface looks like, and then applying that flat image to a 3D computer generated object. Much in the same way that you would hang wallpaper on a blank wall.

The z-buffer aided the process of hidden surface removal by using zels which are similar to pixels but instead of recording the luminance of a specific point in an image, they record the depth of that point. The letter "z" reflecting the depth (as does Y for vertical position and X for horizontal position). The z-buffer was then an area of memory devoted to holding the depth data for every pixel in an image. Today high-performance graphics workstations have a z-buffer built-in.

While Gouraud shading was a great improvement over Flat shading, it still had a few problems as to its realism. If you look closely at the Gouraud shaded torus you will notice slight variations in the shading that reveal the underlying polygons. These variations can also cause reflections to appear incorrectly or even disappear altogether in certain circumstances. This was corrected however by Phong Bui-Toung, a programmer at the UU (of course). Bui-Toung arrived at UU in 1971 and in 1974 he developed a new shading method that came to be known as Phong shading. After UU, Bui-Toung went on to Stanford as a professor, and early in 1975 he died of cancer. His shading method accurately interpolates the colors over a polygonal surface giving accurate reflective highlights and shading. The drawback to this is that Phong shading can be up to 100 times slower than Gouraud shading. Because of this, even today, when animators are creating small, flat 3D objects that are not central to the animation, they will use Gouraud shading on them instead of Phong. As with Gouraud shading, Phong shading cannot smooth over the outer edges of 3D objects.

A major breakthrough in simulating realism began in 1975 when the French mathematician, Dr. Benoit Mandelbrot published a paper called "A Theory of Fractal Sets." After some 20 years of research he published his findings and named them Fractal Geometry. To understand what a fractal is, consider that a straight line is a one-dimensional object, while a plane is a two-dimensional object. However, if the line curves around in such a way as to cover the entire surface of the plane, then it is no longer one dimensional, yet not quite two dimensional. Mandelbrot described it as a fractional dimension, between one and two.

To understand how this helps computer graphics, imagine creating a random mountain terrain. You may start with a flat plane, then tell the computer to divide the plane into four equal parts. Next the new center point is offset vertically some random amount. Following that, one of the new smaller squares is chosen, subdivided, with its center slightly off-set randomly. The process continues recursively until some limit is reached and all the squares are off-set.

Mandelbrot followed up his paper with a book entitled "The Fractal Geometry of Nature." This showed how his fractal principles could be applied to computer imagery to create realistic simulations of natural phenomena such as mountains, coastlines, wood grain, etc.

After graduating in 1974 from UU, Ed Catmull went to a company called Applicon. It didn't last very long however, because in November of that same year he was made an offer he couldn't refuse. Alexander Schure, founder of New York Institute of Technology (NYIT), had gone to the UU to see their computer graphics lab. Schure had a great interest in animation and had already established a traditional animation facility at NYIT. After seeing the setup at UU, he asked Evans what equipment he needed to create computer graphics. He told his people to "get me one of everything they have." The timing happened to be just right because UU was running out of funding at the time. Schure made Ed Catmull Director of NYIT's new Computer Graphics Lab. Then other talented people in the computer graphics field such as Malcolm Blanchard, Garland Stern and Lance Williams left UU and went to NYIT. Thus the leading center for computer graphics research soon switched from UU to NYIT.

One talented recruit was Alvy Ray Smith. As a young student at New Mexico State University in 1964, he had used a computer to create a picture of an equiangular spiral for a Nimbus Weather satellite. Despite this early success, Smith didn't take an immediate interest in computer graphics. He moved on to Stanford University, got his Ph.D., then promptly took his first teaching job at New York University. Smith recalls, "My chairman, Herb Freeman, was very interested in computer graphics, some of his students had made important advances in the field. He knew I was an artist and yet he couldn't spark any interest on my part, I would tell him 'If you ever get color I'll get interested.' Then one day I met Dr. Richard Shoup, and he told me about Xerox PARC (Palo Alto Research Center). He was planning on going to PARC to create a program that emulated painting on a computer the way an artist would naturally paint on a canvas."

Shoup had become interested in computer graphics while he was at Carnegie Mellon University. He then became a resident scientist at PARC and began working on a program he called "SuperPaint." It used one of the first color frame buffers ever built. At the same time Ken Knowlton at Bell Labs was creating his own paint program.

Smith on the other hand, wasn't thinking much about paint programs. In the meantime, he had broken his leg in a skiing accident and re-thought the path his life was taking. He decided to move back to California to teach at Berkeley in 1973. "I was basically a hippie, but one day I decided to visit my old friend, Shoup in Palo Alto. He wanted to show me his progress on the painting program, and I told him that I only had about an hour, and then I would need to get back to Berkeley. I was only visiting him as a friend, and yet when I saw what he had done with his paint program, I wound up staying for 12 hours! I knew from that moment on that computer graphics was what I wanted to do with my life." Smith managed to get himself hired by Xerox in 1974 and worked with Shoup in writing SuperPaint.

A few years later in 1975 in nearby San Jose, Alan Baum, a workmate of Steve Wozniak at Hewlett Packard, invited Wozniak to a meeting of the local Homebrew Computer Club. Homebrew, started by Fred Moore and Gorden French, was a club of amateur computer enthusiasts, and it soon was a hotbed of ideas about building your own personal computers. From the Altair 8800 to TV typewriters, the club discussed and built virtually anything that resembled a computer. It was a friend at the Homebrew club that first gave Wozniak a box full of electronic parts and it wasn't long before Wozniak was showing off his own personal computer/toy at the Homebrew meetings. A close friend of Wozniak, Steve Jobs, worked at Atari and help Wozniak develop his computer into the very first Apple computer. They built the units in a garage and sold them for $666.66.

In the same year William Gates III at the age of 19 dropped out of Harvard and along with his friend Paul Allen, founded a company called Microsoft. They wrote a version of the BASIC programming language for the Altair 8800 and put it on the market. Some five years later in 1980, when IBM was looking for an operating system to use with their new personal computer, they approached Microsoft and Gates remembered an operating system for Intel 8080 microprocessors written by Seattle Computer Products (SCP) called 86-DOS. Taking a gamble, Gates bought 86-DOS from SCP for $50,000, rewrote it, named it DOS and licensed it (smartly retaining ownership) to IBM as the operating system for their first personal computer. Today Microsoft dominates the personal computer software industry with gross annual sales of almost 4 billion dollars, and now it has moved into the field of 3D computer graphics.

Meanwhile back at PARC, Xerox had decided to focus solely on black and white computer graphics, dropping everything that was in color. So Alvy Ray Smith called Ed Catmull at NYIT and went out east with David DiFrancesco to meet with Catmull. Everyone hit it off, so Smith made the move from Xerox over to NYIT; this was about two months after Catmull had gotten there. The first thing Smith did was write a full color (24-bit) paint program, the first of its kind.

Later others joined NYIT's computer graphics lab including Tom Duff, Paul Heckbert, Pat Hanrahan, Dick Lundin, Ned Greene, Jim Blinn, Rebecca Allen, Bill Maher, Jim Clark, Thaddeus Beier, Malcom Blanchard and many others. In all, the computer graphics lab of NYIT would eventually be home to more than 60 employees. These individuals would continue to lead the field of computer graphics some twenty years later. The first computer graphics application NYIT focused on was 2D animation and creating tools to assist traditional animators. One of the tools that Catmull built was "Tween," a tool that interpolated in-between frames from one line drawing to another. They also developed a scan-and-paint system for scanning and then painting pencil-drawn artwork. This would later evolve into Disney's CAPS (Computer Animation Production System).

Next the NYIT group branched into 3D computer graphics. Lance Williams wrote a story for a movie called "The Works," sold the idea to Schure, and this movie became NYIT's major project for over two years. A lot of time and resources were spent in creating 3D models and rendering test animations. "NYIT in itself was a significant event in the history of computer graphics" explains Alvy Ray Smith. "Here we had this wealthy man, having plenty of money and getting us whatever we needed, we didn't have a budget, we had no goals, we just stretched the envelope. It was such an incredible opportunity, every day someone was creating something new. None of us slept, it was common to work 22 hour days. Everything you saw was something new. We blasted computer graphics into the world. It was like exploring a new continent."

However, the problem was that none of the people in the Computer Graphics Lab understood the scope of making a motion picture. "We were just a bunch of engineers in a little converted stable on Long Island, and we didn't know the first thing about making movies" said Beier (now technical director for Pacific Data Images). Gradually over a period of time, people became discouraged and left for other places. Smith continues, "It just wasn't happening. We all thought we would take part in making a movie. But at the time it would have been impossible with the speed of the computers." Alex Schure made an animated movie called "Tubby the Tuba" using conventional animation techniques, and it turned out to be very disappointing. "We realized then that he really didn't have what it takes to make a movie," explains Smith. Catmull agrees, "It was awful, it was terrible, half the audience fell asleep at the screening. We walked out of the screening room thinking 'Thank God we didn't have anything to do with it, that computers were not used for anything in that movie!'" The time was ripe for George Lucas.

Lucas, with the success of Star Wars under his belt, was interested in using computer graphics on his next movie, "The Empire Strikes Back". So he contacted Triple I, who in turn produced a sequence that showed five X-Wing fighters flying in formation. However disagreements over financial aspects caused Lucas to drop it and go back to hand-made models. The experience however showed that photorealistic computer imagery was a possibility, so Lucas decided to assemble his own Computer Graphics department within his special effects company, Lucasfilm. Lucas sent out a person to find the brightest minds in the world of Computer Graphics. He found NYIT. Initially the individual went to Carnegie Mellon University and talked to a professor who referred him to one of his students, Ralph Guggenheim, who referred him to Catmull at NYIT. After a few discussions, Catmull flew out to the west coast and met with Lucas and accepted his offer.

Initially only five from NYIT went with Catmull including Alvy Ray Smith, David DiFrancesco, Tom Duff and Ralph Guggenheim. Later however, others would take up the opportunity. Slowly the computer graphics lab started to fall apart and ceased to be the center of computer graphics research. The focus had shifted to Lucasfilm and a new graphics department at Cornell University. Over the next 15 years, Lucasfilm would be nominated for over 20 Academy Awards, winning 12 Oscars, five Technical Achievement Awards and two Emmys.

Looking back at NYIT, Catmull reflects "Alex Schure funded five years of great research work, and he deserves credit for that. We published a lot of papers, and were very open about our research, allowing people to come on tours and see our work. However now there are a lot of lawsuits going on, mainly because we didn't patent very much. People then subsequently acquired patents on that work and now we are called in frequently to show that we had done the work prior to other people."

Catmull continues, "We really had a major group of talented people in the lab, and the whole purpose was to do research and development for animation. We were actually quite stable for a long time, that first five years until I left. However, the primary issue was to make a feature film, and to do that you have to gather a lot of different kinds of skills; Artistic, Editorial, etc.. Unfortunately, the managers of the school did not understand this. They appreciated the technical capabilities. So as a group we where well taken care of, but we all recognized that in order to produce a feature film we had to have another kind of person there, movie people, and basically those people weren't brought into the school. We were doing the R & D but we just could not achieve our goals there. So when Lucas came along, and proved that he did have those kind of capabilities and said I want additional development in this area (of computer graphics), we jumped at it."

Thus in 1979 George Lucas formed the new computer graphics division of Lucasfilm to create computer imagery for motion pictures. Catmull became vice president and during the next six years, this new group would assemble one of the most talented teams of artists and programmers in the computer graphics industry. The advent of Lucasfilm's computer graphics department is viewed by many as another major milestone in the history of computer graphics. Here the researchers had access to funds, but at the same time they were working under a serious movie maker with real, definite goals.

The ACM in 1976 allowed for the first time, exhibitors in the annual SIGGRAPH conference. This turned up 10 companies who exhibited their products. By 1993 this would grow to 275 companies with over 30,000 attendees.

Systems Simulation Ltd. (SSL) of London created an interesting computer graphics sequence for the movie "Alien" in 1976. The scene called for a computer-assisted landing sequence where the terrain was viewed as a 3D wireframe. Initially a polystyrene landscape was going to be digitized to create the terrain. However, the terrain needed to be very rugged & complex and would have made a huge database if digitized. Alan Sutcliffe of SSL decided to write a program to generate the mountains at random. The result was a very convincing mountain terrain displayed in wireframe with the hidden lines removed. This was typical of early efforts at using computer generated imagery (CGI) in motion pictures, using it to simulate advanced computers in Sci-Fi movies.

Meanwhile the Triple I team was busy in 1976 working on "Westworld's" sequel, "Futureworld." In this film, robot Samurai warriors needed to materialize into a vacuum chamber. To accomplish this, Triple I digitized still photographs of the warriors and then used some image processing techniques to manipulate the digitized images and make the warriors materialize over the background. Triple I developed some custom film scanners and recorders for working on films in high resolutions, up to 2,500 lines. Also in that same year at the Jet Propulsion Laboratory in Pasadena, California (before going to NYIT), James Blinn developed a new technique similar to Texture Mapping. However, instead of simply mapping the colors from a 2D image onto a 3D object, the colors were used to make the surface appear as if it had a dent or a bulge. To do this, a monochrome image is used where the white areas of the image will appear as bulges and the black areas of the image will appear as dents. Any shades of gray are treated as smaller bumps or bulges depending on how dark or how light the shade of gray is. This form of mapping is called Bump Mapping.

Bump maps can add a new level of realism to 3D graphics by simulating a rough surface. When both a texture map and a bump map are applied at the same time, the result can be very convincing. Without bump maps, a 3D object can look very flat and un-interesting.

Busy Blinn also published a paper in that same year on creating surfaces that reflect their surroundings. This is accomplished by rendering six different views from the location of the object (top, bottom, front, back, left and right). Those views are then applied to the outside of the object in a way similar to standard texture mapping. The result is that an object appears to reflect its surroundings. This type of mapping is called environment mapping.

In December of 1977, a new magazine debuted called Computer Graphics World. Back then the major stories involving computer graphics revolved around 2D drafting, remote sensing, IC design, military simulation, medical imaging and business graphics. Today, some 17 years later, CGW continues to be the primary medium for computer graphics related news and reviews. Computer graphics hardware was still prohibitively expensive at this time. The National Institute of Health paid 65,000 dollars for their first frame buffer back in 1977. It had a resolution of 512x512 with 8 bits of color depth. Today a video adapter with the same capabilities can be purchased for under 100 dollars.

During the late 1970's Don Greenberg at Cornell University created a computer graphics lab that produced new methods of simulating realistic surfaces. Rob Cook at Cornell realized that the lighting model everyone had been using best approximated plastic. Cook wanted to create a new lighting model that allowed computers to simulate objects like polished metal. This new model takes into account the energy of the light source rather than the light's intensity or brightness.

As the second decade of computer graphics drew to a close the industry was showing tremendous growth. In 1979, IBM released its 3279 color terminal and within 9 months over 10,000 orders had been placed for it. By 1980, the entire value of all the computer graphics systems, hardware, and services would reach a billion dollars.

| History 1960 | History 1970 | History 1980 | History 1990 | 2000 and beyond |