MicroSoft's Talisman Initiative

Presented by Allen R. Ballman

Link: http://www.research.microsoft.com/SIGGRAPH96/Talisman/

What is Talisman?

• New Architecture for interactive media, principally focused on 3D graphics
• Leverages spatial and temporal coherence in dynamic scenes to significantly improve price-performance
• Developed in conjunction with major PC industry partners and will be broadly licensed.

Fundamental Forces - Limitations of traditional architecture

• Huge memory bandwidth required for high quality
• DRAM bandwidth improving 12% annually on average
• Quality features (i.e. anti-aliasing, texture filtering) drive bandwidth requirements higher
• 640x480, 16-bit color, 16-bit Z, Point sample, no anti-aliasing, 30 Hz refresh = 240 MB/sec
• 640x480, 24-bit color, 24-bit Z, TrilinearMIP, No anti-aliasing, 60 Hz refresh = 1900 MB/sec
• 640x480, 24-bit color, 24-bit Z, Anisotropic, anti-aliasing, 75 Hz refresh = 5300 MB/sec
• Increase the resloution to 800x600 = 8300 MB/sec
• Increase to 1024x768 = 14,200 MB/sec!!!!!
• Animation achieved through brute force
• Re-rendering entire frames at refresh rates

Hardware Architecture of Talisman

• Composited Image Layers
• Independent objects are rendered into seperate image layers
• Object images updated only when they change
• Object resolutions can vary
• Image layers are alpha-composited at video rates
• Full affine transformation of each image layer at video rates
• Typical that 4 frames/oject reuse
• Chunking
• Each image layer is rendered in 32x32 chunks
• All polygons for a 32x32 chunk are rendered before proceeding to next chunk
• Allows a 32x32 depth buffer to be on-chip
• Anti-aliasing supported with depth buffering and tranlucency using on-chip fragment buffer
• Compression
• Image compression used for texture and image layers
• Lossless (i.e. entropy encoding) and lossy (i.e. jpeg-like) compression supported
• Significantly reduces bandwidth and capacity requirements
• 16:1 compression of textures
• 5:1 compression of rendered image layers
• High quality rendering(Increased Realism)
• Anisotropic filtering of textures
• Filter shape is dynamically adjusted based on surface orientation
• Multi-pass rendering
• Filtered Shadows, reflection maps
• Anti-aliasing (simultaneous z-buffering, transulcency)
• Bandwidth reduction
• 640x480 = 87 MB/sec; 800x600 = 135 MB/sec; 1024x768 = 220 MB/sec

Reference Hardware Implementation

• PCI Card

• Polygon Object Processor

• Renders into a 32x32 buffer (32 bits color, 24 bit depth per pixel)
• Seperate buffer maintains edge pixel information
• Texture data is decompressed and filtered
• Anti-aliasing is performed as a seperate step
• Resulting 32x32 image is then compressed and stored
• Image Layer Compositor

• 32 scanlines processed together
• Data structure maintains a z-sorted list of image layers (set of chunks) visible in each 32 s-line region
• Performs affine transformations (scaling, rotation, translation (sub-pixel), and shear)
• Pixel and alpha passed to the Compositing Buffer at 4 pixels per clock cycle
• Compositing Buffer/DAC
• Ping pong 1344x32x24 bit scanline buffers
• display one while updating the other
• 1344x32x8 bit alpha buffer
• Four pixels are alpha-composed into the scanline buffer each clock cycle
• Video data is transferred from the scanline buffer to the LUT-DAC at video rates
• Performance
• 20-30K polygon complexity (1024x768)
• Updated at full video refresh rate
• With anti-aliasing, anisotropic filtering, and z-buffering
• 1344x1024 desktop resolution, 32 bit/pixel
• 40 Mpixel/sec polygon rendering rate
• 320 Mpixel/sec image layer compositing rate
• Reference Implementation Partners
• Media DSP - Samsung Semiconductor, Philips
• Polygon and Image Layer chips - Cirrus Logic, Silicon Engineering, Inc.
• Compositing Buffer/DAC and Interface chip - Fujitsu Microelectronics
• Many other vendors working on derivative designs

Conclusions

• Exploits spatial coherence through image compression
• Exploits temporal coherence through image transforms and video rate compositing
• Dramatically reduces memory bandwidth requirements to enable consumer cost targets
• Supported by major PC industry players