3D Science and the Human Heart

nickcharles · April 4, 2006 12:00 am

Courtesy of 3DScience.com

Bryan Brandenburg is the CEO, founder, and Chief Science Officer of 3DScience.com. Hes also the CEO and Chairman of Zygote Media Group, Inc. These are two companies that share a vision to bring quality modeling to the medical industry. Bryan, along with Roger Clarke and David Dunston, got together in January of 2005 to change the face of scientific visualization. Bar none, the most innovative and advanced 3D human heart has been released for sale at www.3dscience.com, which is the portal for Zygotes medical and science creations. I had the opportunity to chat with Bryan Brandenberg, as well as Roger Clarke, Vice President of production and co-founder of Zygote, and Tim Rush, Public Relations with Zygote. This turned out to be a substantial eyebrow-raising conversation.

Courtesy of 3DScience.com

Zygote unveiled its human heart model in February, 2006 on 3DScience.com. Just looking at it says it's real, and for good reason. Zygotes new heart was scanned from a real human and textured with real images. Thats right, medical technology today can digitally scan human organs. Of course, theres a little more to it than one might think.

The CT-Scan, or Computed Tomography imaging, also known as CAT-Scan (Computed Axial Tomography), has been around since the early to mid 1970s. The CT technology is simply a highly refined x-ray machine that is mounted on a ring and circles the patient who lies down on a platform. As the ring circulates, approximately 1000 x-rays are taken per 360 degrees of rotation. The apparatus then moves down the body a few millimeters and circles again. Each 1000-photo set is subdivided into channels and reassembled into a single image, or slice, by the computer that drives the CT-Scanner. In this way, medical technicians are able to capture images of an organ such as the brain or human heart, slice by slice, so that the cardiologists and neurologists can look at the organ and determine what is wrong.

There are other types of scanning techniques such as MRI (Magnetic Resonance Imaging), NMRI (Nuclear Magnetic Resonance Imaging), and MRA (Magnetic Resonance Angiography). The MRA is among the newest and finest resolution of the radial machines. The cost of a CT or MRI procedure can vary from $300.00 to $2,400.00 (and more), depending on the facility and the organ or system to be scanned. The output, while it can be a series of traditional looking black and white films, is more often simply a CD disc with all of the slices, or images in a self-extracting interface to be viewed by anyone with a home computer.

Technology has reached the level of being able to create a 3D polygonal mesh from the images. On a small scale, you can experiment with this type of procedure simply by using a flatbed scanner to scan some simple x-ray films, and then using the image in your Bryce terrain or lattice editor. You can easily come up with a jagged outline of, say, a bone. If you had enough x-rays of your arm, for example, you might be able to come up with a series of cross sections that can be stitched together in a more powerful modeling program. But what do you do with really complex images that have cavities and voids, different densities of tissue, and different shapes due to movement (beating of the heart) from one slice to the next? What about contrast changes in the images when the medical imaging technician changes the magnetic properties and re-aligns the cells in a way that changes the black and white and graytones? Not surprisingly, there are several companies that have developed software which take these slices and extract the white through black graytones and create a 3D poly mesh.

One practical use of the mesh built by these data extraction software packages, is the output in a format that can be used by rapid prototyping. Rapid prototyping is a device that can read the shape of a mesh and build a real life (small) model. Most prototypers work with models under 12 inches, and are either the additive or subtractive type. Additive types deposit plastic on top of itself until the buildup looks like your 3D mesh. Subtractive takes a block of wood or plastic and uses mechanical tools to reduce or sculpt away areas that dont belong. In either process, a basic heart can be built and used to mold more. However, this is only an outer shell and generally not anatomically accurate enough for medical science. The real precision comes in on the computer level, where computer engineering can take advantage of the highly accurate mathematics of a central processor unit.

Still, the poly mesh created from a CT or MRI is, at best, very rough. While it may be anatomically accurate, it is nearly impossible for a computer to smooth out the mesh, thus also making realistic renders nearly impossible. The real difficulty is in determining the difference between a growth, or abnormality on the organ, and simple roughness in the scanning resolution. Thus the resultant mesh of a human heart will resemble a human heart, but will not be accurate enough to study in fine detail.

Another problem arises when attempting to obtain CT-Scans of organs for 3D projects. Unless you are using your own CT-Scan or MRI, there are privacy issues under U.S. HIPAA (Health Insurance Portability and Accountability Act) laws. Hospitals just cannot hand out CDs of peoples hearts, brains, and other organs for modelers to experiment with.

So, the team at Zygote came up with the perfect plan. They hired a healthy male in his 30s to undergo a simple CT-Scan of the entire body to begin the process of creating a virtual human heart. Thus began the many tedious man-hours of technical and artistry effort to convert the CT slices into a useable model. The Zygote process was even able to interpret calcium deposits, isotopes, and anomalies, and distinguish them from resolution aberrations.

The modeling process of the heart began with a poly mesh produced from the CT slices, and an assembly software from TGF Software. The result was a crude poly mesh. From there, the data was analyzed and used as a template to construct a new model, starting with a cube that smoothed out the resolution issues. The process proved quite beneficial in determining the thickness of the membranes, etc.. For example, the thickness of the ventricle muscle wall is different between the right side and the left side of the heart. This information is valuable to companies, such as those that produce implant devices. The real skill that Zygote brings to the modeling process is building an optimized mesh that renders well using the template for spatial information. Once completed, the mesh represented an anatomically accurate heart.

Courtesy of 3DScience.com

The next part of the process was the UV and texturing. This is where it gets really interesting. Something as complex as a human heart really cannot be textured with procedural shaders. However, photos of the actual colors can be mapped to the polygon model. But where to get photos of a human heart? The team at Zygote utilized all of the resources of cadavers and useable reference imagery already in existence to get started. They also learned that a Bovine heart had nearly identical surface textures and color as the human heart. As such, the Zygote heart is painstakingly texture mapped, polygon by polygon, with all of the above.

Courtesy of 3DScience.com

The next step was the meticulous creation of the morph targets. Using a variety of the most recent scientific research papers and data, Zygote determined the precise muscle contractions and tissue positions based on cardiac cycle data. This is where the Zygote medical expertise brings an astonishing level of realism to the process.

But, to appreciate the animations of the heart in all completeness, one more step is needed; thats the creation of cuts to allow viewing of the heart interior, even while its animating. Separate UVs and texture maps were created for the planar intersection of the cuts. The cut reveals a fleshy surface, as if a knife had been used to cut a 3D solid.

So, now that the world has an anatomically accurate and animateable heart with real textures, what can be done with it? The uses are really unlimited. Biomechanical companies can use the model to engineer their mechanical products that consist of life extending devices. Universities and training facilities can provide a very detailed and accurate model for teaching purposes. Medical Animators can visualize a variety of pathologies related to the heart, merely by manipulating the core model. There really is no limit.

Zygote has already debuted their medical models in the ABC show Miracle Workers, an IMAX movie, and Time Magazine. Even the U.S. Army has picked up some of the visualization models produced by Zygote and available on 3DScience.com. Not since the famous anatomical illustration series of the late Frank Netter, M.D., has any artist or company come up with such detail and accuracy.

The future for Bryan Brandenburg and his vision of medical and science models lies in the word more. More models, more organs, more detail, and more accuracy. On the horizon is a new set of male and female 3.0 models with true-to-life organs.

Courtesy of 3DScience.com

The visualization of science has been Bryans dream for the last 6 years. Bryan said, I have always felt there was a need to have science models and animations in 3D. Its the best way to visualize the invisible. The number One Artist is God in my opinion, however you understand this word. 3DScience.com introduces people to the elegance of life and creation, regardless of their spiritual or scientific background. Since the official launch of 3DScience.com at the 2005 SIGGRAPH, the company has produced over 100 models combined into approximately 1000 products on the site. Next up on the list of items to produce are products like stem cells and carbon atoms. Retro viruses and influenza type models are also in the works. Bryan said the best selling models right now are the Anatomy Collections, DNA, and Human Somatic cells. Molecular biologists are turning out to be a great market for the products, and are embracing the delicate blend of art and science.

Courtesy of 3DScience.com

So how does this effect the hobbyist and entry level 3D enthusiast? Quite simply, its an opportunity for the science enthusiast to produce excellent scientifically accurate images with affordable software. Zygote has a great partnership with e-Frontier, and is committed to producing Poser-ready science models along with the 5 or more formats it currently supports for 3D professionals. The Poser 6 shader engine is quite advanced and capable of producing renders that look like they came from more expensive programs. Consequently, Zygote is also releasing biomedical and science content specifically designed for e-Frontiers Shade software.

Roger Clarke points out that there are a host of opportunities for the 3D science enthusiast. Poser, for example, has better content support, and handles the human figure better than many other applications.

Bryan Brandenburg put it succinctly when he said, Every person on the planet is interested in the human body. The Poser user can produce a wide variety of items, and there are many business opportunities to be had right in their local community. Perhaps a new set of bone and muscle posters or stationary for the local chiropractor, or an outline of the human body on a custom intake form for new patients. Local attorneys may even benefit from having illustrations to show in their upcoming arbitration or trial.

The horizon is there, the bar has been raised. It's now your turn to seize the opportunity!

Be sure to check out Bryan's book
in the Renderosity MarketPlace!

All supporting images are copyright, and cannot be
copied, printed, or reproduced in any manner without written permission from 3DScience.com.

file_236639.jpgEric Post [EricofSD] is a Renderosity Front Page News Staff Columnist.
Visit Eric's Renderosity gallery
April 10, 2006

Article Comments

RDL ( posted at 12:00AM Mon, 10 April 2006

As a retired life science teacher I commend the work done on this product. Too bad that it was not available when I could have used it in the classroom.

Ace10 ( posted at 12:00AM Wed, 12 April 2006

This article is very interesting and well worth the read.

TCopsey ( posted at 12:00AM Wed, 12 April 2006

How awesome! From reading the article, the uses seem endless. Congratulations 3DScience making the dream a reality and I applaud you for sticking to the vision! Great article!