Augmented Reality

UW - Platteville

Augmented Reality

Penn State Web 2009



Presentation and documentation are online @

http://www.uwplatt.edu/web/presentations

Augmented means...

C Major

Augmented Triad B

Most people have heard about Virtual Reality, and everybody has heard of actual reality.

  • Greater than before
  • Increased
  • Amplified
  • Improved

So Augmented Reality is...

... supplementing, enhancing, modifying, improving our reality.

Examples today!

  • Heads up display
    Heads up display
  • NFL - first down marker
  • NHL - puck marker
  • Museum tours
  • Augmented reality is a mixture of the two and is an exciting technology that holds many possibilities for use in day to day life.
  • Heads up display in automobiles provide additional information regarding the vehicle, location, direction, areas of interest, etc.
  • NFL has a first down marker which shows up yellow on TV, but not when in the stadium.
  • NHL uses a red line to mark the motion of the puck for TV, to make the puck more visible for the audience at home.
  • Museums are using more dynamic listening devices that are not guided tours, but guide-less tours. Wander through the museum to points of interest and the device recognizes your location and begins to provide additional information.

AR: Computers

AR can be achieved today, using simple technology. All that is required is:

AR headset
  • video display
  • video camera
  • speakers
  • microphone
  • In essence, basic input AND output

Though the devices required seem to be 'exotic', it is actually very standard. A fancy headset is not required. Most of us have these technologies with us daily... a simple cell phone!

New Tech on the block

Tuesday November 3, 2008, Presidential Election night, CNN revealed a 'new' technology, which they called a "hologram".

Will I. Am became the first "hologram" broadcasted on television.

Will I. Am

Not a "Hologram"

CNN tried to simplify the example about what was going on.

Immediately, I knew what was going on... and it was Augmented Reality.

Frommelt Twitter

CNN explains their "Hologram"

It wasn't a hologram. People in the news room that night saw nothing, save a marker on the floor.

The marker was the key. It told the computer where to place the supplemented information on the display.

CNN explains their hologram

Markers

Markers are unique and asymmetric. The video stream is scanned live looking for the unique pattern.

As a marker is recognized, an augmentation is overlaid in it's place in the live video.

Penn State marker

AR Processor

The image must be processed quickly, because of this, there is a threshold of certainty.

The simpler the pattern, the more reliable the threshold.

Penn State active marker

The level of certainty can be adjusted to accomodate for faster renderings, but with less accuracy.

Virtual Reality

Now that a marker is identified, the video can be modified based on user input.

Virtual reality is perfect for this type of augmentation.

  • Computer Aided Design (CAD) programs
  • Annimation Modeling programs
  • Virtual Reality Modeling Language (VRML) - tutorial
  • 3D Rendering programs - Blender (free software)

AR: Challenge for the Web

Initially AR was designed as a stand alone application at Univeristy of Washington's Human Interface Technology (HIT) Lab - ARToolKit

The application needed to be installed and configured on the client machine.

This is NOT acceptable for the standard web user.

The solution: Flash based AR - FLARToolKit

FLARToolKit

Flash has the ability to access the video camera and microphone.

Papervision3D 2.0 library can display 3D images in Flash.

The ARToolKit was ported from Java into Flash.

As a marker is identified, Action Script will attach 3D objects to the video.

FLARToolKit is AS3 version of ARToolKit. But it is not ported from the original C version but ported from Java version which is called NyARToolKit. (NyARToolKit seems to executes much faster than the original C version after the great effort of nyatla)

FLARToolkit will detect the marker from input image and calculate the camera position in the three dimension space. Something like Helper library are planned to add but further processing like synthesize the 3D Graphics needs to implemented by yourself.” This abstract was taken from the FLARToolKit website located at http://www.libspark.org/wiki/saqoosha/FLARToolKit/en For displaying the 3D images FLARToolKit uses the Papervision3D 2.0 library.

FLARToolKit Usage

  • When FLARToolKit is downloaded, there are sample files installed.
  • There is a sample marker along with the supporting library files.
  • Print off the marker, turn on the camera, and test the sample: 'simple cube'
  • The webcam window will be launched and the ported ARToolKit will look for the marker.
  • The exported Flash movie can then be embedded into a website, giving the user SIMPLE AR.

The FLARToolKit demo that we took to study off of was written in actionscript as stated above. The goal that we tried to achieve was to import our own objects into a flash based AR. The simple cube demo allowed us to do this. It had the simple functionality of being able to detect a marker and display a 3D image on top of it. There were only a few places needing adjustment to customize it for our own objects and markers.

To create our own marker we use the pattern creator provided with the C version of the ARToolkit. It launches a webcam window and tries to locate a marker inside of the image. Then all we have to do is click the mouse button once the marker is located in the window and give it a file name. Once the pattern is created we just need to replace the pattern name in the actionscript.

Replacing the 3D object is as simple as creating an object from our exported 3D model, explanation on that below, and adding it to the base node.

Custom Virtual Modeling

Blender can export 3D objects, or 3D movies.

Steps

  1. Create the model of the object in Blender
  2. Export using ActionScript plugin for Blender
  3. Move the generated AS file, and import it into the master AS file
  4. Create and attach the object to the baseNode() in the master AS file

Implementing objects

  1. Create model in Blender
    1. Model using primitives or vertexes if more detail were wanted.
    2. Join objects together so exports as one object
    3. If texture is needed unwrap using smart projections
      1. Export texture layout, add textures to faces in editing software.
      2. Open image in uv image editor window in blender
    4. Export object using the Actionscript plugin in Blender
      1. Leave the package name blank, give it a class name.
      2. Select Papervision3D 2.0 from dropdown box.
      3. Select a location and click export.
  2. Move generated .as file into folder with main actionscript file
  3. In main actionscript file under any imports type import *.as; (* denotes class name)
  4. Create an object of the imported class type.
    1. The objects are inherited from the triangleMesh class this means they can be manipulated in the same way.
    2. Created a scaleAll() method to adjust the size of the objects when rendered.
  5. Add the object to the baseNode via baseNode.addChild(obj)

Custom Patterns

Make a black and white image (K.I.S.S.)

  1. Create a pattern file using the “C” language ARToolKit: mk_patt is the executable
  2. Webcam window pops up and will try to detect possible markers: red and green angles appear on potential markers
  3. Click the mouse button to capture the pattern
  4. Give it a file name and it is generated in same folder as executable file
  5. Replace the flarLogo.pat with your_pattern_name.pat

The ARToolKit works on Mac, not sure if it works on PC.

The file mk_patt is the executable.

Penn State AR

Blender Skins and Textures

Blender models are created with no lighting and no colors... being grey.

By "unwrapping" the model, it becomes a flat plane which can be textured.

Penn State texture

After the model is created texturing needs to be done. In Blender you are able to “unwrap” the model onto a flat plane so that you can apply textures to each face. This UV unwrapping can be tricky, but the best way to do it for our purposes was to select the object, go into edit mode (tab), and hit the “U” key, and then selecting “unwrap (smart projections).” That lays the squares out as evenly as possible. Then you can save the uv layout to a .tga file and make your texture off of that.

UW-Platteville Engineering Hall

Current AR Technology

The Future of AR

What's next?

Where do you take AR?

AR Resources

Questions ???

Augmented Reality

Augmented Reality



Presentation and documentation are online @

http://www.uwplatt.edu/web/presentations


Email: frommelt@uwplatt.edu

Copyright Information

Copyright Daniel M. Frommelt, 2009. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.