- Q: How can passive markers be used to meet current limitations with building signage for interior wayfinding?
Navigating hallways to find a particular room can be very challenging for building users, particularly visitors unfamiliar with a building’s layout and even more so for individuals with physical limitations or other disabilities. Although wall signs are required by the ADA to be consistently located on the latch side of a door at a height off the ground of 48 to 60 inches, wall signs that are flat against the wall require a building user to walk back and forth and visit each room before knowing where a room is located. In addition, wall signs provide very little information beyond a room number. In cases where more information is given, such as a room function, the ADA does not require this information to be fully accessible. For instance, a sign with “110, Lunch Room” may provide the room number “110” as raised characters and in raised braille for the blind, but only provide “Lunch Room” as flat printed text for sighted individuals. In addition to these lapses in braille information, little or no research has been done on actual usage amounts of the braille signs. Limited or no immediate wayfinding details are provided by wall signs either, requiring individuals to additionally navigate a building to locate a more general building directory. This is especially inconvenient for those with disabilities if the individual must travel back down from another floor to find a directory in the lobby.
A passive visual marker is a marker that requires no power, it is usually printed or engraved to provide visual cues to a human or to a computer program/camera that can interpret the marker to find a position in space or to relay some information about the environment. Some passive markers can be designed to communicate information to humans and computer software. There are others that are not human readable and require a decoding algorithm to be useful. Passive markers could be an excellent augmentation of existing interior wayfinding signage as they are robust, inexpensive, and the mobile phones with cameras capable of detecting them are becoming more common. Existing research on passive marker types and findings from tests of these markers have important implications for building signage in meeting the current limitations discussed above. Existing research on passive marker types and findings from tests of these markers have important implications for building signage in meeting the current limitations discussed above.
- OVERVIEW OF PASSIVE MARKERS
Four marker types we researched are presented in ascending order of information contained by the marker.
- Color Markers
Description: Color markers use color filtering and shape filtering (circle or rectangular for example) to communicate a position and type. These markers can also be used by sighted individuals without access to any mobile camera device as the code can be simple, just a specific color or color and shape.
Advantages: These markers are extremely simple. A filtering algorithm is used to extract specific ranges of color from an image. Making the color mark a simple shape allows for filtering out random noise in images of similar color (eg. only looking for yellow circles).
They can be processed with inexpensive cameras and phones as focus in not needed to see a color. Movement will blur the shape but the color will be identifiable as long as the camera can see it at an appropriate speed for a frame.
Limitations: Because there is no further data embedded into the color it is only useful for identifying a position.
There is the possibility of having many false positives if the color is a common one or if there no shape matching or minimum size filtering as well.
- Image Based Markers
Description: Image based markers use complex feature detection algorithms to turn a specific rectangular image into a set of data that can be recognized as a marker in 3d space.
Advantages: Allows use of user generated markers that have a meaning when viewed even to sighted users without a mobile decoding device (eg. phone, tablet, or laptop).
Allows for personalization of markers.
These markers can communicate more effectively than color based markers to users with no mobile devices.
Limitations: Feature detection to generate the markers is a preprocessing step in most cases. This would make it difficult to use them in a distributed system where a knowledgeable person would most likely be needed to make the image into a marker.
Not all images can be used since the image must not be a repeating pattern and must have distinguishable features that the algorithm can locate.
- Binary Markers
Description: Also known as fiducial markers, these markers use black and white squares in varying patterns to make detection fast. They rely on edge conditions being contrasting and clear. The markers usually only embed their specific ID and no other information. Because they are seen as frames to the detection software their rotation, scale, and perspective in space can be known and used.
Advantages: These markers are easy to detect as they rely on large rotation invariant white and black squares.
Markers will be easier to distinguish while the camera is moving.
Because the Qualcom markers are merely frames, the interior image can be personalized easily.
According to the documentation, these markers can be much smaller relative to the image markers.
Limitations: The Qualcom set of markers has a limited number of 512, so the max number of unique signs is limited. This implies that the software will need an input or way of determining different buildings to attach the marker ID to a set of data in a database.
- QR Codes
Description: A square shape code made of black dots arranged in various patterns on a white background. These codes embed plain text and ascii symbols.
Advantages: There is a standard QR Code that can be scanned and read by all existing mobile devices with cameras, as long as that camera includes an auto focus. The codes are free to use, they are patented but rights are not enforced.
Decoding is fast and takes little processing time even for relatively long codes.
Lots of characters can be embedded into QR codes, upwards of 1000. Codes can also hold as few as 4 characters to make scanning them easier. They get larger and less dense the less information they hold.
They do not need to be attached in a database to any particular ID as they can contain a web address or any information.
They can be generated by new users using many free applications or web based tools, like Qrafter.
Limitations: QR codes cannot be scanned while moving; the markers contain too much data to appear crisp and clear to the detection software when they are not focused on clearly. Moving the camera causes the edges to blur and scanning cannot occur until focus is reached.
QR codes need to be very large to be scanned from afar. Even 150 characters creates a dense image that must be very large. A 3.5” square was able to be scanned reliably from 2’.
We built a series of basic machine vision algorithms to track color markers using the machine vision framework Roborealm. Our first tests only tracked a specific color of a minimum value and hue (red in this case) and performed no other filtering of any kind. These failed in different lighting conditions as the color values of the same object change under different lighting conditions. Another failure point is the large source of error as in any given scene various levels of all colors exist.
To filter more accurately, we added a rectangular background of a highly contrasting color and first filtered for that background color. We next filtered for the color marker foreground color. The next step was to check where these two colors overlapped (http://www.roborealm.com/help/Blob_Overlap.php). Then, a circle filter was used to eliminate any residual noise that might have appeared to be overlapping.(http://www.roborealm.com/help/Circles.php)
In our tests, we could identify these circular two color markers from 30 ft in good lighting, and 15 - 20 feet in uneven lighting. The marker was printed on 8.5x11” paper and the red circle in the center was 4.5” in diameter. Perhaps more importantly, the algorithm is fast and can reliably detect the circle even while the camera is being moved. This is important for a mobile phone application.
Binary or Fiducial Markers
We performed tests on the AR TOOLKIT implementation of binary markers to determine sizing, distance, and camera movement limitations.
In our first set of tests, markers were printed centered on 8.5x11” white paper. The markers were approximately 5” squares, with a 1” white border. The contrasted edge is important for the algorithm to detect the marker.
- Using the embedded webcam on a 2010 MacBook Pro, the markers were detected from 2 - 10 feet while moving, rotating, and in various states of corruption. As predicted the markers could be detected while being moved or drawn on. The markers could not be detected if the vandalism occurred on the edges enough to disrupt the sharp corners of the frame. One marker was also successfully turned into another maker by filling in the squares of the pattern with pen the same color of the marker.
Another set of tests using 5 inch markers in the even-lighting condition of the Health and Sciences library found that the markers could be seen from 30 feet away if the background behind the marker was static. Using the same set of markers inside a darker and less evenly lit condition, we could not reliably find the markers from more than 8 ft away. Therefore, there seems to be a large variability in detection when the white border is disturbed, the lighting uneven, or when possible shadows or glare effect the detection of the contrasted edge of the marker.
*QualcomVuforia comes standard with 512 unique markers.
*TheARToolkit Pattern Maker tool purports to make an almost infinite unique series of markers that can be imported as training data into recognition/detectionsoftware. (Note: this implementation has not been tested on mobile devices and we have only been able to find pc detection software so far, though it’s probably just a matter of no one having a need yet.
We did a small set of tests on QR codes of varying sizes and data densities. We tested 150 character and 4 character QR codes that were displayed on a laptop monitor at 3.5” square. The four-character code could be scanned successfully and reliably from 4.5’ using an iPhone 3gs and Qrafter software. The 150 character code could be scanned successfully and reliably from 2’ using an iPhone 3gs and Qrafter Software.
The higher density of the 150 character code required that each square that comprises the code be smaller. This causes the cell phone camera to lose focus at a greater distance as the resolution remains constant.
We also tested a small, high density 1” square, 991 character, 157 word QR code that could be read from 2.5” away. This small size, high density, and close distance all made reading this marker difficult. It was read successfully, but not reliably when the camera was in approximately the same spot. Increasing the size of the marker would, by relationship to the focal length of the camera, increase the distance the marker is viewed at.
This is just above the minimum focus distance of the iPhone3gs camera. All of these tests required that the phone be held still and close to parallel with the QR code marker.
DISCUSSION AND DESIGN GUIDELINES
Designing with Image Based Markers
Detecting image based markers requires a set of feature data to be loaded as they are specific to individual unique images, and determining the physical location of the markers (angle, rotation, distance) requires longer processing time and a clearer image. This necessitates a better camera and more powerful mobile device.
Image based markers provide a way to create markers that are meaningful even to sighted users or users without a mobile device. There are guidelines and rules for creating image markers that will work reliably. They take more time to process and better cameras to identify so newer devices would be needed. They also take large amounts of memory and so a limited set can be stored to identify without going online and getting a new set of data.
Because of these limitations, a system using image based markers for room signage might work better as a room classification system.
Designing for Reliability
To build a marker based signage system that works reliably, multiple types of markers could be used together to provide varying densities of information that would work reliably from different distances. The inclusion of color-based markers could work well for identifying potential locations where more information would be located.
A specific color can be tracked even while moving a mobile device very quickly as the color is not distorted by the motion of the camera. These color markers could alert the user that a room sign has been spotted. If all color markers are a specific size then that size can be used to make an estimate for distance or relative distance from other markers.
The color marker could be used in conjunction with binary or QR code based markers. The binary marker could also be at multiple scales, repeating within itself so that it can be viewed from far away or close up to receive the specific room number. Finally, a QR code could provide more information.
A unique configuration of bright colors of a color-based marker is not only reliably tracked by a mobile device, but are also easy markers to spot with the naked eye. The strong contrasts makes wayfinding for those without a mobile device still easier, particularly for a visually impaired individual. In our prototype we used two shapes of contrasting colors that used to eliminate false positives. There was still some false positives and misses because of variable lighting levels. Research conducted at Smith-Kettlewell Eye Research Institute proposed a color marker system using relative lighting variances in multiple colors to produce a robust and fast detection algorithm. Their marker system does no shape tracking so it is very fast as it is only color based. (Coughlan, Manduchi & Shen, 2006). Another study by Manduchi, Coughlan & Ivanchenko on a camera phone wayfinding system showed that visually impaired users were able to reliably find color markers (2008).
Designing for a Dynamic Database
This implementation requires a central database and some organized authority to create the database, host the information, and facilitate information input and update.
Using a binary marker system, building users would be required to input their current location explicitly stating what building they are in. The building can be selected from a list of recognized buildings that relate to their current GPS or IP address location or obtained from QRcodes located on the building exterior.
By entering a current building, a limited set of binary markers can be used to identify different rooms in different buildings. This allows the same marker to be used in two buildings but when accessing the database it can pull out different information to present to the user regarding the specific room they are looking at.
Designing for Dynamic User Generated Content
This implementation would require individual room users to generate all content. The QR codes are easy to create using free software.
A binary marker system can be used to identify room number and set of user generated web pages or relevant room data. This system could work by requiring the user to photograph/scan a QR code that is generated by the users of the room. QR codes can hold web pages or just vanilla text data. The binary markers could then be used to identify what rooms are currently visible to the user's mobile device. Then, the user could move closer to the desired room and scan the QR code for that room. One possibility in facilitating locating the QR code is using a slot built into the sign that would place the mobile device into the correct location to scan the code.
- BUILDING SIGNAGE ANALYSIS
- DESIGN PROPOSALGoals1. Provide building users with more information2. Allow any individual easy access to that informationTo meet these goals, we followed our own research-based guidelines for Designing for dynamic user-generated content and Designing for Reliability (discussed above) in developing a wayfinding system with passive markers.
- Signage Prototypes
The two types of signage we used and the size of each marker and information they provide were determined based on our research findings discussed above. Only a rough idea for laying out the markers was needed to test for reliability. Material and other design considerations for the signage itself need to be developed further.
- Smartphone Wayfinding App
We further developed our idea for a wayfinding app that would identify these passive markers by designing a series of user interfaces for each step of the wayfinding process with them.The user may choose to use the app to locate a projected sign, flat wall sign, or either type of sign. If the app is able to pick up both binary markers and color markers from any projected signs, it would list both room numbers and room types. These would be listed in order from closest to furthest distance from the user. The user may also choose to select a room from the list of those identified to keep the camera's focus on that particular marker until that destination is reached.A vibration and/or tone would provide the user with feedback that the app did in fact identify a marker. All text provided to the user visually would be available audibly using the accessibility functions provided in the general settings of the smartphone.
ADA Accessibility Guidelines for Buildings and Facilities
Coughlan,J., Manduchi, R., Shen, H.: Cell phone-based wayfinding for the visuallyimpaired. In: 1st International Workshop on Mobile Vision, Graz, Austria (May2006)
Manduchi,R., Coughlan, J., & Ivanchenko, V. (2008). Search strategies of visuallyimpaired persons using a camera phone wayfinding system. In ICCHP '08Proceedings of the 11th international conference on Computers Helping Peoplewith Special Needs. Berlin, Heidelberg: Springer-Verlag.
Mulloni, A.,Wagner, D., Barakonyi, I., Schmalstieg, D., Indoor Positioning andNavigation with Camera Phones. In IEEE Pervasive Computing (Special Issue onSmarter Phones), Apr-Jun, 2009.
Powered By String, Augmented Reality User Guide
SoftwareDevelopment Guide Qualcom Vuforia
U.S. Department of Justice: Americans withDisabilities Act