Augmented Reality is more than Virtual Reality

Contact lens displays

 

The quest for the most minimized near-eye display hardware already brought up some speculations about building displays into contact lenses. Rumors have it that even research projects have been started in this direction, sponsored by defense funds (probably journalists mixed up things a bit). Let's have a look at this topic (this is entirely new and not yet contained in the 2nd edition of the book!):

Seriously, given the many problems contact lenses already have in everyday applications, it's hardly conceivable how this could work in the battlefield. First, they aren't supposed to be worn more than 12 hours for various reasons. Then, even though you may even swim with contacts, even with open eyes at least in freshwater, if water happens to hit the eye with the contact lens still in tear liquid - a sudden splash for example - the lenses could be washed out.
Vice versa, you drink too little, you dehydrate and the contacts may simply fall out of your eye. Imagine a soldier fumbling around with liquid trying to clean and reinsert his contact lens. Dust and sand a also create problems. Dirt in the eye is much more difficult to get rid of, if contacts are involved.

Apart from this, it is not only a technical but also a major physical problem to design anything that generates an image in the eye while sitting right on the pupil.

This doesn't seem to bother some people, there is even an issued patent, dated 1995 (!) about a contact lens display, (US5682210) that carefully avoids even the slightest remark on how to form an image with this device. Did the applicant have a clue about optics at all, or did the patent officer? Imagine someone from 1492 applying for a patent for a car, without even mentioning that it would need a motor...
Meanwhile, actual '
contact lenses' with some electronics inside have been demonstrated. The wording of the newsflash ("with micro scale components that automatically bond to predetermined receptor sites") however implies that his is rather a misunderstanding about what's actually a retinal implant. The LED array contained in it could not form an image if this was really a contact lens...

Nevertheless, let's do some hopefully more reasonable speculations about the options we may have.

The most straightforward approach of course is sort of a simple display, with many little light sources. But in order or form an image on the retina, the light has to come in approximately parallel, as it gets focused by the eye's lens. Small light sources however do not emit anything parallel. Any single source, the smaller it gets, is emitting a more or less spherical wave. This way we wouldn't get anything like a picture inside of the eye.

Remember: anything being able to focus a light beam sufficiently as to form a crisp pixel has to be 1...2mm in diameter, at least. 2mm would be very crisp (1 arcmin of resolution) but probably not so useful, as the pupil may become as small as 1mm in sunlight and then it's just wasted effort.
Nonsense (left) and SciFi approach (right)

The result would be quite different, if all light sources were in phase. Then they could all together form a beam that could create an image by scanning, or a light field that would create a complete projected image all at once. For a first try this could be a Fourier hologram, only delivering images appearing at infinite, but affordable to calculate. Modifying this for different image distances would still be possible with limited effort, and if we did a dynamic focusing with this approach, it could work just like in the 'classical' vision simulator, adapting to the actual items the user looks at. So full fledged holograms, possible but computing intensive, would not be necessary.

Just using many tiny laser diodes however, would not deliver any global phase relation at all. What we need would be a single laser light source, whose light is then modulated by many single elements distributed right in front of the pupil.
The modulation we would want is phase modulation, which could for example be accomplished by liquid crystals changing their refractive index according to voltage as in some existing LCOS displays, or by movable micro mirrors modulating the length of the optical path.


Moving mirror phased array scanner (left) and Fourier
Hologram (right) using a single coherent light source

Other varieties, with partial reflecting or holographic mirror elements and phase modulator cells, e.g. LCD

Apart from the difficulties of miniaturizing something like this far enough to fit into a thin and hopefully still flexible contact lens, the modulating elements would also affect direct sight, which is of course not intended.
One way of retrieving direct sight is inserting large gaps between the light forming elements. This may work quite acceptable, even if some moiré effects could occur within the projected image.

Shaping of light is not the only challenge with a contact lens display. A major problem is providing energy. Nobody would want a wire going to the lens.
One alternative would be solar cells, powered by environmental light or by infrared emitters close to the eye, or by an external laser source that could concurrently deliver the in-phase illumination for our image forming elements.
Even with the latter approaches, we would still need some frame, helmet, or glasses construction to hold the energy source. It may be questioned where the advantage against 'classical' display glasses here would really be.
The external laser source variant could revive the wig display joke from the book....


The future: ACME [28] 'Hairstyle' display

Problems don't end here: We also have to provide signal processing for the display, and this can be quite heavy (pun intended). Providing a scanned beam would be rather simple and would also have the known advantage of scanners, being able to follow the eye motion without delay. The other method, creating a full parallel light field, would need a lot more computing power, even though we could mostly rely on creating a Fourier hologram.
It's obvious that for any foreseeable future, most signal processing would have to occur outside the lens.
Next point is signal transmission: the display has to be provided with an image of whatsoever kind. This could be modulated onto the energy transmission for example, hence on light, which would deliver the best signal bandwidth anyway.
Still then we need eye trackers to know where the lens bearer looks at. In theory, microscopic cameras inside the contact lens could do this, by looking just where the eye looks, and they could also concurrently deliver orientational data. Nevertheless it would have to be decided which signal processing should or could occur inside the lens, or which should be done outside. Finally, a contact lens could rotate, or be displaced a little, so its position against the iris would have to be known, detectable perhaps by tiny cameras or light sensors inside the lens border, looking towards the eye...

By now it's getting pretty obvious that for a realistic approach, lots of components and processing would still have to remain outside the contact lens, hence the question about realistically conceivable advantages of this approach over display glasses remains wide open.
Completely missing here is the necessary mask display, currently still requiring glasses in any variety.
Which doesn't exclude that this could lead to something useful in the future, and trying to develop the apparently impossible may always lead to some useful results unforeseen.

In a more remote future, we may well be able to build a volume hologram display that could be fully modulated in three dimensions, perhaps consisting of transparent electrode layers with liquid crystal fluid in between, perhaps offering the possibility of regrouping light beams and forming virtual images right out of the incoming light. This could maybe even eliminate the need for a mask display.
We have to take into consideration however, that environmental light is not coherent and not monochromatic at all. We know holograms forming 3D color images just under the illumination of a halogen lamp, but they still depend on a small light source and still do exhibit some blur.
The regrouping of light beams necessary to form a new virtual image from incoming light of another, is principally conceivable. If HOE's can split beams, they can also merge them. The problem is, ghost images will also arise, resulting in both false images and loss of efficiency.

More promising could be an approach that first reflects certain beams, constituting a mask display, and then forms new image parts from an extra light source.
Reflecting a certain light beam of a certain color and direction is simply done with some straight layers of different refraction index, perpendicular to the beam.


Holographic mirror structure (left), several overlayed (right)

Then overlay many such structures, one for each direction, into a complete HOE structure. For a sufficient very high directional selectivity, these elements would also exhibit an extreme color selectivity. We would have to combine many of them, one for each wavelength, to complete the job. And all of this would have to be a dynamic three dimensional structure of several thousand pixels in each direction, billions altogether.
We also have to consider that such a display, in order to modulate all of the incoming light, would have to be as large as the pupil can get, hence up to 9mm in extreme cases (a diaphragm could be used to limit this a bit, of course, but how much degrading of natural sight would we want to accept?).

Hard to say which will be 'easier' to accomplish: this one, or a full featured brain interface...


Rolf R. Hainich, October 3, 2008

 

 

 

home        more notes        order

 

 

 

 

 

 


Copyright © 2006-2011 Rolf R. Hainich; all materials on this website are copyrighted.
Disclaimer: All proprietary names and product names mentioned are trademarks or registered trademarks of their respective owners. We do not imply that any of the technologies or ideas described or mentioned herein are free of patent or other rights of ourselves or others. We do also not take any responsibility or guarantee for the correctness or legal status of any information in this book or this website or any documents or links mentioned herein and do not encourage or recommend any use of it. You may use the information presented herein at your own risk and responsibility only. To the best of our knowledge and belief no trademark or copyright infringement exists in these materials. In the fiction part of the book, the sketches, and anything printed in special typefaces, names, companies, cities, and countries are used fictitiously for the purpose of illustrating examples, and any resemblance to actual persons, living or dead, organizations, business establishments, events, or locales is entirely coincidental. If you have any questions or objections, please contact us immediately. "We" in all above terms comprises the publisher as well as the author. If you intend to use any of the ideas mentioned in the book or this website, please do your own research and patent research and contact the author.