This microscope uses touch

Would *you* call it a microscope?
The sponsor is Jane Street. Find out about their Academy of Math and Programing here: janestreet.com/amp

I'd love to know more about the gel they are using, It seems absolutely incredible that a gel would conform to such small details so perfectly and yet have no memory in that it returns to it's original shape.

I could probably watch about 12 hours of Steve pressing various objects into the gel and just describing what they are before moving onto the next one.

The wierd neon-colored image you can see used to create 3D model is called a normal map and it's meant to hold information about height of the object using different colours for different light angles. It is used widely in video games to create detail in low poly objects by changing how light reflects off of the surface.
I find it fascinating they used normal map and not a regular height map but given the technology of 6 light sources it makes all the sense

It's actually nuts that you're getting such a deep depth of field on such small things, and you can even move them around a few millimeters and still retain focus. And you can capture it all with video and then share it with other people. Amazing.

This is insane. We are getting to watch the first steps of an entire new tool that will change entire fields of study. It is amazing. I'm probably way more hyped about this then I should

We could actually use that almost every day at work to measure engravings on the toolings for chocolate moulds. Would safe us a lot of time.
Super interesting 🧐👌🏼

As a 3D artist myself this was one of the most fascinating things I have ever seen in a long while. If produced massively across the globe. It could be really helpful in 3D Film Industry.

Was really wondering about how easily it was contaminated and how frequently you would need to replace the gel, how expensive is the replacement and how easy is the process? It looked like everything you had on there had some dust or hairs, it seemed like it wasn't like you were constantly scanning dirty pieces but instead that the microscope itself had issue. Incredibly cool technology, I even want one but the dirt issue seems hard to deal with.

This has WILD implications as a mobile highly accurate surface measurer. Damn, I want one just to have. I bet they're expensive. Industry really needs to have this. Feels revolutionary.

I would love a larger scale of this, it would be amazing for modelling small electronics where the manufacturer doesn't provide 3d files

I work with 3D making games for a living. Seeing those 3D models being created was so cool. Essentially using the same information that's used to generate and render normal and displacement maps.

So cool. Its like making a normal map of a object 😍

im a 3d artist and i want one of these to make instant normalmaps from objects ,these look great

Ok, Steve, I'm so glad that youtube thought to put your video in my path. That's an odd microscope indeed, and the gel is strangely sensitive. Removing color does indeed leave us with form, texture, and shadow. Photometric Stereo? Ok then. I wonder what insects would look like with this system? Does that gel wear out? So it has a quality control inspection use, also interesting. Jane Street Academy... sounds excellent. Thank you for all of this information. Well done. But then you already know that.

what a *fascinating* tool. I personally don't have a lot of use for it, but I can just imagine machinists being absolutely delighted at the details and measurements you can get from it.

Never knew such thing existed.
But this is really a valuable tool for a lot of people who do precision work.
So thank you for brining it to my attention it deserves that.

I work at a jet engine company and we use these all the time! So cool to see!

I imagine this could be incredibly useful for making normal maps for added detail in 3d modeling

I was expecting a video on an Atomic Force Microscope, but this is far more fascinating! What a novel way to collect 3d data!

Sounds like it could be a major improvement to a thumbprint scanner. I’m a mechanic and get cuts and callouses on my hands every day. My thumbprint usually only lasts 2 days on my iPhone before I have to rescan it.

This multi-light setup that generates the depth and normal maps is the same method that is used to generate many textures and material used in games and film. You can do it yourself with a camera, a light and either free software or Adobe Substance. You can also remove the reflection from the images using cross polarisation by putting a polarising gel on your light source and in a different orientation on your camera.

Steve: this needs a sound effect
Me: schlorp
Steve: schlorp

I could imagine a larger version of this combined with a traditional 3d scanner to very quickly generate texture maps for game design

This reminds me of when my dad, a master carpenter, taught me to feel if 2 pieces are level on a small scale with my fingers instead of looking because it's way more accurate.

This would be awesome to generate height maps for making smart materials in substance.
Just noticed the height map at 2:41 and wanted to plug it into substance.
Mistook height map and normal map as the same thing by mistake here. Thankfully people seem to get what I meant

We have a 3d scanner at work that works on a similar, off less accurate, principle of shining a light onto something and taking a picture. It is amazing how it will confuse printed text as raised or lowered surface. This seems to be a great workaround the normal "spray paint it all grey" method we have to use.
about a decade ago i worked with a charity to try and replicate a giant version of a medal. I tried photogrammetry to get a scan of a plaster cast of the medal (again, all uniform no shiny colour). it ended up terrible. Something like this would have worked amazing.

I'm now wondering: Isn't this microscope still optical since it uses light to create the shadows that form the picture?

This is like real life normal mapping. Super cool

That mystery item looks like a watch battery and the surrounding parts, with a lot of overlapping layers of parts held in by small screws, I'm going to guess it's a complex mechanism... so, a watch? The movement of an analog quartz wristwatch?
Edit: I realized after posting that I betrayed my knowledge of watchmaking with the terminology. For those who don't know, the "movement" of a watch refers to the mechanism, specifically the moving parts that make it work, the gears, the springs, and so on. In a quartz watch, it'd also include the battery and other electronic parts. The hands are also included but obviously not shown here.

I still saw a mountain even with the image turned upside down. Didn't see a crater-

I have 16 years as a US military aircraft structural mechanic.
I love your presentation style. It's light and engaging.
I have never seen our nondestructive technicians make a cast of an indentation. We use ultrasound, edy current, sometimes x-rays, and more rarely these days optical micrometers.
Never the less. I need one of these, I would love to compare it to currently accepted techniques.

Does the gel leave a residue behind to the point to where you have to refill the gel over time?

Got to play with one last year. They're 40k. We were using it to look for pinholes in our product, but I mostly used it for coins

There seems to be some limitations when used for surface features with high aspect ratio. For example, it wasn't able to discern the vertical fall off from the edges of the Lego brick pegs
But none the less very interesting and could be very useful in a lot of scenarios
By the way, what's the MTBF of the gel? (i.e. how many times can it be used)

I’m surprised they don’t use vacuums or air pressure to get the gel to conform more to the objects

Its incredible how well that gel conforms to objects

Nailed it! I recognized the battery from the mystery device after a bit of pondering, and then just guessed the most common application of that kind of battery.

My past 5+ years of EM experience using various electron detectors approves this video on a fundamental level.

A note on the ML side - the thing Steve refers to is called "top down" reasoning, where the later "high-level" layers representing object types feed back into the earlier "low-level" layers representing shapes and edges. This is fantastically computationally expensive, and is one indicator of just how sophisticated the human visual system is.

I've been following this company for many years. Their technology is highly interesting. I'm mostly interested for bio-inspired designs and collecting surface profilometry of scales, scutes, skin, etc.
A disadvantage of the method is that the surface of the gel degrades with use. Perhaps they are trying to improve this design, but I think it is probably unavoidable to some extent. That consumable cost is kind of expensive for casual use, thus restricting this method to only industrial uses.

The same technology is used in AOI machines worldwide to provide 3d models of the inspected board.
Such precision is used to find out if components were assembled wrong, have defects got dislodged while going through the oven.
Edit: those don't use gel though.

This would be an incredible medium to make some kind of animated movie, even just a short little thing. What i'm imagining is in the same vein as the "A Boy And His Atom" animation made with individual atoms.

It's surprising that the gel doesn't capture more bubbles of air

As someone who has taken apart watches, I knew exactly what that mystery object was 😂

Thanks for using an actual stereographic video and making it easy to cross my eyes to see that sea cucumber swim in 3D.

I've always had an issue viewing the crater of Neal Armstrong's bootprint on the moon. I always see the boot print as raised above the surface. From the knowledge gained in this video, I flipped the image 180 degrees, and voila, now I see the imprint instead. Amazing!

If you're interested in looking at other unique sensors, Event-Based Cameras (also called neuromorphic cameras or silicon retinas) are an interesting rabbit hole to go down. They use a unique approach to electro-optical sensing to get high fidelity of certain information in a very efficient manner.

That zigzag motion of the print head is "boustrophedonic". It is a literal reference to the motion of an ox ploughing a field, and you still usually see it in patents to describe print heads, scanners, etc.

Your prints are pretty good :D
Nice layers no gap, no loose belts, nice

Would probably be good for 3d meshes in video game development

Makes it look like some sort of height map you can generate from 3D objects

I don't know about a microscope, but we'd surely hope to see this tech being used in creating 3D models

This company presented us this technology at work, it was satisfying and playful to try 😀

With the crater illusion, I saw it as a mountain the whole time, even after seeing the shadows falling on it like a crater - the look of the inside of the crater looks so plateu like. Same with the indented text, in fact I think the top light source makes it stronger, and I did eventually recognise the crater, but I had to look closely at the shadows, and looking at the light side makes it look like a mountain no matter what.

5:00 Thanks for making the left/right arrangement compatible with the cross-eye technique! 😸👍

This is a gel sight. When you do analyze you can also create a 3d printable image enhancement.

The "crater" looked like a mountain to me no matter the angle. Even after you said it was a crater and flipped it, my brain couldn't un-mountain it. :D

If possible, it would be interesting to see an example of how the algorithm would interpret a color image. It would demonstrate the gap between how these algorithms interpret images compared to our brains.

This is the real life equivalent of a matcap or clay render in 3D rendering.

I work with lots of 3D LIDAR scans and one huge issue for us is the reflection off of reflective surfaces such as mirrors. Interseting how this gets rid of that, at least on a micro scale.

There was NO way i expected for that to be handheld! Thats NUTS

The crater illusion is tripping me out because of how inconsistent I'm finding it between people.
The crater image only ever looked like a crater to me, while many commenters had the opposite experience. Then with the indented lettering, it looked indented to me the first way, and like it was sticking out while Steve is saying it obviously looks indented now XD
That's the weird part to me.

We use similar methods in the game industry to extract height and normal maps using photogrammetry.

as a watch guy int interesting to see a miyota(owned by citizen watch company) quartz movement inside of a casio watch who can and does make their own quartz modules

Loving the expensive high-tech microphone mounting system you have set up! Hey it works lol

Nearly every mundane object under that type of imaging looks quite mesmerizing! I actually thought your stubble was one of the more interesting ones. It really shows just how cleanly the blades cut the hairs at the ends. The draping problem was the first thing that came to mind as someone who has done a lot of vacuum forming. But that’s very interesting that it’s not much of an issue as long as they can get the depth value. And as long as it’s not deeper than it is wide.

6:07 "This image looks like it's sticking out because the shadows are on the bottom, but by flipping the light source you see the reality that they're actually indented"
That was the exact opposite for me. The first instance was clearly indented while the second was and emboss.

This technology can be insanely useful for 3D artists, although not this one since it's a microscope

I think my reaction to your example at 6:00 brings up an interesting point. See, I DIDNT experience the crater illusion when you used the touch microscope, and I think it's because humans are VERY good at context clues and learning; Because you'd shown me other visuals of the touch microscope, I was able to tell from the patterning around the letters that a flat, intended surface was pressed against the microscope, and that the letters were smooth because nothing was touching the gel surface. If they really had been protruding from the block, it would be the letters that had texturing, not the background.

I'd love to see a record under this

Interesting, the images that the 3d scanner (I think that's maybe more apt than a microscope) puts out already reminded me a lot of normal maps, tech we use in games to cheaply render 3d geometry on a flat polygonal surface. Which actually look exactly like the kind of purple image you see at 2:42 . Funny to see this tech show up in unexpected places

4:37 That's called monocular depth estimation and it very much already exists. There are quite a few open-source models that can estimate depth fairly well from a single flat image, and if you have multiple images from different angles there are neural networks that can build up an accurate model of the entire scene, sufficient to move a virtual "camera" around and through the scene and create new views in the process.

06:55 On an aircraft "Traditionally you would create a cast of the scratch and then measure the cast in the lab" - I never saw that. We could use ultrasonic inspection or other forms of non-destructive testing in-situ to measure the depth of the scratch. In any case, best to smooth it out to avoid stress concentration and the risk of crack initiation - re-checking to ensure that the panel thickness is within tolerance or safe limits.

This would be great for looking at error coins under the microscope !!

wouldn't using combination of lights and cameras like infrared or other lights with different wavelengths can achieve the same function without using gels?

This could be an award winning video for the “oddly satisfying” category, i just can’t stop looking at it

Could it be made into a a 3d scanner for micro resin printing? Maybe. I don't really know.

This is honestly amazing! You're so lucky to be able to play with. Is there any information on whether this is going to be sold commercially? I'd love to get my hands on this.

So far my biggest problem with AI, is it seems to be the new fantasy we cling to about salvation. Also for all the problems ai creates the offered solution is often just more ai lol

The crator illusion worked opposite for me. Looked indented when lit from below and sticking out from the top

Great!! Anyone aware of any similar sensors from other brands that use the same technology?

put a sphere painted in vantablack (or something close) in that machine. i really want to see how the 3d scan turns out

I ran atomic force microscope (AFM) that used a tiny needle to bounce along the surface and generate images in the nanoscale.

IAs VFX artist I love that it creates normal textures. That´s a totally cool device for scanning hightmaps and normal maps. Especially for fabrics and general surface imperfections. Exciting technology

If you are interested in other touch-based microscopic devices, check out ATM (atomic force microscope).
And if you are already checking it out, take a look at STM (scanning tunneling microscope) as well ...
They are crazy devices

The gel seriously had me tricked that it was water. So glad I clicked on this video!

Just the first minute and a half makes this microscope seems interesting asf for me, since this is in my inexperienced short life (atm) very unique, interesting, and weird

when you were explaining the optical illusions of indented and sticking out parts, my brain works exactly opposite for some reason. the ones you called indented looked sticking out to me, and the ones you called sticking out looked indented. interesting...

Mould is on another level with his science videos. He always finds the most interesting and obscure stuff.

Steve. Look up NeRFs! You can do 3d photogrammetry and capture reflective objects. It's a bit different than you described doing but still relevant I think. Unless you mentioned them and I missed it.

Metrology has to be some of the most fascinating disciplines

Coolest technology I've seen in years. Makes you think about what else might be out there and kept from the public.

That is a super cool microscope. I had the watch innards picked pretty much immediately. Watch battery with no Hg, crystal resonator and plenty of mechanical bits that aren't in digital watches.

Nice video, I taught they will use ultrasonic imaging medical sensor to see the depth.

The gel in the microscope reminds me of doing replication of metal micro structures. Where one uses acetate tape or hardened polymer to take a replica of a surface to look at it under a microscope (optical or even SEM I believe).
I’ve also seen demos of measuring devices that scan objects and make 3D model meshes, which are fun.

2:42 So it works simular to a normal map in games and vfx? Amazing!

Apart from the extremely interesting features and applications of this idea, one of my takeaway messages is the nth confirm that Hexagons are the Bestagons 🤓

I’m wondering how big this ‘microscope’ can get. If the depth can’t be more than the width, it would mean bigger microscopes would have more overall use if it were wider/bigger i’d assume

The low framerate of the "microscope" together with it's texture makes the images look like clay animation. The sound effect made it complete.