Keyboard Photogrammetry

[snacksthecat]

Zbrush looks interesting. From my attempts there sometimes are little "hanger" pieces that are hard to remove. So that is cool and looks like it could help.

Also I like your scans of switches. I've been trying to figure out realitycapture for a few days. Not sure what I'm doing wrong but hopefully I can figure it out in the next 2 days, otherwise Ill post in the tutorial thread my issues.

Very cool. I like working in RealityCapture a lot. Whenever something turns out poorly in Metashape, that's my go-to. It produces really good looking meshes and textures. The only warning I have is don't get carried away with control points. I've spent hours upon hours carefully placing control points to help the app with alignment, to no avail. I've read people having success with them by setting very high weighting on each point (e.g. 200 vs. 10). But I myself just haven't been able to effectively use control points for alignment. If I put an hour of work in and the photos still don't align, I should just give up and move on.

I've recently figured out that you can do alignment in Metashape and export to .out (bundler) format. Then import the .out file into RealityCapture. But I'm not sure if the photos should be exported from Metashape as undistorted or not, and haven't had enough time to fiddle with it yet.

I also just got a digital drawing tablet so I can do touch-ups in Zbrush. So I guess you can call me snacksthedigitalartist now.

The next board I scanned is my indie SSK. So far its coming along really nicely but I'm also trying to test out different things to see how it effects the finished product. The main piece I'm adding to my workflow goes like this:

• Run photogrammetry and generate mesh in Metashape
• Export from Metashape, import to Zbrush, clean up bumpiness, holes, etc.
• Remesh, unwrap, export from Zbrush. Import back into Metashape
• Decimate down to ~250,000 faces
• Run "Refine Mesh" tool (high settings)

When running "Refine Mesh", a few things to point out. First, I'm decimating down to a lower poly count because this procedure subdivides the mesh 3 or 4 times so if you don't do this, you end up with an insane number of faces. I'm also running it on "High" rather than "Ultra High" because apparently my PC doesn't have enough video memory for ultra. But that's probably okay because, lastly, I'm disabling all photos except for the ~150 closeup photos I took.

I'll go into detail more later but recently I've found I can really improve the quality of the mesh by taking one set of photos with the whole object in frame, mostly to capture the general structure. Then I come in with a zoomier lens and take some closeup pictures. The pictures from set #1 get the full shape of the object and provide a reference point for getting the camera positions from set #2.

It's a bit of a hassle because you end up taking a ton of photos this way. I'd also like to experiment with markers (like matt3o suggested in the tutorial thread). It might make my life a whole lot easier aligning the "general shape" set of images.

Oh and I've learned one more thing. I've been generating my meshes with way too many target faces. I basically have been allowing it to produce as many faces as needed. I've found that by reducing the target face count (something around like 750,000 - 1,000,000), I get a much prettier model. Fewer craters, bumpiness, etc.

[snacksthecat]

And it's done! I think I've taken this one as far as I can go with it, without taking more pictures. This one I wish the I'd set my white balance correctly (had it leftover from a different set). So it turned out a little bit green/yellow. I adjusted the sketchfab settings to compensate for this. Already went into some detail about what I've done differently these past few scans so not much more to say.

https://skfb.ly/6QSTH

[SneakyRobb]

And it's done! I think I've taken this one as far as I can go with it, without taking more pictures. This one I wish the I'd set my white balance correctly (had it leftover from a different set). So it turned out a little bit green/yellow. I adjusted the sketchfab settings to compensate for this. Already went into some detail about what I've done differently these past few scans so not much more to say.

https://skfb.ly/6QSTH

Hi, that is a gorgeous model!

[snacksthecat]

Hi, that is a gorgeous model!

Thank you! Of course I see everything wrong with it but I've been staring at it for the past few days, after all.

I wanted to mention two things...

First, I'm totally sold on Zbrush. It's amazing what you can do in there in terms of manually fixing defects. I think this is exactly the tool I need to bridge the gap that my photos are falling short on. I've had so much fun with the program in the past week week that I decided to buy a drawing tablet thing. It was relatively inexpensive (I just needed something basic) and will give you more control in Zbrush (e.g. control over pressure). Haven't tried it out yet but I'm ready and excited to flex my nonexistent art skills.

Next thing, I got an ad for a kickstarter product; some sort of thing you attach to your camera to do focus stacking. I wasn't aware what this is but it sounds very cool. For those of you into photography, you almost certainly already know about this. But for those who are not, focus stacking is where the camera takes the same picture multiple times changing the focus each time. With the end goal of getting the entirety of what you're photographing in focus. You can imagine how this would be useful in photogrammetry. If my explanation didn't make sense, it will be more evident in this example from the-digital-picture.com website:

Focus Stacked	Standard

Well I'd gotten a new camera over the holidays. Seeing that kickstarter thing made me think this is some super premium feature/technique. I never imagined that my new camera (Canon RP) actually has this feature built in (which is surprising because it lacks some other things I wish it had). So I'm looking forward to trying it out this weekend. No idea how/if it will effect the software's ability to calibrate images but we shall see.

For anyone who is interested, sketchfab and RealityCapture are running a challenge/contest right now. The contest is to scan your favorite toy using the RealityCapture software, then upload it to the sketchfab website. RealityCapture is pay-per-input, so the software itself is free. Since it costs money to purchase input licenses, they're providing entrants with 20EUR of credits to use for the challenge. The contest runs through March and I'm probably going to try to scan something to enter this weekend.

https://sketchfab.com/blogs/community/s ... orite-toy/

I know, I know, "dear diary" right?

[SneakyRobb]

Interesting contest. I have some time at home right now. I might try this on my Optimus Prime or Tallgeese models

[snacksthecat]

Those would be cool to see!

One additional thing that might be helpful are the coded markers:
https://support.capturingreality.com/hc ... l-version-

They're little barcode-like things that get automatically detected in the images they appear in. The software creates a control point for each of the markers. I'm not sure if it is more effective than manual placement of control points but I wanted to try it out. I don't use this software enough to be able to know if there's a difference, sorry to say.

I'm trying to scan this wifi camera car thing. It's full of pitfalls but it will look so cool if it all comes together.

[SneakyRobb]

Hi. That makes sense and is cool. I know you said that the narrow border areas can be a bit of an issue sometimes, so such markers make sense.

Does this robot device save times in terms of manually turning? Were you using a "lazy susan" sort of thing before where you would manually turn it by hand?


Also this is the dental scanner that my dentist used I think. Pretty fast machine actually, but that is to be expected when it costs tens of thousands of dollars.

https://www.youtube.com/watch?v=a40pLVe3pVY

[SneakyRobb]

Hi,

So I printed some miniatures I was going to send to snacksthecat, but they are at my parents house. They are self isolating with no symptoms. To be on the safeside I will wait for them to hopefully come out of the 14 days symptom free.

I wont send the models until that time. They do exist though and with some sanding/painting I think they would look really cool!

miniatures.jpg (336.33 KiB) Viewed 14390 times

[snacksthecat]

So I printed some miniatures I was going to send to snacksthecat

Very excited to get these. I can't say thanks enough, I've never had anything 3D printed before. Really piqued my interest and got the gears turning in my head.

So I recently bought on MM a hi-pro HHKB with fancy aftermarket domes. I'm really enjoying typing on it, even more than my primary board that I use almost always.



It also looks just amazing with the upgraded caps.

The only thing about it is it has a ridiculous sticker on the back. I won't spoil it but I'll say that it's slightly NSFW, just barely. So maybe that gives it away but you'll see soon enough. Strange as it is though, I think it gives the board some character so I'm going to leave it on. It is on the bottom, after all.

I do have one photography question, and it's a bit of an emergency since I want to start shooting in an hour or so. So if anyone knows the answer or where to direct me, I'd be super grateful.

My question is about lens sweet spots. Is there an easy way for me to figure out the sweet spot for my lens? I'd like to use this Canon R 24-105mm because it takes great pictures. Even my untrained eye can see that much. But I have no idea what aperture / focal length I should use for maximum shapness + good amount of depth of field.

I have been setting the aperture to f22 thinking that's optimal, but I guess I'm pretty far off in my understanding. Yes it gives high depth of field but probably am suffering quite a bit in overall sharpness.

I've tried googling around a little but most of the stuff I found is over my head. It'd be great if someone familiar in the art could guide me a bit. If not, I'll just dial it back a little and hope for the best. The show must go on! (going a little crazy with all this time at home can you tell?)

Anyways, thanks as always for any tips/tricks.

[snacksthecat]

Finished taking all the photos. I ended up shooting with the lens set to 50mm and f11. I guess the sweet spot for most lenses is more toward the center? The pictures were very sharp, but definitely didn't get the same depth of field I had at f22. I'm not sure how much this matters, though, because I've switched up my workflow to be less dependent on the "far away" photos and more on the close-ups.

I think I nailed the white balance; something I usually screw up. This was probably the first time I've taken a proper custom white balance input picture. I guess I could really benefit from a beginners level photography class but I'm lazy and stubborn so that will probably never happen. But yeah, the color matching for this model should be very good.

Going to start processing the pictures now. Hopefully this photoshoot wasn't a waste of time.

[snacksthecat]

Well it looks like my weekend project is all done, and the weekend is only beginning. And just as well, because I'm now wiped out and don't feel like doing much of anything. I feel like this pattern happens to me a lot...

So yeah, I jumped the gun a bit and did this one over a few evenings. It turned out awesome, but it also has many scannable attributes working in its favor (good shape, lots of features, etc.)

It has a few stickers on it. One of which is a "hentai auto club" sticker, so ever so slightly nsfw. I decided to leave these stickers on because they make it more interesting. Here's the scan:

https://skfb.ly/6RBVH

[ramnes]

Just discovering this thread, that's awesome! And that last HHKB JP came out really nice. I didn't even know that photogrammetry was a thing, although I worked in a company that is doing almost exactly this a few years ago.

[snacksthecat]

Over the past few days I've been twiddling my thumbs wondering what to do next. I don't feel like scanning another board right now because it requires me to set a lot of stuff up and creates a big mess. So I shifted my focus to my PC to see what other photogrammetry software could produce. This HHKB dataset is pretty safe and reliable, as far as image alignment goes. So I think it would be a good candidate to try running through some of the other programs out there.

I've already finished trying with 3DF Zephyr and it's pretty interesting to see how the models come out differently. Particularly the texturing produced by Zephyr is very different than Metashape.

https://skfb.ly/6RHoS

Here's a comprehensive list of the paid applications I plan to test out with this dataset:

• Agisoft Metashape
• 3DF Zephyr
• Reality Capture
• Autodesk ReCap
• PhotoModeler (maybe)

I also plan to run it through some of the free programs as well:

• Meshroom
• Colmap
• MicMac
• Multi-View Environment (maybe)
• OpenMVG (maybe)
• Regard3D (maybe)
• VisualSFM (maybe)

Pretty much I'm just using this list as a guide. Focusing only on solutions made for scanning at close range (i.e. not limited to overhead drone images). I'd love to try some of the more "enterprise grade" stuff out there, but unfortunately it tends to be very expensive. So I probably won't get my hands on any of those tools unless I change careers or become quite rich. But this roster should keep me plenty busy anyways.

The last thing I wanted to say is that I tried for the first time Colmap. I'd sort of developed a preconceived notion that since it's free, it must not be very good. However, after playing with it yesterday, I found it to be very capable. And it has sane defaults set so that if you don't know what you're doing, you can get by just fine by following the software's suggested path. So for those reasons, I'm going to start recommending Colmap (over my previous preference, Meshroom) to anyone looking for something free to try out. This is mostly based on the amount of control you get in Colmap. There are parameters for every little itty bitty tweak you could possibly imagine. That might sound intimidating, but you don't have to change any of those settings if you don't want to and you should still end up with something nice (best of both worlds).

I'll post the finished results once I feel I've exhausted my shortlist.

Hope everyone is doing well and finding stuff to occupy them during these quarantines.

[snacksthecat]

I finally got around to scanning this really old keyboard. It gets boring to use the same exact method every time so I always try to introduce some sort of tweak into the process. This time I experimented with focus stacking, as a way to squeeze more depth of field into each photo. The focus stacking process involves taking multiple photos, adjusting the focus slightly each time. Then run the sets of photos through some software that combines them into a single image with better focus.

This is the method that I used first. But it seemed to really confuse the photogrammetry software. So I eventually just dumped all the photos together and ran it that way. This too confused the software, but to a lesser degree. All in all, some aspects of this scan turned out better than usual, but at the expense of others. There's probably a good way to leverage this, but I don't think I'll play around with it much more.

https://skfb.ly/6TFvT

[snacksthecat]

Hey friends, I haven't done a scan in forever and finally decided to scan my brand new F77. I made some tweaks to the process and I'm really happy with the result. The main change was using more of a rotating pedestal vs. a broad turntable, allowing more of the board to be visible in each shot. The new version of Metashape has an option now where you can tell it to ignore features that remain static from photo to photo, so you don't need to have a backdrop. It makes producing masks really simple!

Link to the 3D model on Sketchfab:
https://skfb.ly/opPQz

[Polecat]

Hey friends, I haven't done a scan in forever and finally decided to scan my brand new F77. I made some tweaks to the process and I'm really happy with the result. The main change was using more of a rotating pedestal vs. a broad turntable, allowing more of the board to be visible in each shot. The new version of Metashape has an option now where you can tell it to ignore features that remain static from photo to photo, so you don't need to have a backdrop. It makes producing masks really simple!

Snacks, thanks for sharing! Great job, it's neat to see and hear how it's done.

[dcopellino]

Hi snacks, what a pleasure read you again here. Wow. Outstanding results. You have gained loads of experience on the subject so far, I don't think it's just a matter of the option now present in metashape that makes the difference. Really impressed by so high quality photogrammetry achieved.

[digital_matthew]

Awesome work with the photogrammetry. It's pretty cool being able to inspect keyboards that I probably won't ever see IRL. I was wondering if you ever thought about using these scans to 3D print molds for vacuforming clear plastic dust covers for keyboards?

[snacksthecat]

Thank you guys for the nice comments. I'm glad you enjoy the models.

Awesome work with the photogrammetry. It's pretty cool being able to inspect keyboards that I probably won't ever see IRL. I was wondering if you ever thought about using these scans to 3D print molds for vacuforming clear plastic dust covers for keyboards?

That would be an interesting use! I haven't found a real-life application for doing something practical with the completed model, since most things we'd want to do require more precision than I've been capable of getting. But I'd think that a dust cover would be well suited since there is a bit more wiggle room in the shape/measurements.

[snacksthecat]

I got an illuminated tent to try to improve lighting during the picture taking step. I knew it was pretty big but I must say this is a little bit bigger than I expected . It does provide really nice, even lighting, though. And you can select a backdrop color that will provide a good contrast w/ the keyboard, as well. I tried this using just the two lights that came with the kit.



A few other things I did differently:

• re-focused as a part of the board moved closer to the camera (usually I leave set the focus once-per-rotation to improve consistency, but I think that I've had a misunderstanding about that)
• did a set of closeups on the top (I've done this sometimes, but it has not always been as easy to match the photos. This time I used the "tilt" of the pan/tilt head to do a quick set of closeups at the end. Didn't do the back, but from what I can tell so far, the top benefitted a lot from this)

[snacksthecat]

Result:

https://skfb.ly/opWtL

[snacksthecat]

Televideo 725

https://skfb.ly/orurV

[snacksthecat]

Here's a short timelapse video I made of the whole process.

This one was a big challenge to get right and thus a lot of fun because I took about half the photos of what I usually take.

264 photos might seem like plenty but I wasn't careful to capture the far edges so there was a lot of playing around with settings to get it as nice as possible. Learned some new tricks as well

[dcopellino]

Very impressive workflow. Not bad at all. May I wish to know something more about your rig? Were all pics shot inside the light box? How steep could be the learning curve? Everytime my jag drops down watching your videos, I promise myself to start studying photogrammetry. Earlier or later I'll do it for real.
What do you think of this stuff as starting point:
a lightbox
electric rotating stand

[snacksthecat]

Yes, I have been using a lightbox. At first because I wanted a totally empty background, but now the software has an option to "ignore static background features", so now it is not so necessary. It gives really even lighting so that helps a lot.

I've been using a similar electrical turntable, but with a remote or at least one that will stop at some fixed intervals so that the object isn't moving when the picture snaps.

You could do it for sure. The hardest part to grapple with comes when trying to get the whole object in a single chunk. What I did this time was to align all of the images of the top of the board in one chunk and all of the bottom images in a second chunk -- mixing some of the side photos between both chunks. Aligned the photos in each chunk, then used the align chunks feature to merge them into a single one. Everything else works very well in the program with the default settings generally given a set of decent photos.

[snacksthecat]

Tried a really challenging scan, which was a red Matias switch. It's both very small and transparent! Naturally it didn't turn out quite right but I think it looks kind of interesting.

https://skfb.ly/o8u8u

[Polecat]

Snacks, thanks for sharing, fun to watch as always! Uh the switch housing looks like it's been wax-boiled, or at least what I'd imagine wax boiling looks like. <ducking>

[zrrion]

that's spot on what bag lubing with spray PTFE lube looks like actually

[snacksthecat]

Thank you guys. This was really fun but also absolutely maddening to get the texture to look OK. The switch housing when I originally generated the texture was even more frosted looking. What I ended up doing was disabling all but ~12 cameras, which had a non-glare view of the inside.

In hindsight I actually think doing the reverse of this for the mesh generation would have resulted in a cleaner shape (i.e. disable only the "clearest" ones because they confuse the software, and only leave the glare pictures, since those look more opaque.

Disabling images at certain points in the process is something that I overlooked for the longest time because I had it in my mind that more pictures = better result. But that ain't true! There is even a feature in the software to "thin out" the photoset by dropping images with a lot of overlap.

It's funny you mention the lube spray because I actually bought a can of laser scanning spray a long time ago to help deal with shiny plastic. But in the end I found that it only looked good on white plastic (white on white) and made everything else look dusty. I also didn't want to spray the stuff on anything I cared about

Here's a side by side showing a clear image and a glaring one:

[snacksthecat]

Another thing I just learned: you can decimate the mesh down quite low and still preserve almost all the detail. I'm doing a lot more trial and error comparisons with different settings and it's really been helpful to see clear examples of the differences.

I wanted to try what I had mentioned (using the opaque images for the meshing) but I didn't want to start the whole thing over. So I used "refine mesh", which in my prior experience took forever. What I finally realized is that you're supposed to decimate down very low before running that tool. Whatever it's doing, it results in a massive number of faces compared to the input. In that way it makes sense why having a small input would be better.

I did this process a few times. Here's the best I can muster:

https://skfb.ly/o8vAN