MiDaS_Depth v1.0


 
This location is for Registered Users Only.
Perhaps you need to login or register.
Contributor: Riley Gray
Nuke ML monocular depth estimation tool. The model file (.cat file) can be found here: https://drive.google.com/file/d/1F5D9eWh3gJzPs-vwUMC410t2xyeWo4PH
Requirements:
13.1 or later
Linux, Mac, Windows
27 Dec 2021
629

Example ContactSHeet v001

MiDaS_Depth is a nuke ML tool based on the MiDaS monocular depth estimation repo that can be found on github here: https://github.com/isl-org/MiDaS . It allows for the generation of a depth pass based on a single image with no camera or track required. It is not 100% temporally stable but can be frameblended or frameheld for use on image sequences. Changing the 'Input Resolution Scale' Slider can give much more detailed results but at the sacrifice of depth accuracy.

This tool is only compatible with nuke 13.1 due to the use of the new CatFileCreator node to create the .cat file. Due to the limitations of nuke 13.1 only being able to utilitze pytorch 1.6.0, we were only able to convert MiDaS v2.1 and not the very latest v3.0 model. However the largest model was used to give the highest resolutiuon and best results (sorry for the 400mb cat file). Also, the input colorspace should be SRGB but feel free to experiment with different colorspaces for different results.

Hopefully people can find uses for this tool; I have been experimenting using it to create point clouds that can then be deformed using Mads Hagbarth's Point Render tool to great effect!

 

Model File: https://drive.google.com/file/d/1F5D9eWh3gJzPs-vwUMC410t2xyeWo4PH

Please login in order to download these files.

Comments   

 
+1 # Jung Keun Kim 2021-12-28 07:41
Thank you for sharing~ ;-) I hope we can share more stuff about ML.
 
 
+1 # Way the 2021-12-29 03:59
super
 
 
0 # Max Last 2022-02-18 17:29
Thanks for sharing, this is great to see! I gave this a go in Nuke 13.0v1 and 13.1v2, and got the following error (with GPU switched off - with it on, I just run out of GPU memory).

ERROR: Inference1: Exception caught processing model: The following operation failed in the TorchScript interpreter.
...
run.py(62): run
run.py(154):
RuntimeError: The size of tensor a (24) must match the size of tensor b (23) at non-singleton dimension 2

The full error is here:
https://pastebin.com/DGqm2LSH
 
 
+1 # Max Last 2022-02-18 18:01
Ah no my apologies, this was as a result of using the .cat file directly without the .nk file that sets the resolution correctly. It's working very well!
 
 
0 # K T 2022-03-26 22:03
This is so cool and has already come in handy!

Have you thought about incorporating this?
https://www.youtube.com/watch?v=lDeI17pHlqo
https://github.com/compphoto/BoostingMonocularDepth
 

You have no rights to post comments

We have 5247 guests and 82 members online