An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

thumbor is an open-source photo thumbnail service by globo.com

Survey If you use thumbor, please take 1 minute and answer this survey? It's only 2 questions and one is multiple choice!!! thumbor is a smart imaging

9.3k Dec 31, 2022

Generate your own QR Code and scan it to see the results! Never use it for malicious purposes.

QR-Code-Generator-Python Choose the name of your generated QR .png file. If it happens to open the .py file (the application), there are specific comm

1 Dec 23, 2021

The ctypes-based simple ImageMagick binding for Python

Wand Wand is a ctypes-based simple ImageMagick binding for Python, supporting 2.7, 3.3+, and PyPy. All functionalities of MagickWand API are implement

1.2k Dec 30, 2022

Nudity detection with Python

nude.py About Nudity detection with Python. Port of nude.js to Python. Installation from pip: $ pip install --upgrade nudepy from easy_install: $ eas

881 Jan 06, 2023

The coolest python qrcode maker for small businesses.

QR.ify The coolest python qrcode maker for small businesses. Author Zach Yusuf Project description Python final project. Built to test python skills P

2 Jan 14, 2022

This projects aim is to simulate flowers(Gerbera Daisy) phyllotaxis.

phyllotaxis This projects aim is to simulate flowers(Gerbera Daisy) phyllotaxis. Take a look at the arrangement of this flower's seeds, this project's

3 Dec 10, 2021

A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for quickly creating new images from the one assigned to the field.

django-versatileimagefield A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for creat

490 Dec 13, 2022

EmbedToolV2 - 2.0 Version of DraKenCodeZ/ImageEmbedTool

1 Dec 07, 2021

LabelMe annotation tool source code

LabelMe annotation tool source code Here you will find the source code to install the LabelMe annotation tool on your server. LabelMe is an annotation

1.3k Jan 03, 2023

Panel Competition Image Generator

Panel Competition Image Generator This project was build by a member of the NFH community and is open for everyone who wants to try it. Relevant links

1 Oct 22, 2021

Labelme is a graphical image annotation tool, It is written in Python and uses Qt for its graphical interface

Image Polygonal Annotation with Python (polygon, rectangle, circle, line, point and image-level flag annotation).

9.6k Jan 09, 2023

The aim is to extract timeseries water level 2D information for any designed boundaries within the EasyGSH model domain

bct_file_generator_for_EasyGSH The aim is to extract timeseries water level 2D information for any designed boundaries within the EasyGSH model domain

1 Jul 08, 2022

Qt based ebook reader

Qt based ebook reader Currently supports: pdf epub djvu fb2 mobi azw / azw3 / azw4 cbr / cbz md Contribute Paypal Bitcoin: 17jaxj26vFJNqQ2hEVerbBV5fpT

1.4k Dec 26, 2022

Convert any image into greyscale ASCII art.

Image-to-ASCII Convert any image into greyscale ASCII art.

12 Jan 15, 2022

Rotates your images in the spirit of rot13

Image Rotator (imrot10) Its like rot13 but for images. Calling the algorithm imrot10 for im = image, rot = rotation, 10 = default magnitude. Unfortuna