An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Related tags

Image Processingexecutor-3d-encoder
Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

  • PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
  • PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

  • PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
  • PointConv: Deep Convolutional Networks on 3D Point Clouds
Comments
  • which version should the `finetuner` module be?

    which version should the `finetuner` module be?

    I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

    from finetuner.tuner.callback import BestModelCheckpoint
from finetuner.tuner.pytorch.losses import TripletLoss
from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

    It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

    Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.

    opened by elenacliu 4
  • fix: fix previous structure

    fix: fix previous structure

    I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

    opened by elenacliu 2
  • Add dataset, preprocess and pointnet2

    Add dataset, preprocess and pointnet2

    Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

    @numb3r3 @jemmyshin

    opened by elenacliu 1
  • In `farthest_point_sample` function, the first centroid sampled is not random.

    In `farthest_point_sample` function, the first centroid sampled is not random.

    I'm reading your codes.

    https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

    The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

    opened by elenacliu 1
  • Provide diffusion encoder

    Provide diffusion encoder

    https://github.com/nmwsharp/diffusion-net

    I'm unfamiliar with the jina ecosystem. So this is my first issue.

    Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

    opened by fire 2
Releases(v0.3.0)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
GitHub Repository
Generates images of calendar month tables and can paste them onto suitable photos.

📆 calendizer README Generates images of calendar month tables and can paste them onto suitable photos. A quick way to make your own calendar for prin

Sean Ryan 2 Dec 14, 2022
A tool for hiding data inside of images

Stegenography-tool a tool for hiding data inside of images Quick test: do python steg-encode.py test/message.txt test/covid19.png to generate the test

luke 2 Nov 02, 2021
QR fixer part is standalone but for image to FQR conversion

f-qr-fixer QR fixer part is standalone but for image to FQR conversion it requires Pillow (can be installed with easy_install), qrtools (on ubuntu the

2 Nov 22, 2022
Png2Jpg tool will help you convert from png image format to jpg images format.

PNG 2 JPG All codes assume running from root directory. Please update the sys path at the beginning of the codes before running. Over View Png2Jpg too

Nguyễn Trường Lâu 2 Dec 27, 2021
An agnostic Canvas API for the browser-less and insane

Apollo An agnostic Canvas API for the browser-less and mildly insane. Project Apollo is a Pythonic re-imagining of HTML Canvas element implementati

1 Jan 13, 2022
Repair broken bookmarks to referenced files in Apple Photos

Repair Apple Photos Bookmarks Work in progress to repair file location bookmarks in Apple Photos. Background Starting in macOS 10.15/Catalina, photos

Rhet Turnbull 10 Nov 03, 2022
ProsePainter combines direct digital painting with real-time guided machine-learning based image optimization.

ProsePainter Create images by painting with words. ProsePainter combines direct digital painting with real-time guided machine-learning based image op

Morphogen 276 Dec 17, 2022
LGVL helper script to batch and convert with lvgl offline image converter

script to batch and convert with lvgl offline image converter

Yohann 1 Oct 05, 2022
Quickly 'anonymize' all people in an image. This script will put a black bar over all eye-pairs in an image

Face-Detacher Quickly 'anonymize' all people in an image. This script will put a black bar over all eye-pairs in an image This is a small python scrip

Don Cato 1 Oct 29, 2021
💯 Watermark your images with one line of command

Watermarker 💯 Watermark your images with one line of command 🧐 $ pip3 install

Orhan Emre Dikicigil 3 May 01, 2022
Seaborn-image is a Python image visualization library based on matplotlib and provides a high-level API to draw attractive and informative images quickly and effectively.

seaborn-image: image data visualization Description Seaborn-image is a Python image visualization library based on matplotlib and provides a high-leve

48 Jan 05, 2023
Simplest QRGenerator with a cool feature (-sh=True :D)

Simple QR-Codes Generator :D Generates QR-codes, nothing more and nothing less . How to use Just run ./install.sh to set all the dependencies up, th

RENNAARENATA 1 Dec 11, 2021
Raven is a tool written in Python3 allowing you to generate an unique image with some text.

🐦 Raven is a tool written in Python3 allowing you to generate an unique image with some text. It does it by searching the text on Google, do

Billy 39 Dec 20, 2022
Steganography Image/Data Injector.

Byte Steganography Image/Data Injector. For artists or people to inject their own print/data into their images. TODO Add more file formats to support.

Ori 4 Nov 16, 2022
Python wrappers for external BART computational imaging tools and internal libraries

bartpy Python bindings for BART. Overview This repo contains code to generate an updated Python wrapper for the Berkeley Advance Reconstruction Toolbo

Max Litster 7 May 09, 2022
A QR Code encode and decode python module

A QR Code encode and decode python module

Fayas Noushad 4 Feb 10, 2022
A simple Streamlit Component to compare images in Streamlit apps. It integrates Knightlab's JuxtaposeJS

streamlit-image-juxtapose A simple Streamlit Component to compare images in Streamlit apps using Knightlab's JuxtaposeJS. The images are saved to the

Robin 30 Dec 31, 2022
Fill holes in binary 2D & 3D images fast.

Fill holes in binary 2D & 3D images fast.

11 Dec 09, 2022
Python Image Morpher (PIM) is a program that can take two images and blend them to whatever extent or precision that you like

Python Image Morpher (PIM) is a program that can take two images and blend them to whatever extent or precision that you like! It is designed to emulate some of Python's OpenCV image processing from

David Dowd 108 Dec 19, 2022
Sample data for the napari image viewer.

napari-demo-data Sample data for the napari image viewer. This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugi

Genevieve Buckley 1 Nov 08, 2021