Flow-Guided Transformer for Video Inpainting

Especially in spatial transformer, we design a dual perspective spatial MHSA, which integrates the global tokens to the window-based attention.
https://github.com/hitachinsk/fgt

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

📝We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance.
https://github.com/timdettmers/bitsandbytes

KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints

📝In this work, we investigate common issues with existing spatial encodings and propose a simple yet highly effective approach to modeling high-fidelity volumetric humans from sparse views.
https://github.com/facebookresearch/KeypointNeRF

Collaborative Neural Rendering using Anime Character Sheets

📝Drawing images of characters with desired poses is an essential but laborious task in anime production.
https://github.com/megvii-research/conr

Deep Patch Visual Odometry

📝We propose Deep Patch Visual Odometry (DPVO), a new deep learning system for monocular Visual Odometry (VO).
https://github.com/princeton-vl/dpvo

StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3

📝Notably, StyleFaceV is capable of generating realistic $1024\times1024$ face videos even without high-resolution training videos.
https://github.com/arthur-qiu/stylefacev

Multi-instrument Music Synthesis with Spectrogram Diffusion

📝An ideal music synthesizer should be both interactive and expressive, generating high-fidelity audio in realtime for arbitrary combinations of instruments and notes.

https://github.com/magenta/music-spectrogram-diffusion

MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
📝We adopt a hierarchical query embedding scheme to flexibly encode structured map information and perform hierarchical bipartite matching for map element learning.

https://github.com/hustvl/maptr

YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception

📝Over the last decade, multi-tasking learning approaches have achieved promising results in solving panoptic driving perception problems, providing both high-precision and high-efficiency performance.

https://github.com/CAIC-AD/YOLOPv2

Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition

📝In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations.

https://github.com/hikvision-research/skelact

FILM: Frame Interpolation for Large Motion

📝Recent methods use multiple networks to estimate optical flow or depth and a separate network dedicated to frame synthesis.

https://github.com/google-research/frame-interpolation

Online Decision Transformer

📝Recent work has shown that offline reinforcement learning (RL) can be formulated as a sequence modeling problem (Chen et al., 2021; Janner et al., 2021) and solved via approaches similar to large-scale language modeling.

https://github.com/facebookresearch/online-dt

YOLOX-PAI: An Improved YOLOX Version by PAI

📝We develop an all-in-one computer vision toolbox named EasyCV to facilitate the use of various SOTA computer vision methods.

https://github.com/alibaba/EasyCV

PeRFception: Perception using Radiance Fields

📝The recent progress in implicit 3D representation, i. e., Neural Radiance Fields (NeRFs), has made accurate and photorealistic 3D reconstruction possible in a differentiable manner.

https://github.com/POSTECH-CVLab/PeRFception

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

📝Once the subject is embedded in the output domain of the model, the unique identifier can then be used to synthesize fully-novel photorealistic images of the subject contextualized in different scenes.

https://github.com/XavierXiao/Dreambooth-Stable-Diffusion

CLIP-Mesh: Generating textured meshes from text using pretrained image-text models

📝We present a technique for zero-shot generation of a 3D model using only a target text prompt.

https://github.com/NasirKhalid24/CLIP-Mesh

A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

📝Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models.
https://github.com/aangelopoulos/conformal-prediction

Transformers are Sample Efficient World Models

📝Deep reinforcement learning agents are notoriously sample inefficient, which considerably limits their application to real-world problems.
https://github.com/eloialonso/iris

Text-Free Learning of a Natural Language Interface for Pretrained Face Generators

📝We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis.
https://github.com/duxiaodan/fast_text2stylegan

Behavior Trees in Robotics and AI: An Introduction

📝A Behavior Tree (BT) is a way to structure the switching between different tasks in an autonomous agent, such as a robot or a virtual entity in a computer game.
https://github.com/BehaviorTree/BehaviorTree.CPP

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

📝The emerging paradigm of federated learning (FL) strives to enable collaborative training of deep models on the network edge without centrally aggregating raw data and hence improving data privacy.
https://github.com/adap/flower