Implicit Model

Implicit models represent a growing area of machine learning research focused on defining network layers through equilibrium equations rather than explicit layer-by-layer computations. Current research emphasizes efficient training algorithms, particularly for deep equilibrium models and diffusion models, and explores their application in diverse fields like image generation, collaborative filtering, and reinforcement learning. The ability of implicit models to handle complex relationships with reduced computational cost and improved generalization makes them increasingly significant for both theoretical understanding and practical applications across various scientific domains.

Papers

January 23, 2024

Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning
Zizhao Wang, Caroline Wang, Xuesu Xiao, Yuke Zhu, Peter Stone
Reinforcement Learning Causal Model State Space Implicit Model Bisimulation Metric Causal Abstraction

December 15, 2023

Implicit Modeling of Non-rigid Objects with Cross-Category Signals
Yuchun Liu, Benjamin Planche, Meng Zheng, Zhongpai Gao, Pierre Sibut-Bourde, Fan Yang, Terrence Chen, Ziyan Wu
3D Shape Category Shift Implicit Model Shape Representation Non Rigid Deep Implicit Function

October 28, 2023

TorchDEQ: A Library for Deep Equilibrium Models
Zhengyang Geng, J. Zico Kolter
Easy to Use Library Implicit Model Deep Equilibrium Model Deep Learning Community

August 21, 2023

MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving
Yibo Liu, Kelly Zhu, Guile Wu, Yuan Ren, Bingbing Liu, Yang Liu, Jinjun Shan
Autonomous Driving 3D Reconstruction 3D Point Cloud Implicit Model 3D Car

July 16, 2023

Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks
Haobo Song, Soumajit Majumder, Tao Lin
Vision Task Implicit Model Deep Equilibrium Model Sparsity Tradeoff Weight Tied Model

April 20, 2023

Implicit Temporal Modeling with Learnable Alignment for Video Recognition
Shuyuan Tu, Qi Dai, Zuxuan Wu, Zhi-Qi Cheng, Han Hu, Yu-Gang Jiang
Temporal Attention Video Recognition Temporal Modeling Implicit Model Frame Alignment Self Supervised Alignment

April 8, 2023

High-Fidelity Clothed Avatar Reconstruction from a Single Image
Tingting Liao, Xiaomei Zhang, Yuliang Xiu, Hongwei Yi, Xudong Liu, Guo-Jun Qi, Yong Zhang, Xuan Wang, Xiangyu Zhu, Zhen Lei
Single Image Implicit Model Avatar Reconstruction Non Rigid Deformation Clothed Avatar

February 5, 2023

Explainable Machine Learning: The Importance of a System-Centric Perspective
Manish Narwaria
Machine Learning Importance Aware Explainable Machine Learning Implicit Model DL Model

November 28, 2022

What learning algorithm is in-context learning? Investigations with linear models
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou
Practical Algorithm Context Learning Comprehensive Investigation Linear Model Implicit Model Neural Sequence Model Context Learner Transformer Based in Context

November 26, 2022

Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth
Filipe de Avila Belbute-Peres, J. Zico Kolter
Neural Tangent Kernel Implicit Model Kernel Extension Sinusoidal Activation Structured Initialization

November 15, 2022

IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction
Kennard Yanting Chan, Guosheng Lin, Haiyu Zhao, Weisi Lin
Mesh Reconstruction Implicit Model Pixel Aligned Implicit Single View Human Reconstruction Dual Pixel

November 3, 2022

Contrastive Value Learning: Implicit Models for Simple Offline RL
Bogdan Mazoure, Benjamin Eysenbach, Ofir Nachum, Jonathan Tompson
Offline Reinforcement Learning Reward Function Model Based Reinforcement Learning Implicit Model Step to Step Dynamic Contrastive Reward

October 23, 2022

Deep Equilibrium Approaches to Diffusion Models
Ashwini Pokle, Zhengyang Geng, Zico Kolter
Diffusion Model Generative Model Implicit Model Model Inversion Deep Equilibrium Pixel Wise Random Sampling

October 21, 2022

Implicit Offline Reinforcement Learning via Supervised Learning
Alexandre Piche, Rafael Pardinas, David Vazquez, Igor Mordatch, Chris Pal
High Dimensional Offline Reinforcement Learning Supervised Learning Behavior Cloning Robot Skill Implicit Model

October 17, 2022

Implicit models, latent compression, intrinsic biases, and cheap lunches in community detection
Tiago P. Peixoto, Alec Kirkley
Community Detection Implicit Model Free Lunch Community Detection Problem

October 9, 2022

3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models
Ziyi Chang, George Alex Koulieris, Hubert P. H. Shum
Domain Adaptation 3D Reconstruction Single Image Implicit Model 3D Human Reconstruction 3D Modeling Implicit Reconstruction 3D Domain Adaptation

September 19, 2022

State-driven Implicit Modeling for Sparsity and Robustness in Neural Networks
Alicia Y. Tsai, Juliette Decugis, Laurent El Ghaoui, Alper Atamtürk
Neural Network Native Robustness Sparsity Increase Learning Model Implicit Model Backward Propagation Implicit Differentiation

September 6, 2022

Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distribution
Joel Janek Dabrowski, Daniel Edward Pagendam
Variational Inference Bayesian Neural Network Posterior Inference Posterior Approximation Implicit Model Posterior Predictive Approximate Bayesian Inference Bayesian Neural

August 11, 2022

An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification
Runxue Bao, Bin Gu, Heng Huang
Stochastic Gradient Descent Stochastic Gradient Low Rate ACCELERATION Implicit Model Loss Minimization Proximal Gradient Descent Parameter Identification

July 12, 2022

Conditional Energy-Based Models for Implicit Policies: The Gap between Theory and Practice
Duy-Nguyen Ta, Eric Cousineau, Huihua Zhao, Siyuan Feng
Theoretical Understanding Mutual Information Practice Mode Energy Based Model Implicit Representation Conditional Reasoning Implicit Model Implicit Policy Policy Cloning

Implicit Model

Papers

Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning

Implicit Modeling of Non-rigid Objects with Cross-Category Signals

TorchDEQ: A Library for Deep Equilibrium Models

MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving

Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks

Implicit Temporal Modeling with Learnable Alignment for Video Recognition

High-Fidelity Clothed Avatar Reconstruction from a Single Image

Explainable Machine Learning: The Importance of a System-Centric Perspective

What learning algorithm is in-context learning? Investigations with linear models

Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth

IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction

Contrastive Value Learning: Implicit Models for Simple Offline RL

Deep Equilibrium Approaches to Diffusion Models

Implicit Offline Reinforcement Learning via Supervised Learning

Implicit models, latent compression, intrinsic biases, and cheap lunches in community detection

3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models

State-driven Implicit Modeling for Sparsity and Robustness in Neural Networks

Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distribution

An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification

Conditional Energy-Based Models for Implicit Policies: The Gap between Theory and Practice