Self Using Note
Some Useful Niubi tricks, repositories and paper. Just use Ctrl+F and input key words to search what you what. Strikethrough means I have not gone through yet.
General Structure
Attention is all you need: Masterpiece of transformer and attention. Everybody should read it.
Non local network: Similar with transformer. Sequence processing block with attention. From RBG and Kaiming.
Feature Denoising for Improving Adversarial Robustness: Non local mean and denoising residual block for preventing adversarial sample attack.
Bilinear pooling: Outer multiply two features. A sample code for bilinear pooling is here and here and here.
Bilinear pooling in application
High Resolution Net(HRNet): UNet with convolution in skip connection.
DANet: Dual self attention module, great example of pytorch sync-batch-norm and attention module.
XceptionNet: Reverse seperable convolution without activation.
Independent Components Layer: Combination of batch normalization and dropout layer.
Adaptive Batch Normalization: Related paper AdaIN
Networks Architecture Searching
Detection
Face Recognition
SBR landmark detector: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors. Github
Towards Interpretable Face Recognition
Occlusion Robust Face Recognition Based on Mask Learning with Pairwise Differential Siamese Network
Image Synthesis
Few shot unsupervised image2image: Content module and style module.
Pix2PixHD: Segmentation to image: Enhancer structure.
Semantic Image Synthesis with Spatially-Adaptive Normalization(SPADE): Used in GauGAN, segmentation to image.
SinGAN: Using multiscale patch generator to learn the distribution of target nature image.(ICCV19 Best Paper)
AdaIN: Adaptive instance normalization using in style transfer. Related paper AdaBN
Domain Intersection and Domain Difference
styleGAN: Really impressive with interesting structure. Eight FCs as guidance. Repo is here.
styleGAN2: Handle with naughty artifacts in styleGAN. Repo is here.
Large scale GAN training for high fidelity image synthesis
Face swapping: from beginner to give-up
Face Swapping GAN(FSGAN): Subject agnostic face swapping and reenactiment model.
Face swapping GAN: Introducing self attention, Kalman Filter.
Few-shot Realistic Neural Talking Head Models: Few-shot talking face sequences generator as one of the baselines of face swapping.
Few-shot face swapping GAN: Introducing AdaIN, SPADE.
Deep face lab: deep cyka blyat learning! Deep fake tools.
Expression transfer network: Which is basically one of my original ideas and someone just realised it. Research is struggle.
Everybody's talkin, 3D talking head generation: Very impressive paper for talking head generation from Sensetime, CASIA, NTU and NVIDIA. It combines with NLP and GAN to generate corresponding landmark from voice data.
Few-shot talking head generation: A not-so-impressive few-shot paper for talking head generation from samsung. And NEVER TRY THIS REPO which is totally rubbish!
Face shifter: Cope with occlusion problem in face swapping. Really beautiful demo. From MSRA.
Face anti-spoofing and face forensics
Improving Face Anti-Spoofing by 3D Virtual Synthesis
Static and Dynamic Fusion for Multi-modal Cross-ethnicity Face Anti-spoofing: There is a video feature extraction method in 3.1.SD-Net for Single-model. No idea whether it works or not.
Noise modeling in anti-spoofing
Central Difference Convolutional Networks: Introducing a new convolution structure by subtract central responce to improve feature robustness. And a new networks based on NAS and CDC. Need to check if this can be a universal structure.
Fusion for Multi-modal feature: Includs an intersting video fusion approach and taking advantage of multi-modal feature.
A deepfake detection repository
Face X-ray: Detect forgery depending on blending patterns, SOTA approach of deepfake detection.
DSP-FWA: The improved version is available here
Video general noise feature (PRNU) and steganalysis
Get rich feature from SRM filter: Inspired by Rich Models steganalysis to detect photoshop modification.
Sensor pattern noise identification
Color-Decoupled Photo Response Non-Uniformity
rPPG or related biology pattern (Warning: 玄学)
rPPG: Recommended by ZK.W
Another rPPG: Not really novel.
3D
Thin Plate Spline Interpolation
3D morphable model: from beginner to give-up
Basel Model 09: Standard face 3DMM.
Basel Model 17: Standard face 3DMM. Optimize facial expression.
Gaussian morphable model: Introduction of Gaussian 3DMM.
Markov Chain Monte Carlo for Automated Face Image Analysis
DF2Net: Deep learning approach single image 3DMM generator.
Deep 3D face reconstruction: Code available here
3D face alignment: 3dmm data and code.
Another deep 3D face reconstruction: Code available here
Video Generation
MoCoGAN: Video generation and enhancement
Low Level Vision
Deep Image Prior: VGG will have better result using in perceptual loss compared with ResNet.
Loss Function
Triple consistency loss for pairing distributions in GAN-based face synthesis: Combination of long path loss and short cut loss.
Give wrong labeled pixel penalty: lossfunction(out,gt,reduction=None)*|Sig(out)-gt|
Hinge loss: loss=max(0,1-gt*out)
Explainability and Interpretability
Explaining Explanations: An Overview of Interpretability of Machine Learning