research
∙
07/05/2023
CAME: Confidence-guided Adaptive Memory Efficient Optimization
Adaptive gradient methods, such as Adam and LAMB, have demonstrated exce...
share
research
∙
09/15/2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation
Pre-trained language models have shown remarkable results on various NLP...
research
∙
07/15/2021
AutoBERT-Zero: Evolving BERT Backbone from Scratch
Transformer-based pre-trained language models like BERT and its variants...
research
∙
02/25/2021
SparseBERT: Rethinking the Importance Analysis in Self-attention
Transformer-based models are popular for natural language processing (NL...
research
∙
08/31/2019