queirozf.com

Entries by tag: paper-summary

Including child/synonym tags

Paper Summary: Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study  06 Oct 2024    paper-summary alignment instruction-tuning
Summary of the 2024 article "Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study" by Xu et al. Read More ›

Paper Summary: The Science of Detecting LLM-Generated Texts  28 Jul 2024    paper-summary language-modeling
Summary of the 2023 article "The Science of Detecting LLM-Generated Texts" by Tang et al. Read More ›

Paper Summary: Few-shot Fine-Tuning vs In-context Learning: a Fair Comparison and Evaluation  22 Jul 2024    paper-summary
Summary of the 2023 article "Few-shot Fine-Tuning vs In-context Learning: a Fair Comparison and Evaluation" by Mosbach et al. Read More ›

Paper Summary: Multitask Prompted Training Enables Zero-Shot Task Generalization  31 Mar 2024    paper-summary instruction-tuning language-modeling
Summary of the 2021 article "Multitask Prompted Training Enables Zero-Shot Task Generalization" by Sahn et al. AKA the T0 (T-zero) article Read More ›

Paper Summary: Learning to summarize from human feedback  09 Mar 2024    paper-summary
Summary of the 2020 article "Learning to summarize from human feedback" by Stiennon et al. Read More ›

Paper Summary: Zephyr: Direct Distillation of LM Alignment  02 Jan 2024    paper-summary instruction-tuning
Summary of the 2023 article "Zephyr: Direct Distillation of LM Alignment" by Tunstall et al. Read More ›

Paper Summary: Constitutional AI  16 Nov 2023    paper-summary instruction-tuning language-models
Summary of the 2022 article "Constitutional AI" by Anthropic. Read More ›

Paper Summary: Llama 2: Open Foundation and Fine-Tuned Chat Models  01 Aug 2023    paper-summary instruction-following language-modeling
Summary of the 2023 article "Llama 2: Open Foundation and Fine-Tuned Chat Models" by Touvron et al. Read More ›

Paper Summary: Deep Reinforcement Learning from Human Preferences  15 Jul 2023    paper-summary reinforcement-learning rlhf
Summary of the 2017 article "Deep Reinforcement Learning from Human Preferences" by Christiano et al. AKA the RLHF article. Read More ›

Paper Summary: Fine-tuned Language models are Zero-Shot Learners  02 Jul 2023    paper-summary instruction-following
Summary of the 2022 article "Fine-tuned Language models are Zero-Shot Learners" by Wei et al, aka the FLAN article. Read More ›

Paper Summary: Cross-Task Generalization via Natural Language Crowdsourcing Instructions  25 Jun 2023    paper-summary
Summary of the 2022 article "Cross-Task Generalization via Natural Language Crowdsourcing Instructions" by Mishra et al. Read More ›

Paper Summary: Direct Preference Optimization: Your Language Model is Secretly a Reward Model  23 Jun 2023    paper-summary instruction-following
Summary of the 2023 article "Direct Preference Optimization: Your Language Model is Secretly a Reward Model" by Rafailov et al. Read More ›

Paper Summary: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling  18 Jun 2023    paper-summary language-models
Summary of the 2023 article "Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling" by Biderman et al. Read More ›

Paper Summary: LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention  04 Jun 2023    paper-summary language-modeling instruction-following
Summary of the 2023 article "LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention" by Zhang et al. Read More ›

Paper Summary: LLaMA: Open and Efficient Foundation Language Models  04 Jun 2023    paper-summary llms
Summary of the 2023 article "LLaMA: Open and Efficient Foundation Language Models" by Touvron et al. Read More ›

Paper Summary: Self-instruct: Aligning Language Models with Self-generated Instructions  03 Jun 2023    paper-summary language-modeling alignment
Summary of the 2022 article "Self-instruct: Aligning Language Models with Self-generated Instructions" by Wang et al. Read More ›

Paper Summary: Training language models to follow instructions with human feedback  05 Feb 2023    paper-summary language-models alignment
Summary of the 2022 article "Training language models to follow instructions with human feedback" by Ouyang et al. AKA the InstructGPT article Read More ›

Paper Summary: Language Models are Few-Shot Learners  01 Jan 2023    paper-summary language-models
Summary of the 2020 article "Language Models are Few-Shot Learners" by Brown et al. AKA the GPT-3 Paper. Read More ›

Paper Summary: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding  01 Jan 2023    paper-summary language-models
Summary of the 2018 article "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" by Devlin et al. Read More ›

Paper Summary: Long Short-Term Memory-Networks for Machine Reading  25 Dec 2022    paper-summary attention sequence-learning
Summary of the 2016 article "Long Short-Term Memory-Networks for Machine Reading" by Cheng et al. AKA the "Self-attention" article Read More ›

Paper Summary: Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer  29 Aug 2021    paper-summary natural-language-processing
Summary of the 2020 article "Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer" by Raffel et al. AKA the T5 article. Read More ›

Paper Summary: Identifying Mislabeled Instances in Classification Datasets  28 Jun 2021    paper-summary machine-learning-engineering machine-learning
Summary of the 2019 article "Identifying Mislabeled Instances in Classification Datasets" by Mueller and Markert. Read More ›

Paper Summary: The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets  29 Mar 2021    paper-summary model-evaluation
Summary of the 2015 article "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets" by Saito and Hemsmeier. Read More ›

Paper Summary: Improving Language Understanding by Generative Pre-Training  11 Sep 2020    paper-summary natural-language-processing sequence-learning transformer-architecture
Summary of the 2018 article "Improving Language Understanding by Generative Pre-Training" by Radford et al. Read More ›

Paper Summary: ULMFIT: Universal Language Model Fine-tuning for Text Classification  22 Jul 2020    paper-summary natural-language-processing embeddings sequence-learning
Summary of the 2018 article "ULMFIT: Universal Language Model Fine-tuning for Text Classification" by Howard and Ruder. Read More ›

Paper Summary: Attention is All you Need  27 Jun 2020    paper-summary sequence-learning attention transformer-architecture
Summary of the 2017 article "Attention is All you Need" by Vaswani et al. Read More ›

Paper Summary: Hidden Technical Debt in Machine Learning Systems  23 Mar 2020    paper-summary machine-learning-engineering technical-debt
Summary of the 2015 article "Hidden Technical Debt in Machine Learning Systems" by Sculley et al. Read More ›

Paper Summary: Software Engineering for Machine Learning: A Case Study  25 Jan 2020    paper-summary machine-learning-engineering software-engineering
Summary of the 2019 article "Software Engineering for Machine Learning: A Case Study" by Amershi et al. Read More ›

Paper Summary: Neural Machine Translation by Jointly Learning to Align and Translate  11 Jan 2020    paper-summary attention sequence-learning machine-translation
Summary of the 2014 article "Neural Machine Translation by Jointly Learning to Align and Translate" by Bahdanau et al. Read More ›

Paper Summary: Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift  23 Dec 2019    paper-summary machine-learning-engineering
Summary of the 2019 article "Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift" by Rabanser et al. Read More ›

Paper Summary: Long Short-Term Memory  16 Nov 2019    paper-summary neural-networks sequence-learning
Summary of the 1997 article "Long Short-Term Memory" by Hochreiter and Schmidhuber. Read More ›

Paper Summary: 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com  09 Nov 2019    paper-summary machine-learning-engineering
Summary of the 2019 article "150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com" by Bernardi et al. Read More ›

Paper Summary: TextRank: Bringing Order into Texts  16 Sep 2019    paper-summary natural-language-processing
Summary of the 2004 article "TextRank: Bringing Order into Texts" by Mihalcea and Tarau. Read More ›

Paper Summary: Language Models are Unsupervised Multitask Learners  31 Aug 2019    paper-summary language-models
Summary of the 2019 article "Language Models are Unsupervised Multitask Learners" by Radford et al. AKA the GPT-2 Article. Read More ›

Paper Summary: Sequence to Sequence Learning with Neural Networks  14 Jul 2019    paper-summary sequence-learning
Summary of the 2014 article "Sequence to Sequence Learning with Neural Networks" by Sutskever et al. Read More ›

Paper Summary: A New Probabilistic Model for Title Generation  30 Jun 2019    paper-summary
Summary of the 2002 article "A New Probabilistic Model for Title Generation" by Jin and Hauptmann. Read More ›

Paper Summary: Text Summarization Techniques: A Brief Survey  29 Jun 2019    paper-summary surveys
Summary of the 2017 article "Text Summarization Techniques: A Brief Survey" by Allahyari et al. Read More ›

Paper Summary: From Word to Sense Embeddings: A Survey on Vector Representations of Meaning  02 Jun 2019    paper-summary embeddings surveys
Summary of the 2018 article "From Word to Sense Embeddings: A Survey on Vector Representations of Meaning" by Camacho-Collados and Pilehvar. Read More ›

Paper Summary: DTATG: An Automatic Title Generator Based on Dependency Trees  30 May 2019    paper-summary
Summary of the 2017 article "DTATG: An Automatic Title Generator Based on Dependency Trees" by Shao and Wang. Read More ›

Paper Summary: Scaling Distributed Machine Learning with the Parameter Server  25 May 2019    paper-summary machine-learning-engineering distributed-computing
Summary of the 2014 article "Scaling Distributed Machine Learning with the Parameter Server" by Li et al. Read More ›

Paper Summary: Large Margin Methods for Structured and Interdependent Output Variables  24 Jan 2019    paper-summary multi-label-learning structured-learning
Summary of the 2005 article "Large Margin Methods for Structured and Interdependent Output Variables" by Tsochantaridis et al. Read More ›

Paper Summary: The Tradeoffs of Large Scale Learning  15 Dec 2018    paper-summary machine-learning
Summary of the 2007 article "The Tradeoffs of Large Scale Learning" by Bottou and Bousquet. Read More ›

Paper Summary: Statistical Modeling: The Two Cultures  02 Nov 2018    paper-summary machine-learning
Summary of the 2001 article "Statistical Modeling: The Two Cultures" by Leo Breiman. Read More ›

Paper Summary: SMOTE: Synthetic Minority Over-sampling Technique  02 Sep 2018    paper-summary
Summary of the 2002 article "SMOTE: Synthetic Minority Over-sampling Technique" by Chawla et al. Read More ›

Paper Summary: Multi-Label Classification on Tree- and DAG-Structured Hierarchies  02 Jul 2018    paper-summary multi-label structured-learning hierarchical-learning natural-language-processing
Summary of the 2011 article "Multi-Label Classification on Tree- and DAG-Structured Hierarchies" by Bi and Kwok. Read More ›

Paper Summary: The Natural Language Decathlon: Multitask Learning as Question Answering  30 Jun 2018    paper-summary natural-language-processing
Summary of the 2018 article "The Natural Language Decathlon: Multitask Learning as Question Answering" by McCann et al. Read More ›

Paper Summary: A Simple but Tough-to-beat Baseline for Sentence Embeddings  13 May 2018    paper-summary embeddings compositionality natural-language-processing
Summary of the 2017 article "A Simple but Tough-to-beat Baseline for Sentence Embeddings" by Arora et al. Read More ›

Paper Summary: Context is Everything: Finding Meaning Statistically in Semantic Spaces  01 May 2018    paper-summary compositionality embeddings natural-language-processing
Summary of the 2018 article "Context is Everything: Finding Meaning Statistically in Semantic Spaces" by Zelikman, where the author introduces CoSal weighting for bag-of-words vectors. Read More ›

Paper Summary: Distributed Representations of Sentences and Documents  29 Nov 2017    paper-summary
Summary of the 2014 article "Distributed Representations of Sentences and Documents" by Le and Mikolov. Read More ›

Paper Summary: Multi-instance multi-label learning for automatic tag recommendation  05 Nov 2017    paper-summary multi-label tags
Summary of the 2009 article "Multi-instance multi-label learning for automatic tag recommendation" by Shen et al. Read More ›

Paper Summary: WSABIE: Scaling Up To Large Vocabulary Image Annotation  05 Oct 2017    paper-summary embeddings tags
Summary of the 2011 article "WSABIE: Scaling Up To Large Vocabulary Image Annotation" by Weston et al. Read More ›

Paper Summary: Recursive Neural Language Architecture for Tag Prediction  05 Oct 2017    paper-summary tags neural-nets embeddings
Summary of the 2016 article "Recursive Neural Language Architecture for Tag Prediction" by Kataria. Read More ›

Paper Summary: Translating Embeddings for Modeling Multi-relational Data  01 Oct 2017    embeddings structure paper-summary neural-networks
Summary of the 2013 article "Translating Embeddings for Modeling Multi-relational Data" by Bordes et al. Read More ›