Entries by tag: natural-language-processing

Including child/synonym tags

Paper Summary: Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study  06 Oct 2024    paper-summary alignment instruction-tuning

Summary of the 2024 article "Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study" by Xu et al. Read More ›

Paper Summary: The Science of Detecting LLM-Generated Texts  28 Jul 2024    paper-summary language-modeling

Summary of the 2023 article "The Science of Detecting LLM-Generated Texts" by Tang et al. Read More ›

Paper Summary: Multitask Prompted Training Enables Zero-Shot Task Generalization  31 Mar 2024    paper-summary instruction-tuning language-modeling

Summary of the 2021 article "Multitask Prompted Training Enables Zero-Shot Task Generalization" by Sahn et al. AKA the T0 (T-zero) article Read More ›

Paper Summary: Constitutional AI  16 Nov 2023    paper-summary instruction-tuning language-models

Summary of the 2022 article "Constitutional AI" by Anthropic. Read More ›

Paper Summary: Llama 2: Open Foundation and Fine-Tuned Chat Models  01 Aug 2023    paper-summary instruction-following language-modeling

Summary of the 2023 article "Llama 2: Open Foundation and Fine-Tuned Chat Models" by Touvron et al. Read More ›

Paper Summary: Fine-tuned Language models are Zero-Shot Learners  02 Jul 2023    paper-summary instruction-following

Summary of the 2022 article "Fine-tuned Language models are Zero-Shot Learners" by Wei et al, aka the FLAN article. Read More ›

Paper Summary: Direct Preference Optimization: Your Language Model is Secretly a Reward Model  23 Jun 2023    paper-summary instruction-following

Summary of the 2023 article "Direct Preference Optimization: Your Language Model is Secretly a Reward Model" by Rafailov et al. Read More ›

Paper Summary: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling  18 Jun 2023    paper-summary language-models

Summary of the 2023 article "Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling" by Biderman et al. Read More ›

Paper Summary: LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention  04 Jun 2023    paper-summary language-modeling instruction-following

Summary of the 2023 article "LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention" by Zhang et al. Read More ›

Paper Summary: LLaMA: Open and Efficient Foundation Language Models  04 Jun 2023    paper-summary llms

Summary of the 2023 article "LLaMA: Open and Efficient Foundation Language Models" by Touvron et al. Read More ›

Paper Summary: Self-instruct: Aligning Language Models with Self-generated Instructions  03 Jun 2023    paper-summary language-modeling alignment

Summary of the 2022 article "Self-instruct: Aligning Language Models with Self-generated Instructions" by Wang et al. Read More ›

Paper Summary: Training language models to follow instructions with human feedback  05 Feb 2023    paper-summary language-models alignment

Summary of the 2022 article "Training language models to follow instructions with human feedback" by Ouyang et al. AKA the InstructGPT article Read More ›

Paper Summary: Language Models are Few-Shot Learners  01 Jan 2023    paper-summary language-models

Summary of the 2020 article "Language Models are Few-Shot Learners" by Brown et al. AKA the GPT-3 Paper. Read More ›

Paper Summary: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding  01 Jan 2023    paper-summary language-models

Summary of the 2018 article "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" by Devlin et al. Read More ›

Paper Summary: Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer  29 Aug 2021    paper-summary natural-language-processing

Summary of the 2020 article "Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer" by Raffel et al. AKA the T5 article. Read More ›

Normalize Text for Natural Language Processing Tasks: Reference and Examples  02 May 2021    nlp preprocessing python

A couple of common preprocessing tasks you need in order to be able to use raw text in NLP tools. Read More ›

Paper Summary: Improving Language Understanding by Generative Pre-Training  11 Sep 2020    paper-summary natural-language-processing sequence-learning transformer-architecture

Summary of the 2018 article "Improving Language Understanding by Generative Pre-Training" by Radford et al. Read More ›

Paper Summary: ULMFIT: Universal Language Model Fine-tuning for Text Classification  22 Jul 2020    paper-summary natural-language-processing embeddings sequence-learning

Summary of the 2018 article "ULMFIT: Universal Language Model Fine-tuning for Text Classification" by Howard and Ruder. Read More ›

Project Review: Text Classification of Legal Documents (Another one)  25 Apr 2020    project-review natural-language-processing

Short review with lessons learned for a contract project worked on during early 2020. The aim of the project was to classify documents into classes, with some peculiarities and specific rules. Read More ›

Paper Summary: TextRank: Bringing Order into Texts  16 Sep 2019    paper-summary natural-language-processing

Summary of the 2004 article "TextRank: Bringing Order into Texts" by Mihalcea and Tarau. Read More ›

Paper Summary: Language Models are Unsupervised Multitask Learners  31 Aug 2019    paper-summary language-models

Summary of the 2019 article "Language Models are Unsupervised Multitask Learners" by Radford et al. AKA the GPT-2 Article. Read More ›

Project Review: Text Classification of Legal Documents  02 Nov 2018    project-review natural-language-processing

Lessons learned from a data science project. Read More ›

Paper Summary: Multi-Label Classification on Tree- and DAG-Structured Hierarchies  02 Jul 2018    paper-summary multi-label structured-learning hierarchical-learning natural-language-processing

Summary of the 2011 article "Multi-Label Classification on Tree- and DAG-Structured Hierarchies" by Bi and Kwok. Read More ›

Paper Summary: The Natural Language Decathlon: Multitask Learning as Question Answering  30 Jun 2018    paper-summary natural-language-processing

Summary of the 2018 article "The Natural Language Decathlon: Multitask Learning as Question Answering" by McCann et al. Read More ›

Paper Summary: A Simple but Tough-to-beat Baseline for Sentence Embeddings  13 May 2018    paper-summary embeddings compositionality natural-language-processing

Summary of the 2017 article "A Simple but Tough-to-beat Baseline for Sentence Embeddings" by Arora et al. Read More ›

Paper Summary: Context is Everything: Finding Meaning Statistically in Semantic Spaces  01 May 2018    paper-summary compositionality embeddings natural-language-processing

Summary of the 2018 article "Context is Everything: Finding Meaning Statistically in Semantic Spaces" by Zelikman, where the author introduces CoSal weighting for bag-of-words vectors. Read More ›

Word2vec Quick Tutorial using the Default Implementation in C  23 May 2015    word2vec word-embeddings

Read More ›