Blog Archive

Paper Summary: Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study 06 Oct 2024 paper-summary alignment instruction-tuning

Summary of the 2024 article "Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study" by Xu et al. Read More ›

Paper Summary: The Science of Detecting LLM-Generated Texts 28 Jul 2024 paper-summary language-modeling

Summary of the 2023 article "The Science of Detecting LLM-Generated Texts" by Tang et al. Read More ›

Paper Summary: Few-shot Fine-Tuning vs In-context Learning: a Fair Comparison and Evaluation 22 Jul 2024 paper-summary

Summary of the 2023 article "Few-shot Fine-Tuning vs In-context Learning: a Fair Comparison and Evaluation" by Mosbach et al. Read More ›

Paper Summary: Multitask Prompted Training Enables Zero-Shot Task Generalization 31 Mar 2024 paper-summary instruction-tuning language-modeling

Summary of the 2021 article "Multitask Prompted Training Enables Zero-Shot Task Generalization" by Sahn et al. AKA the T0 (T-zero) article Read More ›

Paper Summary: Learning to summarize from human feedback 09 Mar 2024 paper-summary

Summary of the 2020 article "Learning to summarize from human feedback" by Stiennon et al. Read More ›

Paper Summary: Zephyr: Direct Distillation of LM Alignment 02 Jan 2024 paper-summary instruction-tuning

Summary of the 2023 article "Zephyr: Direct Distillation of LM Alignment" by Tunstall et al. Read More ›

Examples: Installing and Updating Packages with Apt 01 Jan 2024 ubuntu apt

Examples on how to use apt-get and dpkg to install, uninstall, and list packages on Ubuntu and similar systems. Read More ›

Git Examples: Reverting a File from a Branch 27 Dec 2023 git

Many ways to reset files to what they look like on other branches, both locally and on remotes. Read More ›

Paper Summary: Constitutional AI 16 Nov 2023 paper-summary instruction-tuning language-models

Summary of the 2022 article "Constitutional AI" by Anthropic. Read More ›

Pytest Examples: Handling Exceptions 18 Oct 2023 python pytest

Examples on how to use Pytest to handle exceptions Read More ›

Troubleshooting Colima Start Problems 01 Oct 2023 docker colima macos

Troubleshooting problems related to starting colima docker environments on a MacOS box. Read More ›

Pyenv and Jupyter Notebook Integration 17 Aug 2023 pyenv jupyter

Examples and reference on how to create and manage Kernels in Jupyter from a Pyenv environment. Read More ›

Sublime 4 Productivity Examples: Keymaps, Snippets, Macros 05 Aug 2023 sublime-text

Simple reminders on how to add keymaps and run snippets on Sublime text 4. Read More ›

Paper Summary: Llama 2: Open Foundation and Fine-Tuned Chat Models 01 Aug 2023 paper-summary instruction-following language-modeling

Summary of the 2023 article "Llama 2: Open Foundation and Fine-Tuned Chat Models" by Touvron et al. Read More ›

Python Dependency Management: Examples and Reference 22 Jul 2023 python

Examples on how to use python tools to manage dependencies. Read More ›

Paper Summary: Deep Reinforcement Learning from Human Preferences 15 Jul 2023 paper-summary reinforcement-learning rlhf

Summary of the 2017 article "Deep Reinforcement Learning from Human Preferences" by Christiano et al. AKA the RLHF article. Read More ›

Jenv Examples on MacOS 07 Jul 2023 jenv

Examples on how to config multiple java versions on MacOS using jenv Read More ›

Paper Summary: Fine-tuned Language models are Zero-Shot Learners 02 Jul 2023 paper-summary instruction-following

Summary of the 2022 article "Fine-tuned Language models are Zero-Shot Learners" by Wei et al, aka the FLAN article. Read More ›

Paper Summary: Cross-Task Generalization via Natural Language Crowdsourcing Instructions 25 Jun 2023 paper-summary

Summary of the 2022 article "Cross-Task Generalization via Natural Language Crowdsourcing Instructions" by Mishra et al. Read More ›

Python 3 Regex: Named Capture Examples 25 Jun 2023 python-3 regex

Examples on how to use named capture gropus in Python regular expressions. Read More ›

Paper Summary: Direct Preference Optimization: Your Language Model is Secretly a Reward Model 23 Jun 2023 paper-summary instruction-following

Summary of the 2023 article "Direct Preference Optimization: Your Language Model is Secretly a Reward Model" by Rafailov et al. Read More ›

Pyenv Examples: Managing multiple Python versions and Virtualenvs 21 Jun 2023 pyenv

Examples on how to use pyenv to handle Virtualenvs and python versions on Unix-like systems such as Ubuntu and MacOS. Read More ›

Paper Summary: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling 18 Jun 2023 paper-summary language-models

Summary of the 2023 article "Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling" by Biderman et al. Read More ›

Paper Summary: LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention 04 Jun 2023 paper-summary language-modeling instruction-following

Summary of the 2023 article "LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention" by Zhang et al. Read More ›

Paper Summary: LLaMA: Open and Efficient Foundation Language Models 04 Jun 2023 paper-summary llms

Summary of the 2023 article "LLaMA: Open and Efficient Foundation Language Models" by Touvron et al. Read More ›

Paper Summary: Self-instruct: Aligning Language Models with Self-generated Instructions 03 Jun 2023 paper-summary language-modeling alignment

Summary of the 2022 article "Self-instruct: Aligning Language Models with Self-generated Instructions" by Wang et al. Read More ›

As a Manager: Is it Worthwhile? How Worthwhile? 28 May 2023 management

Some thoughts about the frame of mind you need to get into as you transition to a technical management role. Read More ›

As a Manager: Stating the Obvious is Important 28 May 2023 management

Examples and an overview on why managers should state the obvious even when it may seem unnecessary. Read More ›

As a Manager: Drive Growth by Asking Open-Ended Questions 22 May 2023 management

One of the ways you can foster growth in reports is to ask open-ended questions. Read More ›

Paper Summary: Training language models to follow instructions with human feedback 05 Feb 2023 paper-summary language-models alignment

Summary of the 2022 article "Training language models to follow instructions with human feedback" by Ouyang et al. AKA the InstructGPT article Read More ›

Paper Summary: Language Models are Few-Shot Learners 01 Jan 2023 paper-summary language-models

Summary of the 2020 article "Language Models are Few-Shot Learners" by Brown et al. AKA the GPT-3 Paper. Read More ›

Paper Summary: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 01 Jan 2023 paper-summary language-models

Summary of the 2018 article "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" by Devlin et al. Read More ›

Paper Summary: Long Short-Term Memory-Networks for Machine Reading 25 Dec 2022 paper-summary attention sequence-learning

Summary of the 2016 article "Long Short-Term Memory-Networks for Machine Reading" by Cheng et al. AKA the "Self-attention" article Read More ›

As a Manager: Tell Reports "Why" as Often as Possible 12 Dec 2022 management soft-skills

Asking somebody to do something without fully telling them the underlying reasons saves time in the short run but prevents growth in the long run. As often as you can, tell people the why behind a given task or decision. Read More ›

Mini-Tutorial: Installing PyTorch (CPU-only) on Ubuntu 28 Nov 2022 pytorch ubuntu

Some notes on how to install PyTorch (CPU-only for now) on Ubuntu Read More ›

Pandas Fillna Examples: Filling in Missing Data 16 Oct 2022 pandas

Examples on the most common ways you will find yourself using fillna and related functions in pandas. Read More ›

As a Manager: Give Feedback on Specifics Only 15 Oct 2022 management

As a manager, why you should try and keep feedbacks on specific actions and behaviours, rather than giving out generic, subjective advice. Read More ›

Git config Examples 15 Oct 2022 git

Quick examples for some common configuration for git and related tools. Read More ›

Brainstorming Tech Content 15 Oct 2022 meta

Meta content on writing tech content such as this blog Read More ›

Splunk examples: Dealing with null/empty values 10 Oct 2022 splunk

Dealing with NULL and/or empty values in splunk. Examples with the most common use cases and problems you may face. Read More ›

Splunk Examples: Regex command 10 Oct 2022 splunk regex

Ese the regex command in splunk to have regex-like (perl-compatible) queries and filters. Read More ›

Grep examples: Perl-Compatible Regular Expressions 09 Oct 2022 grep regex

Examples on how to use regular expressions in conjunction with GNU grep. Examples on how to modify regular expressions, use multiple types, common issues and solutions. Examples mostly use Perl-compatible regular expressions (PCRE) Read More ›

Splunk Examples: Calculating Ratios Between Events 14 Sep 2022 splunk

Splunk examples on how to take data from several sources and combine them to calculate ratios or percentages between one event and the other, relative counts, etc. Read More ›

Splunk Examples: Manipulating Text and Strings 12 Sep 2022 splunk

Examples on how to perform common operations on strings within splunk queries. Read More ›

qpdf Examples: Merging, Splitting, Compressing PDF Files 01 Aug 2022 pdf

Examples on common operations you may want to do on PDF files using qpdf. Examples from unix-like systems. Read More ›

Pandas Dataframe examples: Plotting Histograms 31 Jul 2022 matplotlib pandas

Several examples on how to draw histograms based on pandas dataframes. Read More ›

Action-based VS Process-based Solutions to Problems 24 Jul 2022 management

Solving a problem once is very different from setting up a system that ensures the problem gets addressed consistently. In other words, solving problems in the small is different from solving those very problems "in the large", at scale. Read More ›

Git Examples: Comparing Branches, locally and remotely 24 Jul 2022 git

Comparing and diffing code across branches and working directories on git. Read More ›

Splunk Examples: Renaming/Replacing Fields and Values 13 Jun 2022 splunk

Renaming and replacing fields, values, etc on Splunk. Examples and reference using the tutorial data from the docs. Read More ›

Pandas Examples: Looping over Dataframe Rows 13 Jun 2022 pandas

Everything you need to know about how to loop and/or iterate over rows in a pandas dataframe, as efficiently as possible. Read More ›

Spark SQL date/time Arithmetic examples: Adding, Subtracting, etc 12 Jun 2022 spark datetime

Examples on how to subtract, add dates and timestamps in Spark SQL Dataframes, along with a summary. Read More ›

Java.sql for Spark Scala: Examples using Dates, Times, etc 31 May 2022 java spark

Common operations to make use of java.sql Date/Time classes, convert to/from java.time Classes, etc. Read More ›

Python Date/Datetime Arithmetic Examples: Adding, Subtracting, etc 08 May 2022 python datetime

Some examples on how to perform basic arithmetic on Python date and datetime objects, examples on timedelta, etc. Read More ›

Add Per-core Metrics on indicator-multiload on Ubuntu 08 May 2022 ubuntu

Add individual metrics for each core, such as usage percentage for the basic ubuntu indicator-multiload plugin. Read More ›

Matplotlib Examples: Drawing Straight Lines 06 May 2022 matplotlib

Examples showing you how to draw several types of straight lines on matplotlib, horizontal, vertical, at angles, dashed, solid, etc Read More ›

Update Alternatives: Changing Java version on Ubuntu 02 May 2022 java ubuntu

How to list and change java version on ubuntu. Read More ›

Pandas Examples: Plotting Date/Time data with Matplotlib/Pyplot 24 Apr 2022 pandas matplotlib

Examples on how to plot time-series or general date or time data from a pandas dataframe, using matplotlib behind the scenes. Read More ›

If you use Git you should be very Liberal in Deleting Stale Code 21 Apr 2022 software-engineering technical-debt

On the costs of keeping stale code around and how to recover old code using git, if needed. Read More ›

Apache Spark Examples: Dataframe and Column Aliasing 29 Dec 2021 spark

Examples on how to add and work with dataframe and column aliases on Spark SQL. Examples using Scala. Also includes common error messages and common questions. Read More ›

How to Ask for Tech Support 24 Dec 2021 soft-skills

Asking questions and requesting support the right way, to increase the odds your problems will actually get solved, while maintaining good relations with support staff. Read More ›

Complex Code is Technical Debt, and it will Charge Interest 28 Nov 2021 software-engineering

Why complex code is technical debt and how it hurts you, your teams and your company. And what to do about it. Read More ›

JupyterLab: Sane Defaults and Extensions for Basic DS Work 08 Nov 2021 jupyterlab jupyter-notebooks

Minimum Viable Jupyterlab configuration for reasonably pleasant DS work in Python Read More ›

Git Examples: Searching the Git History 01 Nov 2021 git

Examples on how to search the git logs and history for modified files, modified code, how to retrieve past files from the history, etc. Read More ›

Bash Options with Set: Examples and reference 31 Oct 2021 bash

Examples and reference for most commonly-used bash options you can set/unset with the set builtin. Read More ›

Paper Summary: Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer 29 Aug 2021 paper-summary natural-language-processing

Summary of the 2020 article "Exploring the Limits of Transfer Learning with a Unified Text-to-text Transformer" by Raffel et al. AKA the T5 article. Read More ›

Python 3 Examples: Creating, Deleting, Updating directories 08 Aug 2021 python files

Lots of examples on how to create, remove, change and otherwise interact with directories with Python 3+ (Assuming unix-like Operating Systems such as Linux and MacOS) Read More ›

Measuring Function Running Times in Python 08 Jul 2021 python

Really simple code to help profile individual method calls to find bottlenecks in Python code. Read More ›

Matplotlib examples: Number Formatting for Axis Labels 29 Jun 2021 matplotlib formatting

Some examples on how to properly format axis labels, add thousands separator, format axis labels to make them easier to read, etc. Read More ›

Matplotlib Examples: Plots with String Axis Labels 28 Jun 2021 matplotlib

Examples on how to create plots where one of the axis is not a list of integers/floats but a list of strings. Read More ›

Paper Summary: Identifying Mislabeled Instances in Classification Datasets 28 Jun 2021 paper-summary machine-learning-engineering machine-learning

Summary of the 2019 article "Identifying Mislabeled Instances in Classification Datasets" by Mueller and Markert. Read More ›

Probabilistic Behaviour can Sometimes Replace State at Scale 28 Jun 2021 scalability software-architecture

You can sometimes forgo keeping state for an application if you are operating at scale and you can afford some innacuracies. Read More ›

AWS Examples: Change EC2 Instance Type without Losing Data 26 Jun 2021 aws ec2 ebs

You can change the instance type for an EC2 instance by detaching the base volume (where the OS is installed), then re-attaching it to another instance (of the desired type) and then copying files manually on demand. Read More ›

Assign Command to Keyboard Shortcut on Ubuntu 30 May 2021 ubuntu

Assign hotkeys and keyboard shortcuts to commonly used terminal commands on Ubuntu Read More ›

Set up Heartbeat Monitoring on AWS Cloudwatch with Boto: Example 30 May 2021 aws-cloudwatch boto

How to create a heartbeat check on AWS Cloudwatch, so that you get notified if some event has NOT happened in the last minute/hour/days etc Read More ›

Pandas Dataframe Examples: Styling Cells and Conditional Formatting 09 May 2021 python pandas

Some examples on how to highlight and style cells in pandas dataframes when some criteria is met. Useful for analytics and presenting data. Read More ›

Python Regular Expressions: Lookahead and lookbehind examples 09 May 2021 python regex

Examples for how and when to use lookaheads and lookbehinds in python regular expressions. These are ways to create matches that depend on what came before or what came after the pattern you want to match. Read More ›

Normalize Text for Natural Language Processing Tasks: Reference and Examples 02 May 2021 nlp preprocessing python

A couple of common preprocessing tasks you need in order to be able to use raw text in NLP tools. Read More ›

Bash examples: redirecting output, error, input, etc 25 Apr 2021

Examples on how to redirect the ouutput of commands and scripts to files, STDOUT, STDERR, etc. Read More ›

Git-LFS: Reference and Examples 11 Apr 2021 git

Common tasks and examples related to using Github's LFS (Large File Storage) functionality Read More ›

Project Review: Generating Article Titles from Keywords 31 Mar 2021 project-review sequence-learning generative-models

Another project summary, this time for a client who needed an API-accessible ML system to generate text to be used as article titles, given a keyword as input. Read More ›

Working with Functions in Bash: Reference and Examples 31 Mar 2021 bash

Simple examples and common use cases for functions in shell scripting using bash (bourne-again shell). Read More ›

Markdown cheatsheet and Reference: Using Images 31 Mar 2021 markdown

Quick cheatsheet with common fixes to use images in markdown documents, such as github READMEs. Read More ›

Features in Boosted Tree Algorithms: Engineering, Encoding, Interaction, etc 30 Mar 2021 gradient-boosting feature-engineering

How to encode and engineer features correctly for use in tree boosting algorithms? How are features allowed to interact and sort of information do data scientists need to provide via manual engineering? How? Read More ›

Paper Summary: The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets 29 Mar 2021 paper-summary model-evaluation

Summary of the 2015 article "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets" by Saito and Hemsmeier. Read More ›

The Great Financial Crisis was Triggered by a Series of Unrelated Factors 21 Mar 2021

Thoughts on the Great Financial Crisis. Techy summary of the book "The Big Short" by Michael Lewis Read More ›

Splunk Regular Expressions: Rex Command Examples 21 Mar 2021 splunk regex

Examples of common use cases and for Splunk's rex command, for extracting and matching regular expressions from log data. Read More ›

Splunk Eval Examples 21 Mar 2021 splunk

Collection of examples of Splunk's eval command Read More ›

Splunk Groupby: Examples with Stats 20 Feb 2021 splunk

Examples on how to do aggregate operations on Splunk using the stats and timechart commands. Read More ›

Python Datetime with Timezones: Examples and Reference 03 Jan 2021 python datetime timezones

Examples on how to work with datetimes and timezones, using the standard python library, datetime. Read More ›

Splunk Examples: Timecharts 19 Dec 2020 splunk

Examples and reference for common configurations and use cases for the splunk timechart directive Read More ›

Suppressing, Ignoring Warnings in Python: Reference and Examples 11 Nov 2020 python

Examples on how to suppress warnings in python code, when needed. Can be used in standalone scripts, structured systems and/or jupyter notebooks. Read More ›

Pull-Requests: Best Practices and Lessons Learned 12 Oct 2020 soft-skills

Some tips and lessons learned on creating useful and pleasant PRs that are easy to review. Also some best practices on interacting with reviewers and peers in general. Read More ›

On Rolling out Backwards-Compatible Software Upgrades 10 Oct 2020 software-engineering software-architecture

Thoughts on how to deal with problems that arise from legacy systems that have grown too much; Read More ›

Pandas Dataframes: Apply Examples 26 Sep 2020 pandas

Examples on how to use pandas apply, on columns, dataframes, etc, with best practices and warnings about performance. Read More ›

Python: Working with Paths & the Filesystem 26 Sep 2020 python files paths

Several examples on how to work with paths and files in python, convert from relative to aboslute paths, check if a path points to a valid file, etc. Read More ›

11 Types of Data Products, with Examples 22 Sep 2020 product-management data-science data-products

Here is a list of data products you can build using various types of data science methods. Includes use cases and main techniques for each. Read More ›

Paper Summary: Improving Language Understanding by Generative Pre-Training 11 Sep 2020 paper-summary natural-language-processing sequence-learning transformer-architecture

Summary of the 2018 article "Improving Language Understanding by Generative Pre-Training" by Radford et al. Read More ›

Bash Examples: String Operations 09 Sep 2020 bash

How to interact with strings on bash scripts. Read More ›

jq: Sorting JSON objects 29 Aug 2020 jq

Just simple examples on how to sort objects with jq Read More ›

Pytest Options: Best Practices and Examples 23 Aug 2020 pytest python testing

Examples on how to use pytest to run tests, configure outputs, etc. Read More ›

Paper Summary: ULMFIT: Universal Language Model Fine-tuning for Text Classification 22 Jul 2020 paper-summary natural-language-processing embeddings sequence-learning

Summary of the 2018 article "ULMFIT: Universal Language Model Fine-tuning for Text Classification" by Howard and Ruder. Read More ›

Bash Shopt Examples 05 Jul 2020 bash

Examples and reference on how to work with options on bash scripts via the shopt command Read More ›

Paper Summary: Attention is All you Need 27 Jun 2020 paper-summary sequence-learning attention transformer-architecture

Summary of the 2017 article "Attention is All you Need" by Vaswani et al. Read More ›

Apache Browser Caching: Reference and Examples 24 May 2020 apache-24 caching

Example configurations and reference for setting up browser-side caching for files served by Apache 2.4. Read More ›

Setfacl Examples: Give Users and Groups Access to Files and Directories 23 May 2020 permissions filesystem

Examples on how to use the setfacl util to set permissions for specific users and groups to access files and directories on unix-based systems. Read More ›

Project Review: Text Classification of Legal Documents (Another one) 25 Apr 2020 project-review natural-language-processing

Short review with lessons learned for a contract project worked on during early 2020. The aim of the project was to classify documents into classes, with some peculiarities and specific rules. Read More ›

Python Imports: Reference and Examples 25 Apr 2020 python jupyter-notebooks

Examples on how to deal with imports in Python scripts and jupyter notebooks, how to add custom directories and import from those, etc. Read More ›

Matplotlib Subplots: Best Practices and Examples 20 Apr 2020 matplotlib

Examples on how to plot multiple plots on the same figure using Matplotlib and the interactive interface, pyplot. Includes common use cases and best practices. Read More ›

Dealing with Newline Issues on Text Files: Examples and Reference 09 Apr 2020 text-processing unix

How to work with problems in newline encodings in text files: Read More ›

Jupyter Notebook Extensions: Examples and Reference 03 Apr 2020 jupyter

Examples and reference on how to install and configure additional extensions for jupyter notebooks. Read More ›

Numpy Display Options: Examples and Reference 24 Mar 2020 numpy

Number formatting, array visualization and other options to visualize numpy elements, being especially useful for working in Jupyter notebooks. Read More ›

Pandas Display Options: Examples and Reference 24 Mar 2020 pandas

Variety of examples on how to set display options on Pandas, to control things like the number of rows, columns, number formatting, etc. Especially useful for working in Jupyter notebooks. Read More ›

Pandas Dataframes: CSV Quoting and Escaping Strategies 24 Mar 2020 pandas

Reading and writing pandas dataframes to CSV files in a way that's safe and avoiding problems due to quoting, escaping and encoding issues. Read More ›

Paper Summary: Hidden Technical Debt in Machine Learning Systems 23 Mar 2020 paper-summary machine-learning-engineering technical-debt

Summary of the 2015 article "Hidden Technical Debt in Machine Learning Systems" by Sculley et al. Read More ›

Scikit-learn Pipelines: Custom Transformers and Pandas integration 08 Mar 2020 pandas scikit-learn

Examples and reference on how to write customer transformers and how to create a single sklearn pipeline including both preprocessing steps and classifiers at the end, in a way that enables you to use pandas dataframes directly in a call to fit. Read More ›

Numpy Sampling: Reference and Examples 07 Mar 2020 numpy statistics

Sample from probability distributions and from lists, with and without weights. Examples using Python, Numpy and Scipy. Read More ›

Spark dataframe Examples: Reading and Writing Dataframes 23 Feb 2020 spark scala

Some examples on how to read and write spark dataframes from sources such as S3 and databricks file systems. Read More ›

Paper Summary: Software Engineering for Machine Learning: A Case Study 25 Jan 2020 paper-summary machine-learning-engineering software-engineering

Summary of the 2019 article "Software Engineering for Machine Learning: A Case Study" by Amershi et al. Read More ›

Git branching: Reference and Examples 23 Jan 2020 git

Common use cases and examples for using branches on git, locally and on remotes. Read More ›

Paper Summary: Neural Machine Translation by Jointly Learning to Align and Translate 11 Jan 2020 paper-summary attention sequence-learning machine-translation

Summary of the 2014 article "Neural Machine Translation by Jointly Learning to Align and Translate" by Bahdanau et al. Read More ›

Paper Summary: Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift 23 Dec 2019 paper-summary machine-learning-engineering

Summary of the 2019 article "Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift" by Rabanser et al. Read More ›

Python Unittest Examples: Mocking and Patching 09 Dec 2019 python testing

Simple examples to help you understand when/where to use mocking and patching, so you don't need to skip testing any part of your code. Read More ›

Pandas Dataframe Examples: Duplicated Data 17 Nov 2019 pandas

Deal with duplicated data in pandas: drop, count, show and mark duplicates in pandas dataframes. Read More ›

Paper Summary: Long Short-Term Memory 16 Nov 2019 paper-summary neural-networks sequence-learning

Summary of the 1997 article "Long Short-Term Memory" by Hochreiter and Schmidhuber. Read More ›

Spark SQL Case/When Examples 09 Nov 2019 spark

Case/when clauses are useful to mimic if/else behaviour in SQL and also spark, via when/otherwise clauses. Read More ›

Spark SQL Date/Datetime Function Examples 09 Nov 2019 spark-sql scala

Examples on how to use date and datetime functions for commonly used transformations in spark sql dataframes. Read More ›

Paper Summary: 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com 09 Nov 2019 paper-summary machine-learning-engineering

Summary of the 2019 article "150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com" by Bernardi et al. Read More ›

Using Command-line Tools for Text Data Preprocessing: Examples and Reference 09 Nov 2019 gnu macos unix linux command-line data-science

Use native command-line tools for common tasks related to text preprocessing, like stripping bad characters, normalizing whitespace/newlines, replacing regular expressions, text normalization, etc. They're very fast and work surprisingly well. Read More ›

Matplotlib Errorbar Examples 06 Nov 2019 matplotlib

Examples on how to plot data including spread information such as standard deviations, variance, etc. Read More ›

Git Diff: Reference and Examples 24 Oct 2019 git

How to compare and show differences in files from different branches, different commits, etc. Read More ›

Scala Fold Examples 10 Oct 2019 scala

Examples and reference for Scala fold method. Read More ›

Paper Summary: TextRank: Bringing Order into Texts 16 Sep 2019 paper-summary natural-language-processing

Summary of the 2004 article "TextRank: Bringing Order into Texts" by Mihalcea and Tarau. Read More ›

Conda, Pip, Virtualenv and Pyenv: Commands Compared 16 Sep 2019 pip conda virtualenv

Equivalent commands for conda on the one hand and pip plus virtualenv on the other. Read More ›

GNU Gzip examples 15 Sep 2019 compression gzip

Examples on how to perform simple tasks using the gzip command (along with sister commands such as gunzip and zcat). Read More ›

Python Open File: Reference and Examples 15 Sep 2019 python files

Quick reference for how to open files in Python, encodings, modes, etc. Read More ›

People Skills for Data Science Projects: Lessons Learned 14 Sep 2019 data-science project-work

See a project go from start to finish, know how to create value with data science and machine learning. Read More ›

Python 3 String Operations 11 Sep 2019 python-3 strings

Common operations you will need to do on python strings, within the context of Natural Language Processing (NLP) or otherwise. Read More ›

Paper Summary: Language Models are Unsupervised Multitask Learners 31 Aug 2019 paper-summary language-models

Summary of the 2019 article "Language Models are Unsupervised Multitask Learners" by Radford et al. AKA the GPT-2 Article. Read More ›

Spark Dataframe Examples: Window Functions 22 Aug 2019 spark dataframes scala

Examples on how to do common operations using window functions in apache spark dataframes. Examples using the Spark Scala API. Read More ›

Pandas Indexing Examples: Accessing and Setting Values on DataFrames 21 Aug 2019 pandas dataframes

Some common ways to access rows in a pandas dataframe, includes label-based (loc) and position-based (iloc) accessing. Read More ›

Spark Dataframe Examples: Pivot and Unpivot Data 10 Aug 2019 spark scala

Pivoting and unpivoting are very commonly-used data transformation operations. Use them when you want to switch from a row-based to a column-based view and vice-versa. Read More ›

Jupyter Notebook Kernels: How to Add, Change, Remove 28 Jul 2019 jupyter-notebooks scala spark

Add, remove and change Kernels to use with Jupyter notebook Read More ›

Paper Summary: Sequence to Sequence Learning with Neural Networks 14 Jul 2019 paper-summary sequence-learning

Summary of the 2014 article "Sequence to Sequence Learning with Neural Networks" by Sutskever et al. Read More ›

Paper Summary: A New Probabilistic Model for Title Generation 30 Jun 2019 paper-summary

Summary of the 2002 article "A New Probabilistic Model for Title Generation" by Jin and Hauptmann. Read More ›

Paper Summary: Text Summarization Techniques: A Brief Survey 29 Jun 2019 paper-summary surveys

Summary of the 2017 article "Text Summarization Techniques: A Brief Survey" by Allahyari et al. Read More ›

Choosing C Hyperparameter for SVM Classifiers: Examples with Scikit-Learn 20 Jun 2019 scikit-learn svm

Analysis of the effect of the C parameter on learning SVM models under a noisy data regime. With examples using the Python Library Scikit-learn. Read More ›

Michelangelo Palette Overview 08 Jun 2019 machine-learning-engineering

Overview of Palette, the feature store system that is part of Uber's Michelangelo Mahcine Learning Platform. Based off the talk given at qcon.ai. Read More ›

Paper Summary: From Word to Sense Embeddings: A Survey on Vector Representations of Meaning 02 Jun 2019 paper-summary embeddings surveys

Summary of the 2018 article "From Word to Sense Embeddings: A Survey on Vector Representations of Meaning" by Camacho-Collados and Pilehvar. Read More ›

Helping Data Science Projects Succeed: 5 Tips on how to Avoid Becoming a Statistic 01 Jun 2019 projects data-science project-work

5 real-world tips to help you avoid failures in data science projects. Suitable for both practitioners and project leads. Read More ›

Pandas Dataframe Examples: String Functions 01 Jun 2019 pandas

Pandas exposes a series of string methods that you can use on Series that contain string objects. These are useful for filtering dataframes among other uses. Read More ›

Paper Summary: DTATG: An Automatic Title Generator Based on Dependency Trees 30 May 2019 paper-summary

Summary of the 2017 article "DTATG: An Automatic Title Generator Based on Dependency Trees" by Shao and Wang. Read More ›

Paper Summary: Scaling Distributed Machine Learning with the Parameter Server 25 May 2019 paper-summary machine-learning-engineering distributed-computing

Summary of the 2014 article "Scaling Distributed Machine Learning with the Parameter Server" by Li et al. Read More ›

Traits of the Senior Developer 03 May 2019 engineering

Are you a senior software developer? Read More ›

Pandas Dataframe Examples: Create and Append data 25 Mar 2019 pandas

Examples on how to create dataframes, using lists, dicts and creating empty dataframes then initializing it with data. Read More ›

Matplotlib Examples: Displaying and Configuring Legends 23 Mar 2019 matplotlib

Multiple examples on how to display and customize legends on matplotlib plots. Read More ›

The Calibration-Accuracy Plot: Introduction and Examples 17 Mar 2019 data-science calibration

Model scores don't always tell the whole story. It is much easier to interpret the outputs of machine learning models when the scores are well-calibrated probabilities. When a model's scores match probabilities, it is said that that model is well-calibrated. Read More ›

Pandas Time Series Examples: DatetimeIndex, PeriodIndex and TimedeltaIndex 10 Mar 2019 datetime pandas time-series

How and when to use special pandas Indexes such as DatetimeIndex, PeriodIndex and TimedeltaIndex. These will help you deal with and perform simple operations on time-series data. Read More ›

Pandas Concepts: Reference and Examples 10 Mar 2019 pandas

Short explanations with examples on the main concepts you'll find when using the Pandas library. Read More ›

Evaluation Metrics for Ranking problems: Introduction and Examples 24 Jan 2019 machine-learning model-evaluation

Explanation and examples on how to calculate the performance of ranked predictions for machine learning. Read More ›

Paper Summary: Large Margin Methods for Structured and Interdependent Output Variables 24 Jan 2019 paper-summary multi-label-learning structured-learning

Summary of the 2005 article "Large Margin Methods for Structured and Interdependent Output Variables" by Tsochantaridis et al. Read More ›

Pandas Dataframe Examples: Manipulating Date and Time 15 Jan 2019 pandas datetime

Some examples on how to manipulate dates and times in pandas Dataframes, perform date arithmetic, etc. Read More ›

Recommender Systems: Introduction and Examples 06 Jan 2019 recommender-systems

Quick introduction to recommender systems, use cases and main types. Read More ›

Git remote examples: Interacting with Github and other External Repos 18 Dec 2018 git

Examples on how to perform some common git tasks related to interacting with remotes Read More ›

Paper Summary: The Tradeoffs of Large Scale Learning 15 Dec 2018 paper-summary machine-learning

Summary of the 2007 article "The Tradeoffs of Large Scale Learning" by Bottou and Bousquet. Read More ›

Presto/Athena Examples: Date and Datetime functions 10 Dec 2018 presto datetime date athena

Functions for handling dates and datetimes in Presto and/or AWS Athena. Read More ›

Pandas Dataframe Examples: Column Operations 09 Dec 2018 pandas dataframes

Examples on how to modify pandas DataFrame columns, append columns to dataframes and otherwise transform individual columns. Read More ›

Numpy Array (ndarray) Examples 08 Dec 2018 numpy

Numpy Array-like data structures (ndarrays) are the bread and butter of a lot of data work done these days. Read More ›

Quick Summary + Thoughts on BigHead: AirBNB's ML Platform 03 Dec 2018 ml-platforms

Notes on AirBNB's Bighead ML platform, based off videos and presentations. Read More ›

AWS Glue: Introduction, Examples and Troubleshooting 01 Dec 2018 glue athena

Introduction, examples and troubleshooting tips for using AWS Glue. Read More ›

Thoughts on Michelangelo: Uber's Machine Learning Platform 20 Nov 2018 machine-learning-platforms

Reading and dissecting the way Uber does Machine Learning. Read More ›

Python 3 Subprocess Examples 03 Nov 2018 python3

Examples on how to spawn and control OS-level processes from a python script, using the new Subprocess module. Read More ›

Project Review: Text Classification of Legal Documents 02 Nov 2018 project-review natural-language-processing

Lessons learned from a data science project. Read More ›

Paper Summary: Statistical Modeling: The Two Cultures 02 Nov 2018 paper-summary machine-learning

Summary of the 2001 article "Statistical Modeling: The Two Cultures" by Leo Breiman. Read More ›

Gnu Awk: One-line Examples 28 Sep 2018 awk

Examples on how to use awk commands to process text information on shell scripts. Read More ›

Interactive controls for Jupyter notebooks: Python Examples 20 Sep 2018 jupyter python

Examples on how to add interactive controls to jupyter notebooks. Read More ›

Caching Function Results in Python: Examples using Joblib and Functools 15 Sep 2018 python caching

In many cases (web development, data analysis, etc) it's useful to be able to store results that take long to re-compute. These solutions allow you to save results (either in memory or on disk) so that all function calls after the first one become constant-time lookups. Read More ›

Risk in Machine Learning Models 06 Sep 2018 data-science

Machine Learning models can make actual decisions that affect your business. However, things can go wrong, which introduces risk that must be dealt with. Read More ›

Paper Summary: SMOTE: Synthetic Minority Over-sampling Technique 02 Sep 2018 paper-summary

Summary of the 2002 article "SMOTE: Synthetic Minority Over-sampling Technique" by Chawla et al. Read More ›

Heads-up for Deploying Scikit-learn Models to Production: Quick Checklist 01 Sep 2018 scikit-learn production machine-learning-engineering

A couple of tips for addressing common problems and unexpected situations when using scikit-learn models in production.. Read More ›

Cross-Validation Examples with Scikit-Learn 01 Sep 2018 scikit-learn

Using cross-validation within scikit-learn. Read More ›

Mutate for Pandas Dataframes: Examples with Assign 15 Jul 2018 pandas

Assign is a function that mutates a dataframe in place and can be used for chained operations. Read More ›

Pandas Query Examples: SQL-like queries in dataframes 05 Jul 2018 pandas

Use SQL-like syntax to perform in-place queries on pandas dataframes. Read More ›

Paper Summary: Multi-Label Classification on Tree- and DAG-Structured Hierarchies 02 Jul 2018 paper-summary multi-label structured-learning hierarchical-learning natural-language-processing

Summary of the 2011 article "Multi-Label Classification on Tree- and DAG-Structured Hierarchies" by Bi and Kwok. Read More ›

Paper Summary: The Natural Language Decathlon: Multitask Learning as Question Answering 30 Jun 2018 paper-summary natural-language-processing

Summary of the 2018 article "The Natural Language Decathlon: Multitask Learning as Question Answering" by McCann et al. Read More ›

Example Project Template: Serve a Scikit-learn Model via a Flask API 27 Jun 2018 flask scikit-learn

Full (albeit simple) example on how to create a simple Flask API to serve predictions using a pre-trained scikit-learn model. Includes supporting features such as logging, error handling, input validation, etc. Full code available on Github. Read More ›

Default Permissions for Users, Directories and Files on Linux: Examples 23 Jun 2018 permissions linux

Set default permissions for all files/directories created by a user on Linux using umask. Additionally, use Access Control Lists (ACLs) for all files and directories created under a given directory too. Read More ›

Add Labels and Text to Matplotlib Plots: Annotation Examples 23 Jun 2018 matplotlib labels annotation

Examples on how to add simple annotations and labels to your matplotlib plots. Read More ›

How to Structure Software Projects: Python Example 22 Jun 2018 software-engineering

Here are a couple of templates to structure software projects. You need to think about structure because it is the single largest threat to your codebase as your project grows from small to medium-largish size. If you neglect to maintain project structure you risk having your project suffer death by entropy. Read More ›

Crypto Asset Overview: Ravencoin 16 Jun 2018 cryptoassets

Short overview with the least you need to know about Ravencoin and the RVN token. Read More ›

Pandas Dataframe: Union and Concat Examples 14 Jun 2018 pandas

Emulate SQL union and union all behaviour, among other stuff. Read More ›

Pandas Dataframe: Replace Examples 14 Jun 2018 dataframe

Call the replace method on Pandas dataframes to quickly replace values in the whole dataframe, in a single column, etc. Read More ›

Crypto Asset Overview: 0x Project and the ZRX Token 05 Jun 2018 cryptoassets

Short overview with the least you need to know about the 0x protocol Read More ›

Latex Image Examples: Sizing, Positioning, Captioning, etc 03 Jun 2018 latex

Examples on how to work with images on LaTeX and platforms that support it, such as ShareLaTeX and Overleaf. Read More ›

Evaluation Metrics for Regression Problems: Quick examples + Reference 26 May 2018 machine-learning metrics

Regression problems are evaluated against specific metrics that analyze whether the residuals (difference between actual and predicted values) indicate that a fitted model is a good fit for the data. Here are some of the most commonly-used metrics in that domain. Read More ›

Vim Examples: Search and Replace 24 May 2018 vim

Examples on how to search and replace text on Vim; simple exmaples, using regexes, etc. Read More ›

Paper Summary: A Simple but Tough-to-beat Baseline for Sentence Embeddings 13 May 2018 paper-summary embeddings compositionality natural-language-processing

Summary of the 2017 article "A Simple but Tough-to-beat Baseline for Sentence Embeddings" by Arora et al. Read More ›

Scikit-Learn examples: Making Dummy Datasets 02 May 2018 scikit-learn

Make dummy datasets to test out classifiers and/or parameter configurations in Scikit-learn. Read More ›

Paper Summary: Context is Everything: Finding Meaning Statistically in Semantic Spaces 01 May 2018 paper-summary compositionality embeddings natural-language-processing

Summary of the 2018 article "Context is Everything: Finding Meaning Statistically in Semantic Spaces" by Zelikman, where the author introduces CoSal weighting for bag-of-words vectors. Read More ›

Podcast Episode Overview: What Machine Learning Engineers need to Know 23 Apr 2018 data-science peopleware data-newsletter-5 machine-learning-engineering

Overview of a great podcast episode on how much (if at all) we need a new role for data teams, namely Machine Learning Engineers. Read More ›

Visualizing Machine Learning Models: Examples with Scikit-learn, XGB and Matplotlib 23 Apr 2018 matplotlib machine-learning scikit-learn

Examples on how to use matplotlib and Scikit-learn together to visualize the behaviour of machine learning models, conduct exploratory analysis, etc. Read More ›

Pandas Dataframe: Merge and Join Examples 17 Apr 2018 pandas

Examples on how to use pandas.merge to do SQL-style joins on pandas dataframes. Read More ›

Introduction to AUC and Calibrated Models with Examples using Scikit-Learn 15 Apr 2018 machine-learning data-science model-evaluation

Inspired by a podcast episode by Linear Digressions, which talks about what AUC is and what it is not and why you need well calibrated models if you want to treat their outputs as probabilities. Read More ›

Corda Framework Overview + Examples 07 Apr 2018 dlt jvm

Overview of the main concepts of the Corda framework for building decentralized applications based on Distributed Ledger Technology (DLT). Read More ›

Python Dict Examples 31 Mar 2018 python

Examples of common dict operations. Includes mapping, filtering, dict comprehension and how to work with ordered dicts. Read More ›

Similarity measures and distances: Basic reference and examples for data science practitioners 10 Mar 2018 data-science

Measuring how far apart two points are is not as simple as you think and knowing how to use each can make predictive or exploratory models perform either very poorly or very well. Reference and examples including euclidean distance, manhattan distance, mahalanobis distance, etc. Read More ›

Python Docstrings: Reference & Examples 24 Feb 2018 python documentation testing

Examples to help you document your Python code using any of the commonly used docstring styles. Read More ›

Crypto Asset Overview: Augur Project 18 Feb 2018 cryptoassets

Short overview with the least you need to know about the Augur project and the REP token. Read More ›

Python Logging: Reference and Examples 17 Feb 2018 python logging

Examples on how to setup and configure logging in Python applications. Examples include log rotation, log rotation with compression, logging in Flask applications, etc. Read More ›

Crypto Asset Overview: RChain Cooperative (RHOC) 16 Feb 2018 cryptoassets

Short overview with the least you need to know about the RChain Cooperative. Read More ›

Python Unit Testing with unittest: Reference and Examples 12 Feb 2018 python testing

Testing is needed in all but the most trivial applications. The unittest module is very widely used in Python because it's part of the standard library, starting with Python 2.1. Read More ›

SAS Examples: Proc Freq 08 Feb 2018 sas

Proc freq is one of the most useful SAS proc for data analysis. Here are a couple of example to help you quickly put it to use. Read More ›

Crypto Asset Overview: Oyster Protocol 29 Jan 2018 cryptoassets ads file-storage

Short overview with the least you need to know about Oyster (PRL). Read More ›

Vim Examples: Configuration and the vimrc file 20 Jan 2018 vim

Examples of configuration items you can adjust on vim to have it behave the way you want. Read More ›

Vim Examples: Navigation 20 Jan 2018 vim

Examples on how to move around in vim. Includes things moving across lines, columns, words, etc. Read More ›

SAS Examples: Survey Select 20 Jan 2018 sas

Examples on ways to generate samples from data using SAS. Read More ›

Crypto Asset Overview: Particl (PART) 13 Jan 2018 cryptoassets

Comment on Particl, a crypto asset built for private marketplaces. Read More ›

Linux/Unix Shell Scripting Examples: SSH 06 Jan 2018 bash shell ssh

Examples on how to use SSH to connect to/from remote Linux/Unix machines. Read More ›

Unix cut command: Reference and Examples 22 Dec 2017 unix

The Unix cut command can be used for extracting parts of strings. It's very often used to parse command output. Read More ›

Bash Scripting Examples: Iteration and Arrays 22 Dec 2017 bash

Examples on how to use arrays, iterate through items in arrays, etc. Read More ›

Pandas Dataframe: Plot Examples with Matplotlib and Pyplot 22 Dec 2017 pandas pyplot matplotlib dataframes

Examples on how to plot data directly from a Pandas dataframe, using matplotlib and pyplot. Read More ›

Python CSV Module: Reference and Examples 12 Dec 2017 python csv

Examples on how to read and write CSV files and configure Python's CSV module. Read More ›

Connect to Jupyter Notebook Running on EC2 Instance 09 Dec 2017 jupyter ec2

Spin up and connect to a Jupyter notebook running on a remote EC2 instance. Read More ›

Failed to Connect via SSH to a new Deep Learning AMI EC2 on AWS 09 Dec 2017 aws

Simple Guide to troubleshoot connection issues on Deep Learning AMIs on AWS. Read More ›

Paper Summary: Distributed Representations of Sentences and Documents 29 Nov 2017 paper-summary

Summary of the 2014 article "Distributed Representations of Sentences and Documents" by Le and Mikolov. Read More ›

SAS Examples: Numeric/Character Conversions 28 Nov 2017 sas

Converting values from/to character and numeric values is one of the most common operations in SAS. Here are some examples of things you may want to do. Read More ›

SAS Examples: Proc SQL 28 Nov 2017 sas

Examples on how to write some common operations in SAS Proc SQL. Read More ›

SAS Examples: Arrays and Variable Lists 28 Nov 2017 sas

Examples on how to use arrays and variable lists in SAS. Read More ›

SAS Macro Examples - Control Structures 28 Nov 2017 sas

Example on how to use control structures on SAS Macros. Read More ›

Churn Analysis 101: Quick Introduction and Key Concepts 27 Nov 2017 churn data-science

Simple definitions for churn analysis. Read More ›

Churn Analysis 101: Quick Introduction, Key Concepts 27 Nov 2017 churn data-science

Simple definitions for churn analysis. Read More ›

Gaussian Processes for Classification and Regression: Introduction and Usage 19 Nov 2017 machine-learning statistics

Study guide for understanding Gaussian Processes (also Sparse Gaussian Processes) as applied to classification in machine learning. Read More ›

Package a Python Project and Make it Available via pip install: Simple Example 15 Nov 2017 python data-newsletter-6

It's easy to package some code you wrote as a package. Publish your Python code to PyPi to have other people use and contribute to it! Read More ›

Pip Install a Project having a setup.py file: an Example 11 Nov 2017 python

You can use pip to install a Python project that uses a setup.py setup file. Read More ›

Paper Summary: Multi-instance multi-label learning for automatic tag recommendation 05 Nov 2017 paper-summary multi-label tags

Summary of the 2009 article "Multi-instance multi-label learning for automatic tag recommendation" by Shen et al. Read More ›

Python number formatting examples 05 Nov 2017 python formatting

Examples on how to format numbers in Python, e.g. rounding, truncating, etc. Read More ›

Linux File Processing: head and tail examples 27 Oct 2017 linux

GNU utilities head and tail are very useful for text file processing under Linux and other unix-based OSs. Read More ›

Scikit-Learn Pipeline Examples 21 Oct 2017 scikit-learn

Examples of how to use classifier pipelines on Scikit-learn. Includes examples on cross-validation regular classifiers, meta classifiers such as one-vs-rest and also keras models using the scikit-learn wrappers. Read More ›

Kaggle NYC Taxi Trips Competition: Overview and Results 17 Oct 2017 kaggle

Overview of Kaggle competition: New York City Taxi Trip Duration. Read More ›

Python on Jupyter notebooks: Reference for Common Use Cases 16 Oct 2017 jupyter python

Getting more done with Jupyter notebook, with useful shortcuts and commonly-used functionality. Read More ›

Pandas DataFrame: GroupBy Examples 11 Oct 2017 pandas groupby

Examples of specific ways to do what you want using groupby on Pandas Dataframes. Read More ›

Scaling Data Teams 09 Oct 2017 data-science data-newsletter-5

Needs of data teams are mostly around data access and sharing; Columnar databases are often more efficient for analytics; MS Excel is useful at many scales; Stakeholder communication is important to make your work more relevant; Use metrics to get to know how data products are being used. Read More ›

Matplotlib: Pyplot By Example 05 Oct 2017 pyplot matplotlib

Examples for common operations on PyPlot, like changing figure size, changing title and tick sizes, changing legends, etc. Read More ›

Paper Summary: WSABIE: Scaling Up To Large Vocabulary Image Annotation 05 Oct 2017 paper-summary embeddings tags

Summary of the 2011 article "WSABIE: Scaling Up To Large Vocabulary Image Annotation" by Weston et al. Read More ›

Paper Summary: Recursive Neural Language Architecture for Tag Prediction 05 Oct 2017 paper-summary tags neural-nets embeddings

Summary of the 2016 article "Recursive Neural Language Architecture for Tag Prediction" by Kataria. Read More ›

Thoughts on App Monetization with Examples from Popular Apps 05 Oct 2017 monetization

A couple of thoughts on what approaches seem to work best when optimizing monetization on web/mobile apps. Tips include: Focus on the First Purchase, Mix Free and Paid Features on the same interface, Give away freebies consistently. Read More ›

Parallel For Loops in Python: Examples with Joblib 02 Oct 2017 python parallel

Joblib.Parallel is a simple way to spread your for loops across multiple cores, for parallel execution. Read More ›

How to Make Gif Animations from Screencasts on Ubuntu 01 Oct 2017 ubuntu animations

To make short gif-videos on Ubuntu, you can use Kazam for the Screencasts and then Gifify to turn those videos into gif animations. Read More ›

How to Change the Default Application for a given Extension on Ubuntu 01 Oct 2017 ubuntu

Change the default applications used by certain file extensions. Read More ›

Paper Summary: Translating Embeddings for Modeling Multi-relational Data 01 Oct 2017 embeddings structure paper-summary neural-networks

Summary of the 2013 article "Translating Embeddings for Modeling Multi-relational Data" by Bordes et al. Read More ›

Feature Scaling: Quick Introduction and Examples using Scikit-learn 27 Sep 2017 data-science python data-preprocessing

Feature Scaling techniques (rescaling, standardization, mean normalization, etc) are useful for all sorts of machine learning approaches and *critical* for things like k-NN, neural networks and anything that uses SGD (stochastic gradient descent), not to mention text processing systems. Included examples: rescaling, standardization, scaling to unit length, using scikit-learn. Read More ›

Matplotlib, Pyplot, Pylab etc: What's the difference between these and when to use each? 26 Sep 2017 python data-visualization data-newsletter-5

Do you often get confused with terms like maptlotlib, pyplot, pylab, figures, axes, gcf, gca, etc and wonder what they mean? Matplotlib is the toolkit, PyPlot is an interactive way to use Matplotlib and PyLab is the same thing as PyPlot but with some extra shortcuts. Read More ›

5 Tips for moving your Data Science Operation to the next Level 26 Sep 2017 data-newsletter-5 data-science best-practices

Principles for disciplined data science include: Discoverability, Automation, Collaboration, Empowerment and Deployment. Read More ›

Highlights of the Talk with Dr. Konstan on Recommender Systems 24 Sep 2017 recommender-systems data-newsletter-5

Some highlights of the Podcast Episode with Dr. Joseph Konstan on interesting topics related to Recommender Systems. Discussed topics include serendipity, serpentining, diversity and temporal effects. Read More ›

Seaborn by Example: Data Visualization and Plotting using Python 09 Sep 2017 python data-visualization plotting

Seaborn is a higher-level interface to Matplotlib. It has a more convenient API and has useful data visualization functions right out of the box. Read More ›

Data Provenance: Quick Summary + Reasons Why 07 Sep 2017 data-newsletter-5 data-science

Data Provenance (also called Data Lineage) is version control for data. It refers to keeping track of modifications to datasets you use and train models on. This is crucial in data science projects if you need to ensure data quality and reproducibility. Read More ›

Thoughts on Engaging Users in Social Tagging Systems 05 Sep 2017 social-tagging

Some thoughts on how social tagging systems can foster user engagement with appropriate incentives. Read More ›

Lessons from the Netflix Prize: Changing Requirements and Cost-Effectiveness 04 Sep 2017 data-newsletter-4 recommender-systems

Netflix never really used the #1 winning solution to the Netflix Challenge. Some of the reasons were that just wasn't cost-effective to implement the full thing and another was that requirements had changed. Read More ›

Winning Solutions Overview: Kaggle Instacart Competition 04 Sep 2017 data-newsletter-4 kaggle data-science

The Instacart "Market Basket Analysis" competition focused on predicting repeated orders based upon past behaviour. Among the best-ranking solutings, there were many approaches based on gradient boosting and feature engineering and one approach based on end-to-end neural networks. Read More ›

A Quick Summary of Ensemble Learning Strategies 01 Sep 2017 data-newsletter-4 machine-learning

Ensemble learning refers to mixing the outputs of several classifiers in various ways, so as to get a better result than each classifier individually. Read More ›

Evaluation Metrics for Classification Problems: Quick Examples + References 31 Aug 2017 data-newsletter-4 machine-learning model-evaluation

There are multiple ways to measure your model's performance in machine learning, depending upon what objectives you have in mind. Some of the most important are Accuracy, Precision, Recall, F1 and AUC. Read More ›

Pandas for Large Data: Examples and Tips 13 Aug 2017 pandas performance

In order to successfully work with large data on Pandas, there are some ways to reduce memory usage and make sure you get good speed performance. Read More ›

Suggestions on how to make LinkedIn more relevant 04 Aug 2017 metrics linkedin

LinkedIn is a nice platform for connecting to professional peers but its real value lies, in my opinion, in its potential to the *the* global professional rating system. But it needs some improvement. Read More ›

Quick Reminder: Clustering 29 Jul 2017 reminder hierarchy clustering

Quick reminder on key points regarding clustering (hierarchical and otherwise) Read More ›

Install NodeJS and NPM on Ubuntu 28 Jul 2017 nodejs npm ubuntu

Installing the latest NPM + NodeJS on Ubuntu Read More ›

Python Pickle: examples and reference 12 Jul 2017 python pickle

Pickle is a well-known Python tool for saving arbitrary variable contents into file. Here are a couple of examples and tips on how you can use it to make your data science work more efficient and easily reproducible. Read More ›

Using AWS CodePipeline to Automatically Deploy and Build your App Stored on Github as a Docker-based Beanstalk Application 07 Jul 2017 codebuild codepipeline docker beanstalk continuous-integration continuous-deployment

A full guide on how to set up a continuous deployment pipeline using GitHub and AWS CodePipeline, in order to deploy a Docker-based Beanstalk Application. Read More ›

Scala Testing with Scalatest: Reference and Examples 06 Jul 2017 testing scala

Simple examples using ScalaTest to test your Scala projects. Full introduction and examples of all major testing styles supported by this framework. Read More ›

Debugging NullPointerException in Apache Spark 04 Jun 2017 spark nullpointerexception

A lot of things can go wrong when writing distributed code in Spark. Here's a couple of ways to work around/debug NullPointerExceptions. Read More ›

7z: 7zip Usage Examples on Linux 03 Jun 2017 7zip linux

Examples on how to use 7zip to extract and compress data on Linux. Read More ›

Machine Learning and Data Science: Generally Applicable Tips and Tricks 18 May 2017 machine-learning data-science best-practices

A couple of general, practical tips and tricks that may be used when dealing with data science and/or machine learning problems. Read More ›

Installing and Configuring Golang 1.8 on Ubuntu 16.04 24 Apr 2017 golang ubuntu 16

Short guide to installing and configuring Go v1.8 on Ubuntu 16.04. Read More ›

LaTeX: Table Examples 08 Apr 2017 latex

Couple of examples on how to achieve certain table layouts in LaTeX and platforms that support it, such as ShareLaTeX and Overleaf. Read More ›

Data-related Job Descriptions: Making of a Data Team 19 Mar 2017 data-science

A simple description of some common job titles / positions of may come across when looking at the data work landscape. See what positions may be best suited for yourself and your company. Read More ›

Scikit-Learn Cheatsheet: Reference and Examples 10 Mar 2017 scikit-learn

Just a couple of things you may find yourself doing over and over again when working with scikit-learn. Read More ›

PostgreSQL Common Operations: Reference + Examples 10 Mar 2017 postgres sql ubuntu

Common operations you may first yourself doing over and over again when dealing with PostgreSQL databases. Unless otherwise stated, examples assume version 9 or later, running on Ubuntu. Read More ›

Tricks for Training Neural Nets Faster 20 Feb 2017 neural-nets performance

Tricks and Practical tips for training neural nets faster. Credit is mostly to Geoff Hinton and Yann LeCun. Read More ›

Error when running plot on Octave: Xlib extension GLX missing on display 07 Feb 2017 octave ubuntu

The easiest way to fix this problem is to configure Octave to use gnuplot instead of OpenGL. Read More ›

Python Json Module: Reference and Examples 16 Jan 2017 python json

Examples and best practices for using the python json module. Includes common operations such as reading, writing and formatting as well as solutions to common problems. Read More ›

Logback Reference + Examples 05 Jan 2017 logback

Read More ›

Python Command-Line Scripts: Reference 04 Jan 2017 python cli command-line

Read More ›

Apache Spark Architecture Overview: Jobs, Stages, Tasks, etc 03 Jan 2017 spark architecture

Quick overview of the main architecture components involved in running spark jobs, so you can better understand how to make the best possible use of resources. Read More ›

HDFS Commands: Reference and Examples 25 Dec 2016 hadoop hdfs

List of HDFS commands for filesystem management, along with common use cases. Read More ›

Installing CUDA TK 8 and Tensorflow on a Clean Ubuntu 16.04 Install 24 Dec 2016 ubuntu 16.04 cuda tensorflow

Guiding you through installing the correct NVIDIA drivers, CUDA, cuDNN and tensorflow on an Ubuntu 16.04 box. Read More ›

AWS Kinesis Common Operations using the AWS CLI: Reference & Examples 21 Dec 2016 aws kinesis awscli

A couple of commands you can issue to a Kinesis stream from the AWS cli. Read More ›

Typescript Examples: Type Casting and Type Conversions 20 Dec 2016 typescript

Examples on how to convert between types and/or cast in Typescript. Read More ›

Troubleshooting Ubuntu 16.04 Installation/Graphics card on a new Dell Notebook 17 Dec 2016 ubuntu 16.04 troubleshooting dell

Read More ›

Commonly-used Docker Commands: Reference and Examples 15 Dec 2016 docker

Docker commands you'll find yourself using often. Things like build images, starting containers, accessing containers, etc. Read More ›

Spark Streaming: Commong Pitfalls and Tips for Long-running Streaming Applications 11 Dec 2016 spark-streaming

Running Spark Streaming applications may introduce a couple of problems that you may not face when you are running Spark on Batch mode. Here are a couple of things you may need to take into account to keep long-running spark streaming jobs running smoothly. Read More ›

Bash Regular Expressions: Reference and Examples 04 Dec 2016 bash" regular-expressions

Lots of examples on how to use regular expressions with bash scripts. Read More ›

ApplicationAttemptNotFoundException: Spark Application Stuck in ACCEPTED state on YARN 13 Nov 2016 emr yarn spark

ApplicationAttemptNotFoundException may be cause because the log directory have become too crowded with data. Read More ›

Akka-http Common Operations: Reference + Examples 03 Nov 2016 akka-http akka wip

A couple of things I find myself doing very often when I use akka-http. I believe these may be useful for others too. Read More ›

Looking for Things and using the History on Git: Examples and Reference 28 Sep 2016 git

Couple of examples of how to correctly use git log and other commands that let you use git to search your history for modifications, commit messages and so on. Read More ›

Setup Keras+Theano Backend and GPU on Ubuntu 16.04 11 Sep 2016 keras gpu theano

A couple of pointers on how to get up and running with Keras and Theano on a clean 16.04 Ubuntu machine. Read More ›

Numpy/Scipy Distributions and Statistical Operations: Examples & Reference 10 Sep 2016 numpy statistics

A couple of examples of things you will probably want to do when using numpy and scipy for data work, such as probability distributions, PDFs, CDFs, etc. Read More ›

Docker examples: Cleaning up unused Resources 24 Aug 2016 docker

Read More ›

Publishing an SBT Project onto Bintray: an Example 20 Aug 2016 sbt bintray

A quick example explaining everything you need to do to package an SBT project and publish the binary JAR to bintray, where it can be made available for others too! Read More ›

Json Schema Examples 17 Aug 2016 json wip

Read More ›

POSIX Shell Tests and Conditionals: Examples and Reference 13 Aug 2016 bash posix-shell

Read More ›

Add a Github Project as a Dependency in SBT 13 Aug 2016 sbt

Small example on how to quickly add a project you found on github to your own project as a dependency. Read More ›

Typescript Control Stuctures: Examples + Reference 09 Aug 2016 wip typescript

Read More ›

Package Management with NPM: Examples and Common Use Cases 08 Aug 2016 wip npm

Read More ›

Javascript/Typescript Date and time Operations: Examples using Moment.js 02 Aug 2016 wip typescript date javascript

Read More ›

Gnu Xargs: Examples 28 Jul 2016 xargs

Read More ›

Angular 2 App Running on Nginx on Docker: A Simple Example 25 Jul 2016 docker angular 2 nginx

Read More ›

POSIX Shell: Basic Examples and Common Operations 13 Jul 2016 shell sh posix-shell

Read More ›

Python Datetime Examples 04 Jul 2016 python3 date datetime

Examples on how to create and manipulate date and datetime objects using Python. Read More ›

Packaging an Akka-http Application using SBT and Docker: Simple Example 29 Jun 2016 docker akka-http sbt

A simple example displaying common akka-http use cases and a couple of tips and hints to guide you through packaging a full app as a Docker container, so that you can deploy it anywhere you want. Read More ›

Migrating a Project from Spray to Akka-http: General Tips and Reference 29 May 2016 akka-http spray scala

Read More ›

Troublesshooting Akka: ClassNotFoundException: akka.event.slf4j.Slf4jLoggingFilter 19 May 2016 akka slf4j scala

Read More ›

Java.time API: Examples and Reference in Scala 01 Apr 2016 java 8 datetime scala

Read More ›

Working with Files & Directories in Python 26 Mar 2016 python

Examples on how to perform filesystem-related tasks using python. Tasks include reading and writing to files, listings files and manipulating the path. Read More ›

Python Environment Overview 25 Mar 2016 python

Short and concise descriptions of the major projects/tools related to managing python projects, dependencies and environments. Includes information on Anaconda, Pip, Virtualenv and related projects. Read More ›

Using the AWS CLI to manage Spark Clusters on EMR: Examples and Reference 23 Mar 2016 emr cli spark

Read More ›

Update Java to JDK 8 on Amazon Elastic MapReduce 22 Mar 2016 emr spark java 8

Read More ›

Python Regular Expressions: Examples & Reference 17 Mar 2016 python-3 regular-expressions

Read More ›

Linux most: Usage Examples for a Little Known Pager Tool 15 Mar 2016 linux

Read More ›

Comparing Interactive Solutions for Running Scala and Spark: Zeppelin, Spark-notebook and Jupyter-scala 07 Mar 2016 notebook interactive scala spark zeppelin

Read More ›

Zip/Unzip on Linux: Reference and Examples 05 Mar 2016 zip linux

Read More ›

Json4s Examples: Common Basic Operations using Jackson as Backend 29 Feb 2016 json scala json4s

Json4s is one of the best libraries for manipulating json in Scala. Here are some examples and common use cases you may have come across and some you perhaps haven't. Read More ›

Apache Zeppelin, Spark Streaming and Amazon Kinesis: Simple Guide and Examples 19 Feb 2016 emr spark zeppelin kinesis wip

Read More ›

Elasticsearch: Bulk Inserting Examples 16 Feb 2016 elasticsearch

Read More ›

Elasticsearch Mappings: Examples on how to Create, Edit, Remove 16 Feb 2016 elasticsearch

Read More ›

Fully Customized Json Validators for the Play 2 Framework: Explanation and Example 05 Jan 2016 play 2 json

Play 2 Allows a lot of flexibility for defining custom rules for JSON validation but if you want full control over how JSON is validated against a case class (including conditions that depend on more than one attribute and custom error messages) you need to write custom readers for your classes. Read More ›

Usability Best Practices for the Busy Programmer: Two Principles to Keep in Mind 30 Dec 2015 consistency usability robustness ux best practices ui

Here I'll explain **two things** you (in the very least) need to take into account in order to have systems you create be efficiently used by your end users. Read More ›

Java 8 Timezones: Examples in Scala 23 Dec 2015 java 8 datetime scala

Many Scala examples of the new Java 8 `java.time` API, focusing on ways to deal with and encode Timezone information in your objects. Read More ›

Pandas DataFrame by Example 15 Dec 2015 pandas python

Lots of examples of ways to use one of the most versatile data structures in the whole Python data analysis stack. Learn how to slice and dice, select and perform commonly used operations on DataFrames. Read More ›

One-Hot Encoding a Feature on a Pandas Dataframe: Examples 27 Nov 2015 pandas

One-hot encoding is a simple way to transform categorical features into vectors that are easy to deal with. Learn how to do this on a Pandas DataFrame. Read More ›

Spark DataFrame UDFs: Scala Examples 11 Nov 2015 spark udf scala

Read More ›

Add an Apache Zeppelin UI to your Spark cluster on AWS EMR 10 Nov 2015 aws emr spark zeppelin

Read More ›

Creating a Spark Cluster on AWS EMR: a Tutorial 10 Nov 2015 aws emr spark

AWS now provides full support for Spark Clusters within Elastic MapReduce (EMR). It's very simples and you just need a couple of minutes to learn how to do it. Read More ›

Window Resolution too Small When Running Ubuntu 14.04 on Virtualbox 31 Oct 2015 virtualbox ubuntu 14

Read More ›

Spark SQL: Examples on pyspark 18 Oct 2015 pyspark spark-sql

Read More ›

Logrotate on Ubuntu: Configuration Examples & Reference 07 Oct 2015 log wip

Read More ›

Basic Vector and Matrix Operations in Julia: Quick Reference and Examples 30 Sep 2015 julia

Read More ›

How to Setup a Proxy over an SSH Tunnel on Ubuntu: Examples with Firefox and Chrome 24 Sep 2015 ssh tunnel proxy ubuntu

Read More ›

Creating Scala Fat Jars for Spark on SBT with sbt-assembly Plugin 18 Sep 2015 sbt hadoop spark sbt-assembly

Read More ›