queirozf.com

spark scala

Spark dataframe Examples: Reading and Writing Dataframes

23 Feb 2020   Some examples on how to read and write spark dataframes from sources such as S3 and databricks file systems.

Read More ›

paper-summary machine-learning-engineering software-engineering

Paper Summary: Software Engineering for Machine Learning: A Case Study

25 Jan 2020   Summary of the 2019 article "Software Engineering for Machine Learning: A Case Study" by Amershi et al.

Read More ›

git

Git branching: Reference and Examples

23 Jan 2020   Common use cases and examples for using branches on git, locally and on remotes.

Read More ›

paper-summary attention sequence-learning machine-translation

Paper Summary: Neural Machine Translation by Jointly Learning to Align and Translate

11 Jan 2020   Summary of the 2014 article "Neural Machine Translation by Jointly Learning to Align and Translate" by Bahdanau et al.

Read More ›

paper-summary machine-learning-engineering

Paper Summary: Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift

23 Dec 2019   Summary of the 2019 article "Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift" by Rabanser et al.

Read More ›

python testing

Python Unittest Examples: Mocking and Patching

09 Dec 2019   Simple examples to help you understand when/where to use mocking and patching, so you don't need to skip testing any part of your code.

Read More ›

pandas

Pandas Dataframe Examples: Duplicated Data

17 Nov 2019   Deal with duplicated data in pandas: drop, count, show and mark duplicates in pandas dataframes.

Read More ›

paper-summary neural-networks sequence-learning

Paper Summary: Long Short-Term Memory

16 Nov 2019   Summary of the 1997 article "Long Short-Term Memory" by Hochreiter and Schmidhuber.

Read More ›

spark

Spark SQL Case/When Examples

09 Nov 2019   Case/when clauses are useful to mimic if/else behaviour in SQL and also spark, via when/otherwise clauses.

Read More ›

spark-sql scala

Spark SQL Date/Datetime Function Examples

09 Nov 2019   Examples on how to use date and datetime functions for commonly used transformations in spark sql dataframes.

Read More ›