Introduction

Overview

This ETL guide builds on the earlier guide written: ‘Starting with ETL’, and goes through a series of new and different techniques, for building on and further developing your ETL skills.

Guide summary

An outline of the main areas covered in this guide is provided below:

01
Combining Mismatched Tables

Use Conversion Tables to help overcome data inconsistencies

02
Preserving Context

Learn how to integrate into a dataset contextual information that maybe stored somewhere else

03
Transforming Data into tables

Understand the most performant way of laying out tables for later analysis

04
Generalised Unpivoting

Learn how to unpivot tables with a N x M hierarchical structure

05
Generalised Function

Create a generalised and re-usable custom function for unpivoting any M x N data table

06
Avoiding commont dataset pitfalls

Learn some tips and tricks for overcoming commonly encountered problems with ETL processes on datasets

07
The Challenge!

A short exercise for how to combine files that are in different layouts and formats

08
Data Cop!

A short exercise for how to uncover changes to a suspected compromised dataset