I need more realism in my data tutorials

I have been wrangling data in one form or another for the better part of 15 years. During this time I have watched many tutorial videos, listened to podcasts, audiobooks and read blog articles in order to gain a better understanding of various topics.

There is a major flaw I feel in a lot of these tutorials ( free or paid ) is that they’re far too idealistic. You’re uploading a CSV file into a system which has excellent data quality and is no problem for the system to understand. Very little work ( if any ) needs to be done to it. I would love this to be the general rule, reality tends to disagree.

Roll with the punches

I have lost count of the number of times where I have connected to what I have been assured is a clean data set into some report system only to find that there are issues abound. These problems are so common in the industry yet I rarely see anyone talk about them in these tutorial series that are made up for the likes of say Tableau, Power BI, Databricks etc.

I have a feeling that a lot of data tutorials are sanitized in this way so that they can be straightforward, easy to follow and, above all, quick. I think this detracts from the longer term usefulness of these tutorials. One YouTuber that I love to watch is Adam Savage because he shows his work. He shows the struggle that he can go through some times to achieve something.

Problems can teach you a lot

Seeing how someone responds when something goes wrong, how they investigate that issue and then deal with it is so incredibly valuable to someone who is starting to learn the craft. Not only does it humanize those we look up to but it also sets the learner up for a reality they are likely to encounter.

I have often heard careers such as Software Engineering or Data Engineering to be very much a problem solving profession. Rarely do I see problems being solved.

One thing I think I’m going to start doing on this side is explaining how I have gotten around a number of issues when doing my job as a Data Engineer. I hope that this will help some people understand how I’ve gotten around this issue so that they can learn from me and hopefully not feel alone.