Welcome to Incremental Social! Learn more about this project here!
Check out lemmyverse to find more communities to join from here!

jas0n ,

I used to write tons of automation in my previous data role. While time saved matters, the other important takeaway is reproducibility. Other people on the team were writing giant SQL scripts and highlight running each one and then manual checking to see if it worked... I'm talking about tables anywhere from 1-100 millions records. You aren't checking shit by skimming a top 1000. And what a ridiculously error prone process that is. Take the human out of that equation!

If the data came out wrong, it would be because the data came in different/corrupted, not because I missed a query. Speaking of different causing problems.. one time a company sent us data that was fixed width by character instead of fixed width by byte. Smh...

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • technology@lemmy.world
  • incremental_games
  • meta
  • All magazines