PicoBlog

As I write this, I am currently in Dubai and about to travel back to the USA. To everyone I met in Bangalore, India, and Dubai, UAE, over the last two weeks - you’re awesome! I’m also bummed to miss dbt’s Coalesce this week, but I was scheduled to speak at Gitex Dubai almost a year in advance.

Thanks,

Joe Reis

P.S. Want me to speak at your event in the first half of 2024? Please reach out to me by 11/6/2023. No matter who you are, I won’t accept talk or event inquiries after this date.

2023 is the year of AI, and if you’re visiting technology or data conferences, you’d be forgiven for thinking that every vendor on the planet is now an AI company. Some of these are hardcore ML/AI companies and deserve much respect for their hard work and technical achievements. The others? I swear they were web3/blockchain or unrelated to AI in 2022. But then was then, and this is now. Now, everyone is doing “AI.”

I gave several talks and spoke on even more panels this week at Gitex Dubai (a massive conference with over 170K+ attendees), mostly about how we need to approach AI deployment with a level head and ignore the noise. My no-nonsense, blunt, and cantankerous attitude spilled over into my talks. A journalist picked up one of my talks; you can read it here. Perhaps I was annoyed by the sheer amount of unbelievable things I’d hear pundits say on stage - “You can plug in AI anywhere, and it will just work,”… “Artificial general intelligence will be here in 2030,”… “AI is easy with our solution.” WTF is this, fantasy land?

It’s not my style to call out specific companies, as I stick to Warren Buffett’s approach of “praise by name, criticize by category.” The category I’m criticizing is the AI-washing companies, saying they’re doing AI while barely or not doing it in reality. I’ve seen these types of companies before in previous ML/AI hype cycles, and they’re very sneaky. It will be hard to distinguish between a legitimate and bullshit company. So it goes when there’s a lot of money to be made in a hype cycle (looking at you, FTX).

The situation reminds me of the carnival barkers from the 1800s, who would travel from town to town selling magical potions and cure-alls to unsuspecting dupes. Most “AI companies” I’ve seen are the modern-day equivalent - opportunistic carnival barkers hawking AI solutions to organizations who often don’t understand what they’re buying.

Buyer beware. Your investment into AI will rarely leave the POC stage, if it even gets there. To succeed with AI, take a sober view of AI and how you can incorporate it into your organization. Not every company needs a LLM. Understand the difference between generative AI (LLMs and similar) and “classical ML” (the stuff that we’ve been using for ages until generative AI took over). Most importantly, ensure your data is in good shape and you’re already doing something useful with data. AI is an accelerator of good existing data hygiene and practices, not a replacement for it.

Listen to the audio clip above on this topic, which is also my 5-Minute Friday on Spotify.

Here are some cool things I read this week. Enjoy!

State of AI Report 2023 (Air Street Capital)

Excellent analysis of the state of AI (particularly LLMs) this year. Worth a read.

"Building Machines That Learn and Think Like People", 7 years later (Gonzo ML)

We’re entering an Uncanny Valley era, and I feel like GPT-4V is just the beginning.

The Grand Rewrite of DataHub (Metaphor)

I’m a fan of the crew at Metaphor. Rewrites are never easy, and this article discusses the good, bad, and the ugly of the DataHub project. It's a good read if you’re working on an open-source project spun out of a big tech company.

Manage Your Capacity, Not Your Time (The Engineering Manager)

I really like this article, because it applies supply chain ideas - capacity planning and safety buffer - to how one should plan their day. The similarities are very real. I try to knock out 1-2 important things daily, and if I get anything else done, awesome.

Estimates are about time, so let's cut to the chase (Nicole Tietz)

“When you get an estimate from an electrician 1 , you would be frustrated to get back a number of points. You typically want to know two things: how long will it take, and how much will it cost? These two are related but distinct: if it takes a week to replace your panel, that's too long to be without power. And if it costs $20,000 to change an outlet, that's too high and you'll look elsewhere. If they give the estimate in points, that may be meaningful to them, but not to you.”

I know the arguments for and against time-based estimation versus story points. Both have their faults and merits. This article raises some solid arguments for time-based estimation (like the example above). Ultimately, do what works with your team, but please ensure you’re consistent with expectations.

We’ll resume the show on 10/23 with our favorite nerd, Kevin Hu, from Metaplane.

Data Warehouses and Semantics Deep Dive, SDF, and more w/ Lukas Schulte - (Spotify, YouTube)

Improving Your Health and Wellness - Techie Edition w/ Colleen Fotsch (Spotify, YouTube)

Data Engineering AMA w/ Matt Housley & Joe Reis (Spotify, Youtube)

5 Minute Friday - AI Carnival Barkers (Spotify)

Johnny Graettinger - A Deep Dive on Streaming and Immutable Logs (Spotify)

5 Minute Friday - Building a Global Data Engineering Community (Spotify)

Bob Muglia - Moving Beyond SQL, Knowledge Graphs, and More (Spotify)

5 Minute Friday - Skills to Pay the Bills (Spotify)

Egor Gryaznov - The "Non-Modern Data Stack", and Getting Out of Our Data Bubble (Spotify)

Canada, 11/2-11/3 - DAMA Toronto - register here

Finland, 11/9 - Agile Data Engine Summit - register here

San Jose, CA, 11/16 - TBA

Las Vegas - Gable’s Party during ReInvent, 11/28

Data Day Texas (Austin) - register here

Data Modeling Zone (Arizona) - register here

Skiers in Data (Switzerland) - TBA

Spain - TBA

South Africa - TBA

Dubai - TBA

Australia - TBA

Friendly reminder - Do you want me to speak at your event in the first half of 2024? Get your inquiries to me by 11/6/2023. I won’t be accepting talk or event inquiries after this date. Thanks.

Thanks for supporting my content. If you aren’t a subscriber, please consider subscribing to this Substack.

You can also find me here:

Monday Morning Data Chat (YouTube / Spotify and wherever you get your podcasts). Matt Housely and I interview the top people in the field. Live and unscripted. Zero shilling tolerated.

The Joe Reis Show (Spotify and wherever you get your podcasts). My other show. I interview guests, and it’s totally unscripted with no shilling.

Fundamentals of Data Engineering (Amazon, O’Reilly, and wherever you get your books)

Be sure to leave a nice review if you like the content.

Thanks! - Joe Reis

ncG1vNJzZmiin5q%2FprXSZ6qumqOprqS3jZympmegZK6qecKaqaehppa5bq7Aq6KeqqNif3F%2BkmacnaGknryv