LLM Demos All Look the Same

Last week I had the pleasure of attending the Snowflake Summit in San Francisco. As you might imagine, AI played a huge part in the conference. During the product feature keynote speech, Snowflake told us the answer to every question is CoCo, or their form of AI that uses Large Language Models or LLMs.

AI is much more encompassing than just LLMs but that is what everyone thinks of when we talk about AI right now. The downside to LLMs is that all of the demos look exactly the same. Snowflake kept trying to show us new features in their product, would bring up a chat prompt and type in some sort of question. It would think about it for a bit and then spit out a text answer. Sure you didn't need to code or type SQL queries to the database but there really wasn't any difference from one demo to the next.

Last year I attended only 3 days of the conference as I had to get home and couldn't stay for the final day. I should have taken a page out of last year's playbook and only stuck around for 3 days instead of the 4 I did this year. By the last day, I lost all excitement and couldn't wait to get home. In fact, I left the conference at noon and tried to catch an earlier flight back to Salt Lake.

Ultimately I came away from the conference feeling like I wasted some of my time. When all you have to do is pull up a chat prompt and enter your question, there is no need to learn the intricacies of the product. Don't know the syntax to join data from two separate data sources, just let the AI figure it out for you. The conference can be reduced from days to hours.

When I left the conference, Snowflake asked me to fill out a survey. They provided dates for the event next year and asked if I would attend in San Francisco. Naturally I politely declined as it just doesn't interest me any more. Now if they held the conference in Hawaii, I'd be there but doubt I would spend any time at the actual Summit. When it came time to put together a trip report, I could always ask AI to do it for me. 

Playing with Large Language Models (LLMs)

Yesterday I participated in a hands-on lab to integrate large-language-model (LLM) technology to help generate complex SQL queries to look into our data. The lab only lasted an hour and I learned a lot during that time. It also highlighted one of the use cases for LLMs for more than just a toy to help write high-school papers.

When ChatGPT released, it garnered a lot of attention. While everyone seemed impressed, I didn't see a lot of people asking how the technology can be used for good. Automatic code generations seem to be one of those good uses that can really help us in our daily lives. Rather than spending countless hours writing thousands of lines of code to do something important, you can specify what you want and have created and the LLM will generate the code for you. That is what yesterday's lab explored.

For those that don't know SQL, it is a very simple and almost English-like language used to query databases. If you have a table that contains information about customers you can use something like this to query the database:

SELECT firsname, lastname, first_purchase_date FROM customers;

This will list out all customers' names and date when they made their first purchases. It is pretty simple, right? Well it gets complicated once you try to filter that data or include info from other tables in the database. While most of my queries are 25 lines or less, it is not uncommon for me to write 500-line ones. This is where an LLM can really help streamline my work.

In order to keep the lab under an hour, all the data and most of the code was created so I just needed to put it in the right place. I loaded the data into an online database server and downloaded the code to my laptop. The only modification I needed to do to the code was update the connection string to point to my online database. Then we ran through the code see how it worked.

The code included a file to describe the 3 tables in the lab and that turned out to be the secret sauce. I closely looked through the description file and realized that it would take about an hour per table to duplicate this file for the database I work with on a daily basis. When there are only 3 tables, this isn't a large effort. Unfortunately my database has thousands of tables and it would be a huge undertaking to use this technology in my environment.

I had hoped that the LLM would be able to look at my database tables and infer what information they contained. This is possible but unfortunately is not nearly as accurate as having a data expert provide this detail first.

I'm glad I attended the lab and I learned a lot. It also demystified what is going on with this particular LLM. While I hoped I could use the output of the lab to make it easier for non-data scientists to query the database, I did come away with a strong understanding of the underlying technology.

90 Minutes Saved Me A Week In Las Vegas

This morning I started work a little earlier than usual as I attended a summary of Amazon's recent announcements from the AWS re:Invent conference held recently in Las Vegas. When I first started my career, I loved traveling and going to conferences in other cities. Then I spent a lot of time in planes to the point I now prefer to stay home, if possible. So any time I can have the host company of a conference summarize all of the important points in a 90-minute video conference, I am extremely happy.

AWS made quite a few announcements and they distilled them down into 3 different topics for the call this morning. I won't run through everything I learned as there are 2 announcements that really stood out. The first topic dealt with artificial intelligence. About a year ago, ChatGPT started getting a lot of attention. Naturally AWS is incorporating this technology in their product offerings. They also addressed how they are keeping the data used to train their large-language model (LLM) called Q, separate from your company's proprietary data. That is very important as you don't want someone outside your company asking Q for sensitive information that he/she shouldn't be able to access.

The second announcement I found interesting is using Q to generate database queries. I learned structured-query language (SQL) back in 1986 and thought it to be very natural-language like. As databases have grown in size and contain data in a variety of different tables, those queries have become less and less like natural language. You can now ask Q questions about your own data and as long as it has access to it, will show what you are looking for without knowing how to write the correct SQL. I will be interested to see how Q works in actual practice. I don't worry about it taking my job creating complex queries. Instead I see myself being able to do much more with its help.

My call this morning condensed a week-long conference into 90 minutes. I then pulled out 2 announcements that are important to me. Now I can go back to our AWS sales representative and get the in-depth information I need. To me, that is a step forward towards greater efficiency. For those that don't get to travel often, you may prefer the week in Las Vegas.