LLM Demos All Look the Same

Last week I had the pleasure of attending the Snowflake Summit in San Francisco. As you might imagine, AI played a huge part in the conference. During the product feature keynote speech, Snowflake told us the answer to every question is CoCo, or their form of AI that uses Large Language Models or LLMs.

AI is much more encompassing than just LLMs but that is what everyone thinks of when we talk about AI right now. The downside to LLMs is that all of the demos look exactly the same. Snowflake kept trying to show us new features in their product, would bring up a chat prompt and type in some sort of question. It would think about it for a bit and then spit out a text answer. Sure you didn't need to code or type SQL queries to the database but there really wasn't any difference from one demo to the next.

Last year I attended only 3 days of the conference as I had to get home and couldn't stay for the final day. I should have taken a page out of last year's playbook and only stuck around for 3 days instead of the 4 I did this year. By the last day, I lost all excitement and couldn't wait to get home. In fact, I left the conference at noon and tried to catch an earlier flight back to Salt Lake.

Ultimately I came away from the conference feeling like I wasted some of my time. When all you have to do is pull up a chat prompt and enter your question, there is no need to learn the intricacies of the product. Don't know the syntax to join data from two separate data sources, just let the AI figure it out for you. The conference can be reduced from days to hours.

When I left the conference, Snowflake asked me to fill out a survey. They provided dates for the event next year and asked if I would attend in San Francisco. Naturally I politely declined as it just doesn't interest me any more. Now if they held the conference in Hawaii, I'd be there but doubt I would spend any time at the actual Summit. When it came time to put together a trip report, I could always ask AI to do it for me. 

Artificial Intelligence Being Wrong

I recently visited Alaska to do some skiing and spent time away from my home ski areas Snowbird and Alta. First off, Alaska is a whole new kind of cold when it comes to skiing. I'm glad I opted to bring my warmest gear. 

While driving to the airport after a day of skiing at Aleyeska, I got a call from my ski buddy Jim. He said that AI told him Snowbird would be closing for midweek skiing at the beginning of April. Normally Snowbird stays open daily until it gets close to the end of the season and then it closes during the week and is only open on weekends. That normally doesn't happen until May. This is important as my buddy Jim will be coming to Utah for a week in April to do some skiing and that is kind of tough to do if the ski area is closed.

Yesterday I visited Snowbird and made sure to ask around if they announced any changes in their schedule. They have not. This begs the question how AI came up with a false schedule change. The answer is very important as it will help you understand why AI often provides the wrong answer to questions.

There are multiple ways that AI came up with an answer to when Snowbird would switch to weekend only skiing. The simplest answer is someone speculated on a date and the AI found such a posting and incorporated it into its knowledge base. This often happens in real life. Someone will state a guess as fact and we all believe the person. My dad once told me that magnets are created by hitting a piece of iron with a hammer really hard. While that may be one way to create a magnet, that is not how they are made. I recently corrected his misunderstanding. Doing the same thing with AI is a bit more involved.

Another way that AI can come up with false knowledge is through inference. We are having a particularly warm winter here in the western United States. When temperatures get above a certain point, snow starts melting quickly. This week we are expecting record highs and AI could recognize that when the average temperature gets above a certain point, Snowbird switches to weekend-only skiing. Unfortunately temperature is not how Snowbird determines when people are done skiing. They look at hotel reservations at their lodges as well as a number of other factors and will try to keep the daily schedule as long as Mother Nature allows.

The important thing to gather from this post is that Artificial Intelligence is often wrong. I don't know how many times I have asked Google a question only to have it spit back the wrong answer. While I have not kept close statistical records, I feel like I get the right answer from AI only about 20% of the time. It may point me in the right direction but I always verify the answer and augment what I am told with real human intelligence.  

How Did 2025 New Year's Resolutions Go?

It is almost time to change the calendar to 2026. I thought about posting something about New Year's Resolutions but realize most of them are broken in the first 30 days of the new year. So I thought about how I did with mine this year. As I think about it, I did pretty good but there is always room for improvement.

One of the most easily broken resolutions relates to weight loss. I find that there are smart goals and there are stupid goals in this area of my life. A stupid goal is that I want to shed a certain number of pounds. It is stupid because it doesn't include how to achieve the goal. A smart goal is setting a number of days I want to ski. Skiing is exercise but it is something I love to do. The more days I ski means I exercise more. This year I set the goal to ski over 100 days for the 2024/25 ski season. I ended the season skiing 126 days. Goal achieved!

Unfortunately I can't ski all year long. So what is a smart summer exercise goal? The past couple of years I have not done as many bike rides as I would like and so I set a goal to ride over 1000 miles for the year. I have had some years where I achieved over 6000 so my goal is very doable and stretched what I accomplished the previous year. I have one more day to go but have over 1200 miles ridden so far this year. Goal achieved!

Now while we all have that stupid goal to lose a couple of pounds, did my smart goals help contribute to what I really wanted to do? I can say, "Yes." I am at my healthiest when I weigh less than 160 pounds. The skiing and biking goals helped get me down to 156 but you don't lose weight in the gym, you lose it in the kitchen. I set some smart goals related to eating well and that is what did it for me. If you are trying to lose weight without GLP-1 drugs, the way to do it is by leaving every meal hungry. So hungry you cry. If you can do that, you will drop weight easily. Well I guess it is not so easy as you cry 3 or 4 times a day.

I also had some mental and professional goals and did fairly well with them. This coming year I have set a technology goal that actually applies to computers and technology. My recent post about a single 4K monitor not being enough has me wanting to build a new computer with two 4K monitors. There is more to think about than just purchasing the hardware. I will have to redesign my office to make the space for both 43-inch monitors. I will also need space for my work computer and its second or perhaps third monitor. That is a lot of screens to look at and so I am going to have to build up . . . and maybe down. I will be sure to update readers with how the project is going.

One goal I had for this past year is doing more with artificial intelligence (AI). Fortunately I still have a few months left in my company's fiscal year so I'm not quite out of time. I did get a work environment set up for myself and all I need to do now is a bit of tweaking to tune it for my sample use case. Ultimately I want to be able to type natural language and have it generate the complex SQL to pull the data from a sample database where information is spread across multiple tables. So instead of typing code, I want to type, "What is the most popular movie over the past year and did it effect any video game sales?" The first question is a pretty simple query to a single table. The second is significantly more complicated. I'm close to being able to ask my question but have a little more Symantec modeling to do, which is really just accurately describing the data in natural language terms. I am close though.

As you start the new year, don't create New Year's resolutions without thinking about how you did last year. If you kept some of them but not others, think what you can do to set smart goals and not just boring number ones. Get creative and figure you what you can do to become excited to accomplish what you want. Right now, I'm thinking about going to various Disney parks 4 times in 2026. If you can avoid eating all the food, you really do a lot of walking and can surpass 25,000 steps in a single day. That ought to help with anyone's diet.

Using AI Appropriately

I had the pleasure of helping develop the PlayStation 5 video-game console when I worked in Research and Development at Sony Interactive Entertainment. One of the advantages of the hardware is that the file system is fast enough so it is not necessary to keep multiple copies of digital assets. What does that mean? When you create a virtual world, it is made up of a number of digital assets such as trees, rocks, buildings, or any number of other things we find in the real world. Most of the time, those assets are used over and over again. For the sake of an example, a forest is made up of a lot of trees. There may be only 5 different tree models created for a game and then they are reused many times to give the illusion of a forest. With the PS4, game designers would copy those 5 trees hundreds of times. That isn't necessary on a PS5 as you can have just the 5 tree types and point back to each one on the file system any time you need it. Assuming each tree requires 4KB of memory (an arbitrary value pulled out of thin air) and you have 1000 of each tree in a game, the PS4 would require 4MB for each tree used while the PS5 only needs 4KB. Theoretically PS5 games should be significantly smaller than for other game consoles. This becomes very helpful for games like Red Dead Redemption 2 that required 2 Blu-ray disks for the game. For physical game disks, it saves a bit of money and for digital downloads, they don't take as long to put on your system.

So what does that have to do with using AI appropriately? It boils down to why game developers only make 5 tree models. Most gamers don't slow down and look at every tree in a forest to see if it is different than all the others. So why take the time to create more than 5? If you can throw the problem at a generative AI program, you can let it create hundreds of different trees. This provides a level of uniqueness currently missing in a lot of games. This totally eliminates the benefit of the PS5 over other gaming consoles as its games once again become bloated. The reality is that while the PS5 doesn't require duplication of digital assets doesn't mean game studios are using the feature. You will still find copies of digital assets spread throughout quite a few games. Why not make the games more unique?

I have been thinking about this issue for the past couple of days and came up with another area where the use of AI should be applied. I walked through my kitchen this morning to get myself some breakfast. My floor is a high-quality laminate with a simulated pine surface. While pine is great looking, it is a very soft wood and makes horrible flooring that is easily scratched and dented. Using a high-quality laminate allows it to be almost indestructible yet look beautiful. The downside to a laminate is that there are only about 5 patterns on the boards. They repeat quite often and if you look closely you notice a lot of the boards are the same. The laminate floor company could use AI to create 100 different patterns instead of just 5. This would create a much more unique floor.

Yes these two solutions for generative AI don't seem to be high priority. After all if they were, companies would spend the effort to make their products more unique. The beauty of using AI is that products can become more unique and special without humans having to spend more time making it happen. You will still need those humans to fine-tune what is generated but their time can be spent being creative and not doing repetitive and mundane tasks.

 

Phishing, Smishing, and Vishing

Every year Sony makes me take mandatory security training and I recently went through the exercise again. Whenever there is annual training, it is best to try and figure out what is new or has been changed. This year I learned two new terms: Smishing and Vishing.

I already know what Phishing is. Anyone who has had an e-mail account knows that scammers will try to send you an e-mail that plays on your emotions to get important personal information. Most of the time the e-mails are easy to spot and I quickly delete them.

The younger generations have mostly ignored e-mail and prefer phones. They fit in your pocket and so texting has become second nature to them. I used to hate texting as I saw it screw up a lot of meetings and slow things down. Now I don't mind it as it is an efficient way to communicate. This is what scammers use for Smishing, your mobile phone. It is the same thing as a Phishing attack but comes over your phone instead of through e-mail. Someone asking for the recently received code your bank texted to you would be an example of Smishing. The term is a mix of Phishing and SMS messaging. As with a Phishing attack, you should never share personal information via text messaging.

So what is Vishing? That is when a scammer calls you or uses voicemail to request sensitive information like passwords or bank information. With current artificial intelligence technology, scammers could call using your spouse or child's voice asking for a password. Therefore it is important to verify phone numbers whenever anyone asks for sensitive information. Even better would be to call the person back as phone numbers can be spoofed as well. Perhaps this is why there is a new word for something I have always considered Phishing until now.

There are relatively few scammers in the world but it feels like they are everywhere. This means we all need to be vigilant about not sharing secret information that would allow them to get into our bank accounts or steal our identity. Vocabulary for Phishing, Smishing, and Vishing just means that they are using every tool possible.

A New AI Toy: ChatGPT

Several years ago I worked on a chat engine and learned a lot in the process. At the time we had a long way to go to create a program that did a great job mimicking human conversation. OpenAI is a company that released ChatGPT in November of last year and it does a pretty good job of realizing that goal. Today I decided to sit down and play with it for a bit and see how good it is.

One of my colleagues played with it as well and fed it a multiplication problem with 2 4-digit numbers. It came close to providing the right answer but a calculator revealed it didn't get the problem correct. Unless you pulled out a calculator though, you wouldn't know because it stated the answer with such certainty.

Armed with the information that it was good at answering questions but it wasn't that great at Math, I asked a different kind of question. I thought back to my college English courses and asked why experts consider "Moby Dick" to be such an example of classic American literature. I suffered through the book in college and hated it. Then I decided to pick it up after learning to sail in the hopes of understanding it better. While I understood the vocabulary, I actually prefer Herman Melville's "Typee" over "Moby Dick." I think Mr. Melville did as well.

ChatGPT came back with an amazing answer that would have greatly improved my grade in College. The answer had 5 parts discussing the detail the author included in the story indicating that Herman Melville actually spent time on a whaling ship. It talked about the struggle between good and evil as well as the futility of seeking revenge. Having read the book twice and really understood it once, I had to agree with the answer.

I don't think ChatGPT is going to take over the world. It would need to have a better understanding of mathematics and engineering to do that. It will give college professors something to worry about though. If I was back at the University, I would immediately use ChatGPT to help me start any papers I need to write. So is that plagiarism? Probably but if done correctly, it would be impossible for anyone to tell. I'm glad I am not a professor. About they only thing they can do is move from written papers and exams to oral ones.

New Year's Resolutions

This morning I headed into work on my bike and then to the company gym where I usually start my day. Keeping with my early-rising habit I developed over the Christmas break I found myself there earlier than usual. I expected the gym to be packed with those seeking to lose a bit of weight gained over the Holidays. Nope, it felt really empty. Perhaps the idea of making and then breaking resolutions has gotten to the point where the new trend is to not make them in the first place. I hope I am wrong about that as the new year is always a great time to reflect on how we can improve ourselves.

On Sunday my wife asked if I had set any goals or resolutions and I had to confess that I have not made the effort yet. I sort of have a few ideas that carry forward from previous years like skiing more and dropping a bit of weight. So far I have started the year off right as I am down a few pounds which is a freaking miracle considering all of the feasting most people do during the holidays. I have also come back to work with a sore body from skiing so much over the break.

In this vein of looking at ways of improving myself over the next 12 months, I think I would like to learn a new technical skill or two. I have a very strong understanding of relational database systems and have even dabbled with other database technologies. Therefore those are not good areas to expand. Instead I should look at where my weaknesses are and set some goals to improve those parts of my professional skill set. I have used both Amazon's (AWS) and Google's (GCP) cloud platforms but could definitely improve there. Our company has a learning portal and should probably leverage it first and then fill in any shortcomings with a local conference.

Another area I would like to focus some effort is improving my artificial intelligence (AI) knowledge. Last year I ran through a Reinforcement Learning book and learned a lot. The book wasn't all that great but I did enjoy the code segments and doing the examples. There are a number of new AI tools emerging and I will try to find one that catches my interest and do something with it.

Finally I want to do more with gaming technology. I have dabbled with the Unity game engine but think I could learn a lot by learning Unreal as well. There are so many samples and tutorials that it would be good to spend an extra hour on Tuesday and Wednesday evenings hanging out at the office and building something interesting.

While it is nice to create a list of improvements, be sure to write them down. Someone once said that a goal not written is just a wish. If you want the wish to come true, start by writing it down and then coming up with a plan. I have the starting part down, now I should follow through and make a proper plan for each of these resolutions. Good luck on your own New Year's Resolutions.

AI: The Next Big Tech Trend

About 30 years ago everyone thought that artificial intelligence (AI) would be the next big thing. Companies invested in expert systems only to discover they didn't have much power and we had a long ways to go before we could do anything really spectacular. Now it seems like you can't click through a website without hearing about AI, machine learning (ML), or deep learning (DL). This leads one to ask if AI really is the next big thing.

This morning I came across 2 articles on AI that I felt merited reading. The first talked about a new form of machine learning where the program or agent is given the freedom to explore new ideas randomly instead of being rewarded for achieving some goal. Should a goal be reached, the agent stores the steps necessary to repeat the process. If a more efficient way of achieving that goal is found, the memory is overwritten with the better steps. The reason this is of interest to me is that is has been used to play several computer games successfully when other AI agents based on reinforcement learning have failed. Interestingly this is how some of our most successful innovations have emerged: thinking about novel ways of solving an existing problem.

The second article I read today came from a consulting organization talking about how AI is going to revolutionize our economy. While they admit that a number of jobs will be replaced by AI, they also point out that a higher number of jobs are being created to implement it than those being lost. The article then went on to estimate the amount of money being spent developing and using AI. It is not a small number.

My takeaway from both articles is that the field of artificial intelligence is getting a lot of interest and is showing the same promise that the personal computer industry had 30 years ago. There are also a lot of new ideas in AI and it will be an area of tremendous innovation. If I had a son or daughter headed off to college and wondering where the money is, I would definitely steer him or her towards AI.

A Morality Setting for Artificial Intelligence

Many of us interact with artificial intelligence (AI) on a daily basis without realizing it. If you look at the recommendations provided by Netflix or Amazon, then you are being served results from some sort of AI. Most of the time, those recommendations are based on previous purchases or actions. If you watched one of the Marvel superhero movies, you might get recommended another one that you may not have seen.

Now imagine that you want to change your behavior. Perhaps you have decided that smoking is bad for your health and want to quit or that you want to stop watching violent movies. At first you may have a firm resolve to not do those things any more, then you get an e-mail telling you about a sale on violent movies or smoking products. While you may not immediately give in, you may find it difficult to resist the temptation to give in and go back to your old ways.

So how do you tell online retailers and service providers that you would like to change your behavior and to please stop sending you recommendations for products you are no longer interested in purchasing? Right now, you can't without doing something drastic like changing your phone number, e-mail address, and creating all new accounts.

This also leads to the question of if service providers should build in some sort of morality. When you go to watch the latest superhero movie, how would you feel if you received the message, "Violent crimes are up, perhaps you should watch a romantic comedy." Or you go to purchase cigarettes online (I don't even know if that is possible) and see, "Smoking is bad for you, how about some nicotine gum instead?" My instinct says nobody would be happy even though we would all be better off with such suggestions.

Machine Learning for Predictions

This past week I have been working on trying to predict the future based on historical data with the help of machine learning. This is one of those areas where you have to be careful because if it was easy then everyone would be doing it to cash in on the stock market or for bets on who will win the next sporting event. However there are certain variables that can be used to increase the probability of an event happening. For instance, if you have clouds in the sky, there is more likely a chance of rain than on a sunny day. The trick is figuring out what those variables are and this is what is know as feature selection.

If you think about your favorite sporting event, think about all of the variables that go into a game. Who are the players and what are their various statistics? Does the weather impact those statistics or are the games played indoors? You can eventually see that complexity spirals out of control if you are trying to predict the winner of a game.

The problem I have been trying to solve is something akin to when someone will go out to dinner next based on historical information. An overly simplistic algorithm might look at the average time between restaurant visits. For someone that dines out on a regular basis, this will work. Someone else might only go out on special occasions and that won't work unless those special occasions are evenly distributed on the calendar. Think about variables that contribute to when you eat out. Some might include:

• Day of the week - Perhaps you eat out only on Friday or Saturday evenings and never on Sunday.
• Holidays - It is always more difficult to get a restaurant reservation on a holiday.
• Proximity to Payday - While it is tough to generalize when everyone's payday is, if you have that information, it might be a factor in some people's choice to go out.
• Weather - A snowstorm has the potential to greatly reduce people's desire to go out.

With a bit more time and thought, I'm sure you could come up with a number other factors that could be used to predict if you are eating out or at home. In machine learning terms, these variables are called features and can be used to accurately predict a person's behavior.

When I look at my own restaurant habits, I know that I generally eat at the airport on Thursday evenings as I catch a flight from San Francisco to Salt Lake. Unfortunately that won't be the case this evening as I decided to buy a big lunch and only eat half of it, saving the other half for this evening. Therefore it is important to remember that predictions are no guarantee that an event will happen.

A Scary Thought for Halloween

This morning I answered a one-question survey about what is the most scary Halloween character. A number of the usual suspects could be chosen. As a child, a monster would have definitely have made the list. Now that I am an adult, the most scary things in my life are much different. I am scared of things like getting extra random screening at the security in the airport or having the IRS decide I didn't pay my fair share of taxes. I am also afraid of illnesses like cancer or Alzheimer's disease.

If I stop to think about monsters, the scariest one on the earth today is man. Think about the recent mass shooting in Las Vegas and it has totally stolen the spotlight from natural disasters like hurricanes Irma and Maria. This has me rethinking several of the projects I am researching at work. I am helping to build some really fun technology, but in the wrong hands, it could be used to do much more scary things than any mass shooting to date.

Artificial Intelligence is one of the subjects I am researching and still think it is pretty stupid. However there are some areas where great progress is being made. Pull out any of the latest smartphones and try to take a picture of someone. The first thing you notice is that someone's face gets highlighted so you can provide a name or tag to be used by Facebook or some other social media site. This means that a tiny device that fits in your pocket knows the difference between a person and a random object. Now couple that with acoustic analysis where a couple of speakers can tell the direction that person's voice is coming from. A number of toy-robot researchers are using this technology to create robots that look at who is talking in a group of people. All of that seems pretty harmless and fun, right?

Now let's take that same technology that can recognize faces and knows how to aim at a person. I don't want to elaborate any more as I don't want to give anyone any ideas. However my realization this morning left me with a sick feeling in my stomach and the thought that while it isn't quite SkyNet from the Terminator movies, it is moving in that direction. Perhaps I shouldn't be so quick to dismiss the scary uses of Artificial Intelligence. Monsters are not the things scaring me this Halloween season.

Demystifying Artificial Intelligence

One of the nice things about my job is that I get to play with some of the latest technology. Right now I am doing a lot with artificial intelligence (AI) and machine learning (ML). I remember back in the 1980's and how AI had the potential to be something great. Then it stagnated for 2 decades. Due to the low cost of computer processing power and memory, AI is making a comeback. Hopefully it has a bit more staying power this time around.

As I research new technologies it is important not to buy into the hype. I read one article today claiming that those that don't make the effort understand AI and ML will begin to look at computers as magical machines that they understand less and less. That can be true of any technology and so if you find yourself thinking of a new product as magic, that is what the marketing people want you to believe.

The more I work with the current state of artificial intelligence, the more I realize it is just really good at guessing the right answer. Most AI engines get fed a bunch of numbers (characters and words can be represented by numbers) and what those numbers mean. The engines then try to figure out what a different set of numbers mean. It then becomes very important to train the AI engine with a lot of data as well as the correct data.

What happens if you train your engine with the favorite breakfast cereals of children between the ages of 5 and 7? When you ask it what an adult would like for breakfast, you will most likely get the wrong answer. While I do like Fruit Loops, I don't eat them for breakfast anymore. What happens when you recognize the age mistake but only train the engine with data from Americans? If you ask what someone from Japan eats for breakfast, once again, you will get the wrong answer.

Currently artificial intelligence is at an interesting point. We are creating useful applications for it that can greatly help us in our daily lives. It is also important to remember that it isn't magic and is only as smart as the data with which it has been trained.

Machine Learning and Lots of Data

At the beginning of the month I wrote about artificial intelligence and how it is not going to take over the world any time soon. I have continued to play with it and have been working on a branch called supervised learning. My basic example or use case is to feed my simple program a bunch of training sentences that are categorized as questions or statements. Then I have a test group of sentences to see how well my program has learned.

I started with a very small set of training sentences thinking that a person would be able to distinguish between a question and a statement fairly easily using just these examples. My training set began with only 30 sentences and my test set had 20. After training, my program correctly identified 16 of the test set. That sounds pretty good at 80% but I really need to get closer to 100%. So I added more training data. I found a list of 800 random questions and added them as well as several pages of text from two popular books I found online: Uncle Tom's Cabin and The Old Man and the Sea. That brought me closer with 18 sentences correctly identified as statements or questions.

The statement, "I like to ski," was wrongly classified as a question while "Who is your favorite actor?" got classified as a statement. So I added more training data until I got 100% correct classification. My original training data started with 30 sentences and is now close to 1000. That seems like a lot of extra work.

Now it is time to tune my algorithm. There are some things I can do to get better results with less data. However if you plan to embark on your own supervised learning project, be prepared to collect a lot of training data. You will need it.