Instructions Look Easier Than The Actual Experience

I have been watching a lot of cowboy movies and reading a lot of stories about the Wild West. Yesterday I got the idea that it might be fun to get a lever-action rifle to hang on the wall. We constantly hear how easy it is to purchase guns in the United States and so I thought it would be an easy process. It turns out that purchasing a rifle is a lot more complex than I thought. It leads me to believe that people pushing for stricter gun laws have no idea how tough the process actually is. Yes, there are shortcuts, the most glaring one being purchasing one from a private seller.

I am a huge fan of the Second Amendment here in the United States but have never owned a gun. I have often joked about how easy it is to get one in Utah and that we sell them in convenient stores here. That can't be farther from the truth. I read through the process and quickly wondered why people are pushing for more and more gun control. This reminded me of what happened the first I tried a few machine learning experiments.

When I first started playing with machine learning I thought it would be as simple as finding the right algorithm, putting in some training data, and then pulling out some magnificent results. Then I tried my first exercise and realized that the bulk of the work turned out to be massaging the training data into the right format. Then I cycled through a number of different algorithms to figure out which one provided the closest results only to discover that some don't produce consistent values. Finally I had to figure out how to take a trained model and turn it into a production system.

Ultimately I have found that it is not enough to read about doing something, especially technical tasks. In order to truly understand the task at hand, you have to run through the process. It is not enough to read the instructions, you have to pull out your tools and run through the steps yourself.

Quantum Computing

A few days ago I had an old friend reach out to me on LinkedIn. He asked if I knew anything about Quantum Computing among other topics and I had to confess that I didn't really know much. Today I came across it again as it relates to machine learning and so I did a bit more research.

There is a different vocabulary associated with Quantum Computing and that takes a bit of getting used to. With traditional computing you have a bit which represents either the number 0 or the number 1. Quantum computing uses qubits which can be either a 0 or a 1 at the same time. Now string a bunch of bits together and you can represent things like numbers or letters. If you have two numbers you want to add together, you can add their bit representations and you have a single number as the result. With qubits, you add two representations of numbers and have all possible sums for all possible inputs with a single operation. So if you have an 8-bit number, you can count from 0 to 255. Assuming you have space for the results, adding all possible 8-bit numbers results in 512 operations. With Quantum Computing you can get all 512 results in a single operation.

Now start thinking about really large numbers. The larger the numbers, the more operations it takes to come up with all possible outcomes. This idea starts to open up a number of very difficult problems that can now be solved.

If you can do addition on a computer, you can also do multiplication. If you can do that multiplication very quickly then you have something that will help with encryption and decryption of confidential information. Some worry that Quantum Computers can be used to hack into secure banking systems. Fortunately there are secure encryption and decryption techniques using Quantum Computing that are unbreakable and so the moment the technology becomes viable, we will just need to change our current security protocols.

Some scientists believe that Quantum Computing is still a ways off. Others see it as being much closer. The real question is what will the applications be. If we go back 50 years and try to come up with possible uses for traditional computers, we couldn't possible imagine how they are being used today. The same will be true for Quantum Computing and that is what makes it interesting.

We Learn by Doing

Thursday I spent the afternoon in a First Aid, CPR, and AED class. My office provided it free of charge to those of us willing to take the time to get trained and certified. This is something I try to do every couple of years but had let my certification lapse. One important thing I took away from the course is the importance of practice.

CPR is one of those skills that everyone should know but nobody should ever want to have to use. I have a close friend who had to use it on a work associate. Things worked out and the person survived. My buddy went to visit him in the hospital and apologized for breaking the guy's ribs. The survivor didn't mind as being alive sure beats being dead.

I have practiced CPR on a purpose built dummy but it is always good to get a refresher. Everyone in my class on Thursday had his or her own practice dummy and we all simulated giving CPR at the same time. While it is not the same as doing it on a real person, it is a lot better than simply talking about the procedure.

So what does this have to do with computers and technology? Lots. I am currently going through a book on Reinforcement Learning (a sub-topic of Machine Learning) and there are a lot of examples in it. Rather than simply reading about the process, it is important to work with a computer at the same time I am going through the book. Practicing what I read is really vital to my understanding of the topic. So if you find yourself trying to learn a new technology, don't just read about it, practice using it and you will learn more in the process.

Machine Learning for Predictions

This past week I have been working on trying to predict the future based on historical data with the help of machine learning. This is one of those areas where you have to be careful because if it was easy then everyone would be doing it to cash in on the stock market or for bets on who will win the next sporting event. However there are certain variables that can be used to increase the probability of an event happening. For instance, if you have clouds in the sky, there is more likely a chance of rain than on a sunny day. The trick is figuring out what those variables are and this is what is know as feature selection.

If you think about your favorite sporting event, think about all of the variables that go into a game. Who are the players and what are their various statistics? Does the weather impact those statistics or are the games played indoors? You can eventually see that complexity spirals out of control if you are trying to predict the winner of a game.

The problem I have been trying to solve is something akin to when someone will go out to dinner next based on historical information. An overly simplistic algorithm might look at the average time between restaurant visits. For someone that dines out on a regular basis, this will work. Someone else might only go out on special occasions and that won't work unless those special occasions are evenly distributed on the calendar. Think about variables that contribute to when you eat out. Some might include:

• Day of the week - Perhaps you eat out only on Friday or Saturday evenings and never on Sunday.
• Holidays - It is always more difficult to get a restaurant reservation on a holiday.
• Proximity to Payday - While it is tough to generalize when everyone's payday is, if you have that information, it might be a factor in some people's choice to go out.
• Weather - A snowstorm has the potential to greatly reduce people's desire to go out.

With a bit more time and thought, I'm sure you could come up with a number other factors that could be used to predict if you are eating out or at home. In machine learning terms, these variables are called features and can be used to accurately predict a person's behavior.

When I look at my own restaurant habits, I know that I generally eat at the airport on Thursday evenings as I catch a flight from San Francisco to Salt Lake. Unfortunately that won't be the case this evening as I decided to buy a big lunch and only eat half of it, saving the other half for this evening. Therefore it is important to remember that predictions are no guarantee that an event will happen.

Demystifying Artificial Intelligence

One of the nice things about my job is that I get to play with some of the latest technology. Right now I am doing a lot with artificial intelligence (AI) and machine learning (ML). I remember back in the 1980's and how AI had the potential to be something great. Then it stagnated for 2 decades. Due to the low cost of computer processing power and memory, AI is making a comeback. Hopefully it has a bit more staying power this time around.

As I research new technologies it is important not to buy into the hype. I read one article today claiming that those that don't make the effort understand AI and ML will begin to look at computers as magical machines that they understand less and less. That can be true of any technology and so if you find yourself thinking of a new product as magic, that is what the marketing people want you to believe.

The more I work with the current state of artificial intelligence, the more I realize it is just really good at guessing the right answer. Most AI engines get fed a bunch of numbers (characters and words can be represented by numbers) and what those numbers mean. The engines then try to figure out what a different set of numbers mean. It then becomes very important to train the AI engine with a lot of data as well as the correct data.

What happens if you train your engine with the favorite breakfast cereals of children between the ages of 5 and 7? When you ask it what an adult would like for breakfast, you will most likely get the wrong answer. While I do like Fruit Loops, I don't eat them for breakfast anymore. What happens when you recognize the age mistake but only train the engine with data from Americans? If you ask what someone from Japan eats for breakfast, once again, you will get the wrong answer.

Currently artificial intelligence is at an interesting point. We are creating useful applications for it that can greatly help us in our daily lives. It is also important to remember that it isn't magic and is only as smart as the data with which it has been trained.

Having Fun with Sentiment Analysis

Recently I have been playing around with sentiment analysis and thought I would have some fun. I decided to take my wife to lunch today and am hanging out in her law office waiting for her to finish her day so we can drive home together. She has put me in one of her unused conference rooms with my laptop which has allowed me to continue my playing.

If you want to play with sentiment analysis, you can do so with Python using the Natural Language Toolkit (NLTK). Here is a very simple program that is only 2 lines:

import nltk.sentiment.util as s
print(s.demo_vader_instance("Text to analyze here"))

The program uses the Valence Aware Dictionary for sEntiment Reasoning (VADER) created by C.J. Hutto and Eric Gilbert. It is a great little algorithm that is quite accurate.

I started by analyzing the sentence, "I love my wife" and got a score of .6369 (scores run between -1 and 1). This happens to be the same score I got for, "I love to ski." So according to VADER, I love my wife about the same as I love to ski. I showed that to my wife and got a laugh out of her. Next I added a smiley face ":-)" to the sentence and watched the sentiment increase to .7351. More laughter.

I now have a chat log with 2.3 million lines I will run through it and see what happens for each line. I will then only check a select few but time how long it takes. If you have Python and want to have some fun, load up the NLTK and give it a try.

Machine Learning and Lots of Data

At the beginning of the month I wrote about artificial intelligence and how it is not going to take over the world any time soon. I have continued to play with it and have been working on a branch called supervised learning. My basic example or use case is to feed my simple program a bunch of training sentences that are categorized as questions or statements. Then I have a test group of sentences to see how well my program has learned.

I started with a very small set of training sentences thinking that a person would be able to distinguish between a question and a statement fairly easily using just these examples. My training set began with only 30 sentences and my test set had 20. After training, my program correctly identified 16 of the test set. That sounds pretty good at 80% but I really need to get closer to 100%. So I added more training data. I found a list of 800 random questions and added them as well as several pages of text from two popular books I found online: Uncle Tom's Cabin and The Old Man and the Sea. That brought me closer with 18 sentences correctly identified as statements or questions.

The statement, "I like to ski," was wrongly classified as a question while "Who is your favorite actor?" got classified as a statement. So I added more training data until I got 100% correct classification. My original training data started with 30 sentences and is now close to 1000. That seems like a lot of extra work.

Now it is time to tune my algorithm. There are some things I can do to get better results with less data. However if you plan to embark on your own supervised learning project, be prepared to collect a lot of training data. You will need it.

Learning from the Internet

The Internet is full of wonderful information and can be a great source of learning. It can also be full of lies and misdirection so you need to be careful with the information you find.

Today I am working on machine learning where I classify types of sentences. There are plenty of examples already created and so I am trying to leverage them instead of trying to recreate something. One of my Internet searches brought me to an interesting article that has nothing to do with machine learning or even sentence classification. It is an article about 5 writing tips that C. S. Lewis sent to a young aspiring author back in 1954.

I wrote a book back in 2001 called "Programming with Oracle Developer." Now I consider it a cure for insomnia but I am still proud that I managed to devote enough time to write a book that got published. During the process I would submit a chapter to the publisher who had several editors review it. One focused on the technical correctness while another fixed the grammar and wording. That process taught me that my writing needs a lot of fixing. So if you are like me and interested in learning about how to be a better writer, take a look at the link above.