I spent my day working on deploying software in our company's private cloud. It may sound complicated, but isn't really once you get past a few basic concepts. Remember that "The Cloud" means different things to different people, but generally means computer servers running in an environment that is accessible on the Internet. We have all sorts of security to keep the wrong people from being able to do bad things to our computers and so that adds a level of complexity to install software. Furthermore, we are still testing our cloud and that means servers need to be easily configured even with all of the added security.
There are several open-source tools to help with this type of software deployment. We have started using Chef and it is supposed to make our life easier. At first glance, Chef looks fairly complicated. There are recipes and cookbooks with all sorts of different options and bits to twiddle. Today I worked with another one of our software engineers and we worked on getting MongoDB up and running on three different servers. If you go to the Opscode website for Chef, you can see that there is already a cookbook for MongoDB. We just downloaded it and made simple modifications for our specific environment. Now if we need to add a fourth MongoDB server, it is just a matter of running a single command on the server and everything is ready to go.
Sir Isaac Newton once said something along the lines of, "All that I have achieved is because I have been able to stand on the shoulders of other great men." He didn't worry about recreating work done before him, he just made sure to add to it. Open-source software and Chef are good examples of that principle in action today.
Showing posts with label MongoDB. Show all posts
Showing posts with label MongoDB. Show all posts
Friday, May 17, 2013
Wednesday, February 9, 2011
Document Store Databases
Today I am doing some testing on MongoDB. We are getting ready to launch a new project and so I have spent the first part of the month getting things configured. I now have our staging area completely set up and ready as the rest of the software comes online.
Most database software like Oracle or PostgreSQL is relational. That means data is stored in tables and those tables can be related to other tables. If you have a PERSON table with five attributes (e.g., last name, first name, gender, etc.), those same attributes are stored for every person in the table. MongoDB is a document store. This means that you can keep track of different attributes for each person. You may store that Jimmy has green eyes and not care about that for any other person in the database.
The reason we are using MongoDB is because of its speed. While we can get 1,000 transactions per second out of Oracle or PostgreSQL, we are getting 25,000 from MongoDB using equivalent hardware. That is a huge difference.
MongoDB also has the ability to automatically partition the data to run on separate servers. If your database starts getting too big, simply add another server and the data is automatically partitioned onto it. Then when you go to look for Jimmy's data, both machines look through half as much data so you get even more speed.
Of course MongoDB does have a drawback. For starters, there are not really any reporting tools. In our system we will be doing nightly pulls of the data and inserting it into a PostgreSQL database. Then we can run reports on a daily basis using tools like BIRT or JasperReports. There is also a bit of a learning curve for database administration. However these issues are a small price to pay for such speed.
Most database software like Oracle or PostgreSQL is relational. That means data is stored in tables and those tables can be related to other tables. If you have a PERSON table with five attributes (e.g., last name, first name, gender, etc.), those same attributes are stored for every person in the table. MongoDB is a document store. This means that you can keep track of different attributes for each person. You may store that Jimmy has green eyes and not care about that for any other person in the database.
The reason we are using MongoDB is because of its speed. While we can get 1,000 transactions per second out of Oracle or PostgreSQL, we are getting 25,000 from MongoDB using equivalent hardware. That is a huge difference.
MongoDB also has the ability to automatically partition the data to run on separate servers. If your database starts getting too big, simply add another server and the data is automatically partitioned onto it. Then when you go to look for Jimmy's data, both machines look through half as much data so you get even more speed.
Of course MongoDB does have a drawback. For starters, there are not really any reporting tools. In our system we will be doing nightly pulls of the data and inserting it into a PostgreSQL database. Then we can run reports on a daily basis using tools like BIRT or JasperReports. There is also a bit of a learning curve for database administration. However these issues are a small price to pay for such speed.
Tuesday, December 14, 2010
Teamwork
This morning I began documenting a project I just started. I am using a new product called Eucalyptus and want to write down the steps I went through to get everything running. Then if I have to go through the same steps in the future, I won't have to figure it out again.
As I was adding comments to my notebook, I turned past a number of entries for new products I have learned this year. While there are the usual notes about PostgreSQL, MySQL, and other databases, there are also notes for MongoDB, BIRT, Fabric, and other software. I wish I could say that I discovered all these products myself, but that would be a lie. The reason I am using them is because of various people I work with and their recommendation to help make my job easier.
I used to do consulting and had my own bag of tools. While I would work with others, rarely would they make recommendations of other products that might help me out. Now that I am in more of a team environment, there is an open atmosphere of sharing. I find this to be a refreshing change and am enjoying the chance to add to my toolbox. So if you find yourself struggling with a particularly nasty problem, seek out a trusted resource and see if there isn't another tool out there to make the problem go away.
As I was adding comments to my notebook, I turned past a number of entries for new products I have learned this year. While there are the usual notes about PostgreSQL, MySQL, and other databases, there are also notes for MongoDB, BIRT, Fabric, and other software. I wish I could say that I discovered all these products myself, but that would be a lie. The reason I am using them is because of various people I work with and their recommendation to help make my job easier.
I used to do consulting and had my own bag of tools. While I would work with others, rarely would they make recommendations of other products that might help me out. Now that I am in more of a team environment, there is an open atmosphere of sharing. I find this to be a refreshing change and am enjoying the chance to add to my toolbox. So if you find yourself struggling with a particularly nasty problem, seek out a trusted resource and see if there isn't another tool out there to make the problem go away.
Subscribe to:
Posts (Atom)
