Just finishing up the final touches on my first big cloud computing project, and there was much to learn here, some pitfalls and some promises in the cloud that might help you with your cloud computing project.
As I wind down the infrastructure part of my first big cloud computing project, there are some rewards and risks, along with some pitfalls to cloud computing that you might encounter when you do your big project. Here is what I have learned.
User Management – not so easy, if you stick with the Amazon Web Services console, you are simply down to one person logging into the system. Using shared credentials is a big no no when it comes to providing security and accountability with your project. I did not find a software or product that would allow me to make multiple administrator accounts that would work for me. There are software programs that exist, but I found most of them lacking what I was looking for leaving me with the AWS console and access to the computer being shared by multiple people and one login. If anyone has a great tool that will provide the ability to audit, report, and provide multiple account support for multiple AWS images, then I would love to hear about it.
Updates – if you want to make sure your system is updated, then I would highly suggest you make your own AMI on a fully supported operating system (regardless of flavor). One of my biggest issues was finding out that YUM on Fedora would not update on the Amazon AMI, and kept on coming up with out of mirror errors. Fedora needs to step up to the plate and provide a fully functioning mirror, or better yet you should make your own AMI for AWS that provides the support you need for what you are trying to do. There is nothing more dangerous on the internet than an Operating System that cannot be patched or software that cannot be updated. If you are using a Windows AMI, make sure that everything is installed or that you have an I386 directory, there is nothing more amusing than being prompted to provide the installation disk when your OS is in the clouds.
Socialization of the product – Cloud computing is scary for many people, because it puts everything you have in someone else’s data center. For some the concept must be killed now, for others they see it as a way to save money. Regardless of what your approach is, you have to make sure you have full executive support and buy in for what you are doing; otherwise your project will languish for weeks if not months waiting for people to grab onto the concept. You should be prepared to deal with fear, FUD, and other undesirable business ways of killing off the cloud computing project.
Failure is going to happen – make sure you snap shot your Cloud Computing system before you start tinkering with it. While backups are always a good idea, it is also an excellent idea to simply snapshot what you are doing before you make huge changes to the system. This is often easier than trying to back out a pile of changes and you have no idea which change broke what.
Never ever hit “Terminate Instance” – if you hit terminate instance, it is gone, gone for good, and you cannot get it back, pray you made a snapshot before you “turn the computer off” for the night. Otherwise you get to start from scratch on the whole thing. This will cause amusing chuckles from your co-workers, and bemused responses from your Project Manager. It might also set you back unless you made a snapshot.
Logging – make sure you have a nice way of getting logs back to the company, or that you are going to be able to process logs on the cloud computers. You will want this for compliance, and to know if someone is doing something unusual with your system. Syslog over the internet is not a great way to get logs off the cloud back to the company. You want to use a VPN to make this happen, and it is easy to set up regardless of chosen operating system.
Documentation – you want to make your own documentation on the project you are working on, do not rely on AWS documentation to help you, often the AWS documentation is difficult to follow, understand, or read. The approach to documentation is make your own, and do it in such a way that anyone can follow what you did, how you did it, and what they need to duplicate your processes. Like any good system design or development, cloud computing systems should be documented just like any other internal project.
Make sure you have a good ROI – make sure that if you go into the cloud that there are obvious measurable cost savings in the offering. Some setups make no sense economically, while for some companies they do. Make sure you do not overspend on your project.
Do not underestimate the amount of time required – one of the biggest shortfalls of the project was incorrectly allocating time, or grossly under estimating the amount of time it would take to complete the project. In most projects I am usually dead on with time estimates, and found that I was under estimating time to completion for a first cloud computing project by upwards of 4X. If I thought it would take 30 minutes, it often took 2 hours to complete a task. Most of this is attributable to new technology, new ways of doing things, and general training ramp up time. While not a barrier to use, it was a barrier to completing the project on time.
URL’s and IP Addresses – AWS will give you a temp URL to work with when you are prototyping your system, if you use that then assign an IP address later, then a Domain Name after that, and you did not program the base URL as a configurable item, then you will have to go back and change that. The temp URL with AWS does not work after you assign an IP address. It might be easier to have already set up and IP address and DNS entry before you let the developers at the system.
Open Source Software is the best way to go here – mostly because you do not have access to a DVD drive on the cloud, and software that wants a DVD to verify you own it, or needs a DVD to load is an oxymoron in the cloud. Otherwise, make your own AMI, and make sure that what you are building is able to work without a DVD drive, or disks, or other items we take for granted in the data center where we can touch physical items.
Those are the big key issues and some interesting differences between being in the data center and working through the cloud. These are my own experiences working with Amazon Web Services over the last 60 days. Overall though I would say that the experience with AWS has been positive, but given the ramp up time, learning curve, and support issues, this project actually lasted longer by 75% than it was initially estimated to last. Most of that is again, attributable to learning curve, the promise of cloud computing for rapidly deploying a system should hold true for any other deployments that we will do in the cloud. But for the first big project, make sure you either already have trained folks in Cloud Computing, or that your project can go over time without detriment.
Make sure you are ready to be audited as well, that includes log management, user accounts, and patching/updating. While I highly recommend you make your own tested AMI, some people will not do that and whatever AMI exists already for the project they are doing. You will want to make a careful risk evaluation of that stance depending on what you are using your cloud computing system for. If you fall under SOX, HIPAA, or anything else, you want to make sure you stay in compliance.
As I wind down the infrastructure part of my first big cloud computing project, there are some rewards and risks, along with some pitfalls to cloud computing that you might encounter when you do your big project. Here is what I have learned.
User Management – not so easy, if you stick with the Amazon Web Services console, you are simply down to one person logging into the system. Using shared credentials is a big no no when it comes to providing security and accountability with your project. I did not find a software or product that would allow me to make multiple administrator accounts that would work for me. There are software programs that exist, but I found most of them lacking what I was looking for leaving me with the AWS console and access to the computer being shared by multiple people and one login. If anyone has a great tool that will provide the ability to audit, report, and provide multiple account support for multiple AWS images, then I would love to hear about it.
Updates – if you want to make sure your system is updated, then I would highly suggest you make your own AMI on a fully supported operating system (regardless of flavor). One of my biggest issues was finding out that YUM on Fedora would not update on the Amazon AMI, and kept on coming up with out of mirror errors. Fedora needs to step up to the plate and provide a fully functioning mirror, or better yet you should make your own AMI for AWS that provides the support you need for what you are trying to do. There is nothing more dangerous on the internet than an Operating System that cannot be patched or software that cannot be updated. If you are using a Windows AMI, make sure that everything is installed or that you have an I386 directory, there is nothing more amusing than being prompted to provide the installation disk when your OS is in the clouds.
Socialization of the product – Cloud computing is scary for many people, because it puts everything you have in someone else’s data center. For some the concept must be killed now, for others they see it as a way to save money. Regardless of what your approach is, you have to make sure you have full executive support and buy in for what you are doing; otherwise your project will languish for weeks if not months waiting for people to grab onto the concept. You should be prepared to deal with fear, FUD, and other undesirable business ways of killing off the cloud computing project.
Failure is going to happen – make sure you snap shot your Cloud Computing system before you start tinkering with it. While backups are always a good idea, it is also an excellent idea to simply snapshot what you are doing before you make huge changes to the system. This is often easier than trying to back out a pile of changes and you have no idea which change broke what.
Never ever hit “Terminate Instance” – if you hit terminate instance, it is gone, gone for good, and you cannot get it back, pray you made a snapshot before you “turn the computer off” for the night. Otherwise you get to start from scratch on the whole thing. This will cause amusing chuckles from your co-workers, and bemused responses from your Project Manager. It might also set you back unless you made a snapshot.
Logging – make sure you have a nice way of getting logs back to the company, or that you are going to be able to process logs on the cloud computers. You will want this for compliance, and to know if someone is doing something unusual with your system. Syslog over the internet is not a great way to get logs off the cloud back to the company. You want to use a VPN to make this happen, and it is easy to set up regardless of chosen operating system.
Documentation – you want to make your own documentation on the project you are working on, do not rely on AWS documentation to help you, often the AWS documentation is difficult to follow, understand, or read. The approach to documentation is make your own, and do it in such a way that anyone can follow what you did, how you did it, and what they need to duplicate your processes. Like any good system design or development, cloud computing systems should be documented just like any other internal project.
Make sure you have a good ROI – make sure that if you go into the cloud that there are obvious measurable cost savings in the offering. Some setups make no sense economically, while for some companies they do. Make sure you do not overspend on your project.
Do not underestimate the amount of time required – one of the biggest shortfalls of the project was incorrectly allocating time, or grossly under estimating the amount of time it would take to complete the project. In most projects I am usually dead on with time estimates, and found that I was under estimating time to completion for a first cloud computing project by upwards of 4X. If I thought it would take 30 minutes, it often took 2 hours to complete a task. Most of this is attributable to new technology, new ways of doing things, and general training ramp up time. While not a barrier to use, it was a barrier to completing the project on time.
URL’s and IP Addresses – AWS will give you a temp URL to work with when you are prototyping your system, if you use that then assign an IP address later, then a Domain Name after that, and you did not program the base URL as a configurable item, then you will have to go back and change that. The temp URL with AWS does not work after you assign an IP address. It might be easier to have already set up and IP address and DNS entry before you let the developers at the system.
Open Source Software is the best way to go here – mostly because you do not have access to a DVD drive on the cloud, and software that wants a DVD to verify you own it, or needs a DVD to load is an oxymoron in the cloud. Otherwise, make your own AMI, and make sure that what you are building is able to work without a DVD drive, or disks, or other items we take for granted in the data center where we can touch physical items.
Those are the big key issues and some interesting differences between being in the data center and working through the cloud. These are my own experiences working with Amazon Web Services over the last 60 days. Overall though I would say that the experience with AWS has been positive, but given the ramp up time, learning curve, and support issues, this project actually lasted longer by 75% than it was initially estimated to last. Most of that is again, attributable to learning curve, the promise of cloud computing for rapidly deploying a system should hold true for any other deployments that we will do in the cloud. But for the first big project, make sure you either already have trained folks in Cloud Computing, or that your project can go over time without detriment.
Make sure you are ready to be audited as well, that includes log management, user accounts, and patching/updating. While I highly recommend you make your own tested AMI, some people will not do that and whatever AMI exists already for the project they are doing. You will want to make a careful risk evaluation of that stance depending on what you are using your cloud computing system for. If you fall under SOX, HIPAA, or anything else, you want to make sure you stay in compliance.
No comments:
Post a Comment