Oct
13

2 Pizzas or 1 Duck

One of the questions I get asked about all the time when starting new cloud projects is what should the team look like that’s building the cloud. I usually respond by saying “you don’t want anymore people on the core team than you can feed with 2 large pizzas”. Of course if you’re in China then I guess you could say no larger than what you can feed with a Peking Duck. This little saying has pretty much held true after over 100 cloud build-outs. If your core, decision making team is larger than about 6 people then your cloud project will go on and on and has about a 90% chance of dying. Stick at or under that team number and you’re good to go.

It’s nice enough to have an idea on the number of people on the core team but who should really make up this team? Recently I came across a Microsoft TechNet article that listed out some people for the cloud team. The article is an ok read. It was obviously written by someone that doesn’t have a lot of practical experience building clouds but that aside it does have some ok points. I want to take a different tact here and tell you what we at VMware ask for in a cloud team knowing full well from a lot of practical experience that this is what it really takes.

Project Manager – First and foremost – a cloud Czar (aka Project Manager). I can’t stress this enough that you absolutely must have a really good project manager looking over the entire project. This isn’t just someone who tracks the project and looks at the milestones. This is someone that will really own the project, drive people to do what they need to do, document the decisions being made, and most importantly enforce the decisions that are made across the company. It’s critical that you give this person enough authority over everyone in IT and enough power behind them to enforce your decisions company wide. If you fail in this one aspect then your cloud project will fail – I will personally guarantee that.

Executive Sponsor – Successful clouds are built from the top down meaning you need a good and powerful executive sponsor. You need someone that can work across business units to get things done that need escalation. You’ll need someone that has good friends at high levels outside of IT as well.

Network Administrator – Networking is perhaps what I spend most of my time on when building out a cloud. It gets complex fast. There’s also a lot that sometimes needs to change – at least from a policy standpoint – when building out a cloud. Networks tend to be fairly static and regulated environments for good reason. However, in cloud we need to leverage new technologies to make the network become more dynamic and to scale much higher than ever before. The network architect is going to be a key cornerstone to this project.

Storage Administrator – Another thing that gets complex is the storage layout. This is another typically static or at least not very dynamic environment that gets changed a lot with cloud. Not to mention the storage people aren’t used to a lot of unknown growth or usage that isn’t planned out and officially requested. This can be challenging to work into the existing storage environments.

Software Developer – A large part of cloud is automating the environment. Sometimes there comes a need to augment the portal with new UIs as well. A good java developer is always someone you want to keep close at hand in any cloud buildout.

Operations Manager – Eventually you’re going to have to hand off your new, shiny cloud to the ops team. You’re going to be handing them something that’s dynamic and multi-tenanted and stretches across several different disciplines. Clouds have a habit of throwing the ops teams for a loop so the more you can do to have the operations manager involved in the cloud buildout and decisions the better off you’ll be.

Cloud Architect – I guess this one goes without saying, but you need a Cloud Architect. You need someone who’s going to guide the decisions that are made in the entire cloud project. Someone who can make the final decision on which way to go and why you’re going that way. The person you pick for this role should have a wide background in technology and be deep in at least storage, networking, and programming.

Other people that you will reach out to but don’t need to be on the core team:

Windows Admin – Eventually you’ll get around to building templates for different Windows applications. When that time comes you’ll need a Windows admin to assist. You’ll also use the Windows admin to build out some of the infrastructure of the cloud.

Linux Admin – See Windows admin. Replace Windows with Linux. :)

Database Administrator – A lot of the moving parts of cloud consume database services. Whether it’s the front-end portal or the various back-end components or the message bus or the CMDB the cloud is going to need to hook into one or more databases. Do yourself a favor and just get the DBA as part of your team now to avoid future headaches.

Infrastructure Admins/Architects – You’ll obviously need the infrastructure admins as part of the extended team so you can build out the cloud.

Finance Representative – This is the person that everyone overlooks but is critical for cloud. Licensing is one of the 3 cloud killers and the finance representative holds the keys to that kingdom. The finance person can also help you adjust the supply chain to get infrastructure components moving in faster so your cloud can scale dynamically. Definitely don’t overlook having a good finance person as part of the extended cloud team.

So there you have it. My list of the team you want to pull together to build your cloud. If you have any additions or comments please leave them below for all to read.

Oct
10

How-to Setup AMQP with vCloud Director

Earlier this week I went about setting up notifications in VMware vCloud Director. Notifications (or callouts as some people call them) is a way for vCD (vCloud Director) to notify external systems of events. These events can come through as just standard notifications that are non-blocking or more advanced messages which are blocking. In the case of non-blocking events every action in vCD gets published to the message bus. This is good for things that external systems want to be aware of but don’t need to really act on before vCD continues with what it’s doing. You can also setup blocking tasks which allow for external systems to still get notifications but then vCD will wait for a reply message to continue, fail, or abort the task before proceeding. For either type of notification you need to setup a AMQP 0.9 or higher message bus in order to receive the notifications. That will be the focus of this blog post.

First thing is first, you’ll need an installation of AMQP. There are several AMQP compliant products out there. For this demo I am choosing RabbitMQ because it’s free and VMware owns it. RabbitMQ also happens to be extremely simple to use with a wide array of support tools which makes things even better.

Our first step is to install RabbitMQ. It supports a lot of different platforms. I chose to do the install on Windows just because I happened to have a system already running and was too lazy to go and install something else. For quick and easy steps to install RabbitMQ on your system just go here. RabbitMQ is built on Erlang so the first thing to do is go and install Erlang. All you have to do is download the latest version of Erlang here and run the installer (next, next, next, etc). Done.

The next step is to download the latest RabbitMQ installer from here and then run the install program (next, next, next, etc). Done. This will setup a default installation of RabbitMQ which will be just fine for what we need to do. For more advanced configurations or suggestions on how to get ready to go into production (changing passwords, hardening, etc) you’ll want to see the customization page on the RabbitMQ site.

The last thing you’ll want to do is an extra step to install the management site for RabbitMQ. You could setup RabbitMQ by using the command line tool or by editing configuration files but it’s much easier to install the free management UI. You can find instructions and the download link for the management plug-in here. The management plug-in consists of 6 files that you’ll need to download. The files are Erlang files and so they have a .ez extension.

Once you download them you’ll want to put all 6 of these files in the plug-in directory. For my Windows setup the directory was C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-2.6.1\plugins. You can find your plugins directory path here. Once you’ve got the files copied to the right place you just need to restart the rabbitmq service. Again, on my Windows system this was just the following 3 commands:

> rabbitmq-service.bat remove

> rabbitmq-service.bat install

> rabbitmq-service.bat start

Once this is done you can just open a browser and go to http://server-name:55672/mgmt/. You will need a username and password. The default install of RabbitMQ sets it to guest/guest.

Now that RabbitMQ is running we need to setup a new exchange and queue and then point vCD to that new setup.

The first thing to setup is the exchange. The exchange is what you’ll point vCD to. You can setup an exchange in one of several different types: direct, topic, fanout, system, or header. The exchange describes how external systems attach to the bus and “exchange” information with it. The exchange also describes how the messages get to different queues. It’s a little long for this blog post but it’s important to note for this setup that you want to AVOID direct. A direct exchange will only deliver message to a queue if they directly match the exchange routing key and the queue routing key. Since we’re sending lots of messages over we want to setup a topic or fanout exchange. Both of these exchanges will route messages based on pattern matching and as such will get our vCD messages to the right queue.

UPDATE (October 11, 2011) – Some added notes on exchanges based on some feedback from one of our gurus inside of VMware. These are great notes and I wanted to make sure they were in here.

  • With a fanout exchange the routing key is ignored by the broker (both in the message and in the binding) and the message is broadcast to all queues bound to the exchange.
  • Using # in a binding is a wildcard that matches one or more segments in the routing key. * matches a single segment.
  • VCD does in fact set sane routing keys that you could route messages on to filter in/out messages you care about. It’s particularly useful when consumers only care about particular kinds of messages.

To setup the exchange just go to the exchange tab in the management interface and click the triangle next to “Add a new exchange”. Fill out the information below that box making sure to enter “topic” for the exchange type and also make sure to select “Durable” for the Durability type. Durable means that the exchange will be started with server reboots and available all of the time.

Once you’re done with the exchange you will see it in the list of exchanges as follows.

Now that we have our exchange we need to setup a new queue for the messages to land in. Go to the Queues tab for this task and click on the triangle next to “Add a new queue”. This box is pretty easy to fill out. Create a new queue and give it a nice name and make sure to make it Durable as well. NOTE: For this example I create a queue called “vco” because I was also using this for a vCO demo.

Once you’re done you should see the queue in the list. NOTE: Your queue should show 0 ready and total. My queue has been running for a little while and has vCD properly configured already.

The last thing we need to do is to bind the queue to the exchange so messages can flow into the queue correctly. In order to do this click on the name of the new queue you created in the list. Then scroll down to the bindings section. If needed you’ll need to click on the triangle next to “Bindings” and then scroll down to the “Add binding” section. Fill out the name of the exchange we created earlier. For this demo the exchange was “vCD”. Enter the “#” sign for the routing key. The routing key is in front of every message to demark the beginning of a new message and to tell the bus what message string the new message belongs to. vCD uses the “#” key. Your form should look like this.

OK. Now all of the RabbitMQ stuff should be setup and ready to go. Now it’s time to head to vCD for the final configuration. Head to the vCD UI and login as the system administrator. Once you are logged in navigate to the Administration tab and then go to Blocking Tasks. Once on the Blocking Tasks go to the Settings tab. On this tab you will see the AMQP Broker Settings.

The form is pretty straightforward at this point. Fill out the host name or IP of the RabbitMQ server. Change the exchange name to the exchange we created earlier “vCD”. change the username and password to guest/guest.

Once the form is filled out and applied click the Test AMQP Connection. If all goes well you should get a good message.

The last thing to do is to check the “Enable Notifications” checkbox and click apply again.

Once all that is done you should be able to go back to the RabbitMQ management page and see some messages coming into the queue. You may have to do some things in vCD to start generating events. Try and deploy a VM for example.

Congratulations! You now have messages going to the queue. Now what? What in the world do you do with these things? That’s the topic for another post. One of the things you can do is to follow this vCO blog post on consuming AMQP messages. I happen to have a little internal-only tool that I use in order to see these messages. You can see a screenshot below. In the mean time keep watching the blog for how to consume these and how to interact with blocking tasks.

Jun
12

How to Import Existing VMs into vCloud Director

Recently I was asked by a customer how they could import VMs from an existing vSphere environment into a vCloud Director environment. This particular customer is aiming to pull VMs from an existing managed hosting business so they’ll use the third of three options that I’ve described below.

For pulling VMs into vCD you basically have 3 options.

1) You can log into the vCD interface and upload VMs manually through the interface. This is done one by one. This is a good option if you’re an end user that doesn’t have vCC setup (option 2) or doesn’t want to go through the process of using the vCloud API (a more automated version of this option). The downside of this option for my customer is it would require additional storage since the VMs would first upload to a staging area (the vCloud Director transfer pool) and then copy to their destination.

2) You can setup vCloud Connector (vCC) in your existing vSphere environment and use that to move VMs from the vCenter Client interface into vCloud Director. This again is done one at a time. The benefit is you’re using your existing tool set (vCenter Client) to perform this operation. The down side is you’ll need to configure vCC for each user of the vCenter Client that wants to do this operation. This would be a viable option for my customer since they own the source vSphere environment. It still isn’t the most straightforward for them and it can’t be automated with an API call so this option is out.

3) If you own the vSphere environment like this customer does and you want to migrate that vSphere environment into a parallel vCD environment then you can just add the managing VC to the vCD environment and import VMs directly into vCD. This is the method that I’m going to detail below.

If you’re ready to go about moving VMs into your vCD environment then the first thing you need to do is prepare to add your managing VC to your vCD environment. To do this you’ll need to deploy a vShield Manager instance. Even if you don’t use the security features of vSM you’ll still need to have one present so you can get through the add VC wizard in vCD. vSM comes as a virtual appliance so we’ll begin by just adding it through the deploy OVF wizard in VC.

After vSM is deployed you can see it as a resource VM in our current vSphere environment. After we finish importing everything we can remove the vSM VM again.

Now that we have vSM in place we can go and attach our managing VC to our vCD environment. Right now my test vCD environment already has one VC under its control as you can see here.

Now we’re going to go through the wizard to add our second VC which is the one we want to import our VMs from. The wizard is pretty simple. First we give it our VC name and credentials.

On the next screen we give the name and credentials to the vSM we deployed earlier.

After the wizard completes you can see we have our old VC and our new VC under the control of our vCD environment. This does not preclude you from continuing to use your vSphere environment as you always have. It just now allows vCD to see and consume some of those resources such as the existing VMs.

The next step is very important. We need to login to vCD as the system admin. This is because org admins don’t get to see the underlying vSphere resources and so they won’t be able to do the import straight from VC. Once we’re logged in as the system admin we can open up an organization to perform the import of VMs into that org. In my case the test organization is called MikeD. You can see the System tab in the screenshot below as well which indicates that we were initially logged in as the system admin and simply opened the organization in another tab.

Now that we’re inside the organization of our choice we can click the import VC button on the top toolbar.

The import vApp/VM wizard should appear and we can see a listing of all of the VMs in our vSphere environment. To import one it’s as simple as selecting the VM, giving it a name, telling vCD which orgVDC we would like to place it in and then hitting ok. There is one last choice to make and that’s whether or not to copy the VM to vCD which will put a new instance in the orgVDC but leave a copy on VC or move the VM which will delete the VM from the original VC environment and make a new copy in the vCD orgVDC resource destination. This choice is up to the person doing the migration and both choices have merit. A lot of people like copy because they have the original copy in place should a customer decide they need to continue using it or they just want it for backup.

After you’re done with the wizard your VM will start to import. After a little while you’ll see the finished VM in the interface. Of course you’ll also see the in progress VM along with a task that’s running.

As you can see the overall process for setting this up is not that labor intensive. It is a little labor intensive to go into the interface to do the import one VM at a time. Of course you could always use the vCloud API to do the imports once you have the vSphere environment added to vCD.

All of this will allow current managed hosting providers to migrate their VMs to the new vCD based clouds they are creating. Hopefully this has been useful. As always if there are any comments or questions then just put them below and I’ll get back to you.


Jun
06

VMworld 2011 – vCloud Networking Finally Explained

VMworlds are right around the corner. I just got notice that my session on vCloud Networking got accepted. Looks like I’ll see some of you in Vegas. I actually presented this topic at Partner Exchange earlier this year and it was very well received by all. I’m adding in the new stuff we’re working on for later this year to make the presentation even better. This is my official call for content.

If anyone has anything they’ve wanted to know about how vCloud Director networking actually works or how it ties into vShield then now’s the time to ask so I can get it into the presentation. Just comment below and we’ll get it sorted out.

Here’s the session information for those of you building out your schedules (when that becomes available). I’ll be presenting at the US show only since my wife and I have a baby due at the beginning of VMworld Europe this year.

Session ID: CIM1600
Title: vCloud Networking Finally Explained
Track: Enterprise Hybrid Cloud


May
16

Update on Blog

Well, things are back up. New theme. New client. No content. Wait…where’s the content? Turns out my database is really messed up from the old blog. Going to try and create a new blog site and restore the data there and see if that works. Not really sure if that will fly or not. In the mean time I’ve been able to open up my old blog client and find the posts back to June of last year there. Of course my more popular posts are older than that but for now I’ll open each of those old posts up, create a new post with the same content and the same dates, and get that content on the blog.

Unfortunately it doesn’t look like the old comments are going to come back to the new posts of old content. I still have all of the old content but without a lot of magic and time doing redirects I don’t think the comments are going to come over. We’ll see. Disqus is a pretty awesome system but I don’t think it’s gotten to that point just yet.

Anyhow, that’s the latest. Going to finish adding the last part of the blog pieces into place today and then work on some new content for you to read so you’ll forget all about the old stuff.


May
12

What The?

What in the world is going on here? Where’s Mike D’s blog, man? Well, to make a long story short, technology didn’t really go my way today. I was trying to put up a new post and images just wouldn’t upload from my existing blogging client (MarsEdit). Luckily the guys at Red Sweater support were johnny on the spot and helped me with a lot of troubleshooting. The general diagnosis was something is bad wrong with your WordPress setup. I was hosting it on Bluehost.com and access to log files there isn’t really all that stellar so it was really hard to find out what was going on with my blog. I started to tweak things and stuff started getting worse. It wasn’t fun.

So, to keep this sort of short here we are with a completely new blog setup. I’m also using a new blog client on my Mac – Blogo. The new client really wasn’t necessary but I thought I’d try it out since MarsEdit really wasn’t adding features or fixing bugs as fast as I’d hoped. I also like the new interface a little better.

So here we are with the new blog. It’s going to take some time to get everything setup right again. Not sure I’m entirely crazy about the theme but I thought I’d go with more built in widgets for building the pages versus hand banging php. We’ll see.

I’ll try to get the old content back up and running soon with a link to it or importing it here. I’ll just have to see how things move along. A lot going on over at VMware right now.

Thanks for the patience. Watch for more updates soon!


May
02

VMware is Building Clouds?

I was sitting here going through some email today when I got a nice slide sent to me from some of the vSpecialists at EMC. I hope they won’t mind me sharing this with you but I thought it drove home a point I’ve made 100 times before – if it’s not the Amazon, Microsoft, Google, or Rackspace cloud then it’s a cloud built on VMware. Why do I drive this point home? Well, I guess a lot of people read stuff written by Simon Crosby. Personally I stopped reading his writings or more like rants well over a year ago. I just didn’t have time to pay attention to people that live in alternate universes when there’s so much work to do in our own universe where most people live and work today. Just recently Simon started pitching OpenStack really hard (which I don’t blame him for). He starts off with:

I hear from almost every Enterprise and service provider that vCloud is not what they want. Why? A significant concern is that it locks you into a single vendor cloud model that is utterly undifferentiated yet appallingly expensive, nickel and dime-ing you on a per-VM basis for every possible feature. Project Redwood – a massive undertaking from my friend the Sheriff of Redwood Valley to build the perfect, yet perfectly mismatched cloud platform - really is deadwood.

Well if that isn’t an interesting opening paragraph I don’t know what is. Maybe I should start off that way with my blog posts. This paragraph got me thinking – have I really just been running around the world spinning my wheels? I mean it seems like the team has built a lot of VMware powered clouds running vCloud Director. I know of at least 80 successful projects on the Enterprise and Service Provider side around the world. I know we keep announcing more and more and more and more huge providers launching vCloud Datacenter services. This brings me back to the slide I got from the EMC guys. What’s nice is if you follow the lines for the clouds that most people have heard of they all point back to VMware.

VMware Powered Clouds

Yeah people are using other products as well which I’m not surprised about. It goes back to Simon’s point that some people just want to have options and that’s fine. But remember, most of the people out there want to be able to get into the cloud easily as well as back out of the cloud easily. That’s the key to federation. It’s baked into the NIST definition of cloud. It’s baked into the Gartner definition. It’s baked into the VMware definition. It’s a core tenant of cloud to be able to move back and forth easily. This is one of the reasons that we (VMware) built the vCenter Cloud Connector – so you can move workloads back and forth to and from the cloud with the same tools you use to manage your VMs today. And given that most reports still have 89% of the VMs on the planet as VMware VMs it only makes sense for people to build VMware powered clouds to make this transition easier.

Yes, I know that Amazon has a slick little tool to help you move VMware VMs into the Amazon cloud. But how do you get back out of the Amazon cloud? Uh, oh. There’s even companies like CloudSwitch that will help you move stuff around. Go ahead and try and use them to yank stuff back out of the cloud to your own datacenter. Don’t mind the fact that now you have yet another tool you have to go use to find your VMs and manage them and that none of your other management tools talk through that interface. You’re starting to get my point. VMware is still an open system. The vCloud API is in place and allows anyone to write tools to talk to these vClouds running around the world and many people have those tools written today. Go use the tool that makes sense for you. You also get the choice between a lot of different cloud providers and can move freely between them. Is one charging you too much or doesn’t offer services that you want or in the location you want? Go use another one. Just drag and drop in vCenter Cloud Connector and you’re over to the new cloud in a flash. That’s federation. That’s open. That’s choice.

So while Simon will keep ranting in his alternate universe the rest of us have real things to worry about and real jobs to perform and for that VMware will continue to help Enterprises and Service Providers alike continue to build open clouds with more features and more choice to make your lives easier. Execution is the key and I think the slide I got from EMC drives that point home on who is really executing and who is just blowing smoke.

 

Mar
31

Automate Your UCS Environment

I’ve seen a ton of UCS environments out at the various cloud providers and enterprises I’ve been working with. You have to hand it to Cisco – they have really built something fantastic and know how to sell it. That’s not the point of this post though. This is to announce our new vCO plug-in for UCS. We developed this in conjunction with the UCS product engineering team at Cisco. The plug-in has over 90 out-of-the-box actions and workflows, the plug-in adds management of compute hardware to vCO,  therefore enabling the automation of the entire physical and virtual stack. Together with the vSphere and vCloud Director plug-ins, the UCS Manager plug-in provides the missing piece to automate several use cases, including:

1. Provisioning UCS blades for virtualization
2. On-boarding a new tenant for a service provider
3. Automatically scaling UCS compute resources

Below are some resources to help you better understand the value and scope of the Cisco UCS Manager plug-in:

Plug-in download page: http://downloads.vmware.com/d/details/vco_ucs_plugin_1_0/ZHcqYnR0KmpiZCpwcA==
Release Notes: https://www.vmware.com/support/orchestrator/doc/ucs_plugin_10_release_notes.html
User’s Guide: http://www.vmware.com/pdf/ucs_plugin_10_users_guide.pdf

All of this is free including the very powerful vCenter Orchestrator solution. As I’ve said before, if you haven’t looked at vCO then the time is now. More plug-ins are in development for this incredible platform.

Mar
30

VMware OS Compatibility – Upgrading Windows

In the past when I’ve been at customer sites and in competitive situations I’ve often brought up the broad support for different OSs as a VMware strength. I think at last count there were over 80 different versions and flavors of various operating systems supported on the VMware platform – probably more than that today. What brought this back to mind was a recent video where someone used VMware Workstation to show every Windows upgrade/install from every version of Windows ever. First, I’m amazed the guy had that much media saved up to do the installs. Second, I’m really impressed that he got through all of the upgrades without anything crashing. Third, this really shows that anything can run in a VMware VM. That last point is for all of those people out there that still think virtualization isn’t for them and it’s just a passing fad.

Anyhow, here’s the video. It’s a little boring to watch all the way through but never-the-less very impressive.

Mar
29

Another Day, Another Cloud

Today, SingTel announced their PowerOn cloud. This is a new VMware powered cloud based on the VMware vCloud Datacenter Service. VMware had previously announced a partnership with SingTel but now that cloud is up and operational and ready to take on load in the Asian market.

I really have to thank my team members in all of APAC for pulling together to get this one off the ground. It was a really big team effort but I especially want to thank Phil Weiss, Andrew Mitchell, and Ray Budavari. For those of you in APAC building clouds you no doubt already know these excellent individuals. For those of you that haven’t met these individuals yet you should make sure to find them at your next regional event.

The team has many more public cloud projects in the works so stay tuned for more places you can seamlessly federate your cloud workloads to.

Older posts «