Petri.com forums Home Forums Start Page Forums Frequently Asked Questions FAQ Member List Members List
Go Back   Petri IT Knowledgebase Forums > Virtualization > VMware Virtualization
Petri.com is happy to award RicklesP the title of Most Valuable Member !!!
Register Calendar Calendar Search Petri IT Knowledgebase Forums Search Todays Posts Today's Posts Mark Forums Read
Notices

RE:How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

RE:How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

this thread has 11 replies and has been viewed 18723 times

Closed Thread
 
Thread Tools Search this Thread Display Modes
  #1  
Old 7th June 2008, 02:47
GeoffM GeoffM is offline
Casual
Casual
 
 Join Date: Jun 2008
  6 month star 12 month star
 Posts: 7
 Reputation: GeoffM is on a distinguished road (10)
Default RE:How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

First off Excellent article, no need for negitive comments here, you addressed a needed niche, we're right in the middle of testing our VC3 Infrastructure deployment and before I order my c-Class Blade and iSCSI SAN I was looking for a way to test the full functions of VM. Thank you!

Now if only you could find a way to turn a WD My Book (Networked attached) in to a sedo iSCSI I'd really have it made
  #2  
Old 7th June 2008, 18:21
jasonboche's Avatar
jasonboche jasonboche is offline
Moderator
 
 Join Date: Apr 2006
  6 month star 12 month star
 Location: Minnesota
 Posts: 1,539
 Send a message via ICQ to jasonboche
 Reputation: jasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nice (409)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

You could connect your book to openfiler or a Windows iSCSI machine to be used as storage, but performance wouldn't be real good fo running more than 1 or 2 VMs concurrently.
__________________
VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
boche.net - VMware Virtualization Evangelist
My advice has no warranties. Follow at your own risk.
  #3  
Old 9th June 2008, 12:17
Matz's Avatar
Matz Matz is offline
Casual
Casual
 
 Join Date: Jun 2008
  6 month star 12 month star
 Posts: 8
 Reputation: Matz is on a distinguished road (10)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Hi.

-> jasonboche
OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
Why, what are the bottleneck?

Have you tried to use VMotion, did it work?


-> GeoffM
There are also FreeNas and sw based StarWind, both are good and easy to set up.

Thanks.
Matz
  #4  
Old 10th June 2008, 00:26
cmarkley cmarkley is offline
Casual
Casual
 
 Join Date: Dec 2006
  6 month star 12 month star
 Posts: 7
 Reputation: cmarkley is on a distinguished road (10)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

I tried FreeNAS and OpenFiler and didn't have much luck. I friend turned me onto DSS and it rocks. We are running Vmotion and HA w/o any issue. It runs on a USB thumb drive and they have a 1TB version for free that we use.

http://www.open-e.com/dss_lite/data_...ow=description
  #5  
Old 11th June 2008, 03:26
jasonboche's Avatar
jasonboche jasonboche is offline
Moderator
 
 Join Date: Apr 2006
  6 month star 12 month star
 Location: Minnesota
 Posts: 1,539
 Send a message via ICQ to jasonboche
 Reputation: jasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nice (409)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Quote:
Originally Posted by Matz View Post
Hi.

-> jasonboche
OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
Why, what are the bottleneck?

Have you tried to use VMotion, did it work?


-> GeoffM
There are also FreeNas and sw based StarWind, both are good and easy to set up.

Thanks.
Matz
The limitation isn't iSCSI, it's the disk you'd be runnig it on. Because you'd be running VMs on a single IDE or SATA drive in a USB "book" drive, disk I/O would start to show serious contention and bottleneck probably after the 2nd VM.

VMotion works just great on iSCSI. Been using it and Openfiler for close to 2 years.
__________________
VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
boche.net - VMware Virtualization Evangelist
My advice has no warranties. Follow at your own risk.
  #6  
Old 12th June 2008, 09:40
Matz's Avatar
Matz Matz is offline
Casual
Casual
 
 Join Date: Jun 2008
  6 month star 12 month star
 Posts: 8
 Reputation: Matz is on a distinguished road (10)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Hi

Thanks for great input.

-> jasonboche
Put together some old hw.
Installed OpenFiler on a HP DL380 with 2x36GB for sys and 4x 300GB for the data.
1xGbit nic.

1. So you mean, you like OpenFiler and have used it successfully to use VMotion between 2 ESX (3,5) server(s)?

2. Any tweaks adjustment you did have to do?

3. What about performance?

*******************************
I have also installed DSS-Lite to see if that is a great iScsi application
Have you or anyone else tesed it?
What about the $$ version - Open-E Data Storage Server / Open-E iSCSI-R3 ?

*******************************

Thanks again for all input.
Matz
  #7  
Old 12th June 2008, 16:07
jasonboche's Avatar
jasonboche jasonboche is offline
Moderator
 
 Join Date: Apr 2006
  6 month star 12 month star
 Location: Minnesota
 Posts: 1,539
 Send a message via ICQ to jasonboche
 Reputation: jasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nice (409)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Quote:
Originally Posted by Matz View Post
Hi

Thanks for great input.

-> jasonboche
Put together some old hw.
Installed OpenFiler on a HP DL380 with 2x36GB for sys and 4x 300GB for the data.
1xGbit nic.

1. So you mean, you like OpenFiler and have used it successfully to use VMotion between 2 ESX (3,5) server(s)?
I have been using Openfiler as an iSCSI target for ESX 3.0.x and 3.5.x for the past 2 years which allows me to use and test all the enterprise features in virtual infrastructure such as VMotion, DRS, HA, DPM (experimental), etc. Shared storage really does provide a lot of flexibility for maintenance. Once you go shared storage, you'll never want to go back.

Quote:
Originally Posted by Matz View Post
2. Any tweaks adjustment you did have to do?
Well there's the Openfiler setup which is pretty straight forward. Then there's the configuration in Openfiler to present the disk(s) to ESX. It is pretty simple and can be done relatively quickly. Openfiler is a very fast build. As far as any specific settings or hardware you might want, I"ll speak to them below.

Quote:
Originally Posted by Matz View Post
3. What about performance?
Openfiler will install on a very wide variety of x86 and x64 hardware (if you have x64, make sure you are using the x64 compiled version of Openfiler!). As such, performance can vary greatly depending on your hardware setup. You can go as cheap as you want, recycling old hardware and minimum disk spindles which will result in a shared storage solution, but it may not perform the best. Alternatively, you could spend some money on ebay or at your local electronics store and make subtle improvements in your Openfiler hardware which will enhance the performance of your box resulting in better performing VMs and more capacity for more running VMs.

Let's go over a few of the key bottlenecks you want to address with iSCSI.

1. iSCSI is disk over network. Naturally for your iSCSI box you will want a Gigabit network card which your iSCSI traffic will traverse. A big bottleneck here would be going with a Fast Ethernet 10/100 NIC (or wireless!!), or an improperly configured Gb NIC. Naturally, you will want the other end of that Gb cable plugged into a Gb switch that supports Gb networking. Taking it a step further, you will want your ESX NICs to be Gb and also plugged into a Gb switch! Hopefully you are getting the point that sticking a 10/100 NIC anywhere in the mix will result in a less than stellar performing iSCSI datastore. While we're on the subject of NICs, I want to point out that there does exist a NIC bonding driver which rPath Openfiler Linux will support which means you can multihome your Openfiler box and bond the NICs to aggregate traffic throughput resulting in 2Gb+ speeds. This is a highly complex tweak and there is some fine print and caveats that you will have to take a look at but the option is there for you. I can provide you with the documentation on this if you'd like to explore it further.

2. Processor(s): You will find out by running TOP on your Openfiler box that the processor will become loaded when throwing a lot of iSCSI traffic at it. That said, the faster the processor, the better. Openfiler supports SMP so if you can find a multi proc box, definitely go that route. My Openfiler box is currently a 4 way P4 2.x GHz.

3. RAM: Doesn't appear to come into play much so you can rest easy in this category. My box has 9.5GB of RAM of which I don't think I'm even getting close to using a gig of it..

4. Array controller, disk spindle count, and disk spindle speed. I saved the best for last. By far the greatest improvements (and most expensive) you can make to your iSCSI box are increasing your disk spindle count to the most possible (or affordable) and going with an array controller that will be able to keep up with the number of disk spindles and the I/O reads/writes. My server has a Compaq SMART Array 6400 array controller and 12x 146GB 10KRPM hot pluggable SCSI drives. While I can't afford 15KRPM SCSI drives, that would make another great improvement in my setup. Maybe someday.

That's all for now, I'm about to get off the bus here. Good luck!!

I came back to tack on a concluding statement now that I have a little more time. I wanted to point out that even the fastest of iSCSI hardware or the average fiber channel SAN will flounder a bit when overloading it with disk I/O requests from the ESX hosts. Examples usually include doing something with multiple VMs at the same time which are disk I/O intensive. For instance, powering on several VMs at the same time. Installing a large service pack on several VMs at the same time, etc. This is where you'll see your Gb NIC shine, but your disks will be working overtime trying to keep up with all the requests. If you run TOP, you'll also likely see the processors on your Openfiler box are buried. Another thing to remember is that if you're using the swISCSI initiator in ESX, there's some overhead in that guy too on the ESX side. You'll see this by running ESXTOP on your ESX hosts when your iSCSI box is working hard. Let me clarify - it's not that your VMs will start failing as the Openfiler box works hard, they will just be slower as it churns through the disk queues. Any shared storage solution has its limitations and assuming you cover all areas, your bottlenecks will usually be disk I/O so focus on disk I/O throughput when building an iSCSI box for ESX.

Quote:
Originally Posted by Matz View Post
*******************************
I have also installed DSS-Lite to see if that is a great iScsi application
Have you or anyone else tesed it?
What about the $$ version - Open-E Data Storage Server / Open-E iSCSI-R3 ?

*******************************

Thanks again for all input.
Matz
__________________
VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
boche.net - VMware Virtualization Evangelist
My advice has no warranties. Follow at your own risk.

Last edited by jasonboche; 12th June 2008 at 16:32..
  #8  
Old 1st July 2008, 21:00
GeoffM GeoffM is offline
Casual
Casual
 
 Join Date: Jun 2008
  6 month star 12 month star
 Posts: 7
 Reputation: GeoffM is on a distinguished road (10)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Quote:
Originally Posted by Matz View Post
Hi.

-> jasonboche
OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
Why, what are the bottleneck?

Have you tried to use VMotion, did it work?


-> GeoffM
There are also FreeNas and sw based StarWind, both are good and easy to set up.

Thanks.
Matz
Thanks Matz I'll look over both (along with DSS) very througly - I provide some SMB consaulting as well and got to thinking this would be a great cost point solution when combined with a Western Digital Networked MyWorlBook (Or simular) they have a 1 TB and a redundent 1TB (or 2 TB's if not used in redundentcy mode) and while they have a 1 gbs Network interface I do understand it's not a true iSCSI with I/O and other performance capabilities.

Last edited by GeoffM; 1st July 2008 at 21:03..
  #9  
Old 13th October 2008, 10:04
sven.jambor sven.jambor is offline
Casual
Casual
 
 Join Date: Oct 2008
  6 month star 12 month star
 Posts: 1
 Reputation: sven.jambor is on a distinguished road (10)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Hi,

Just a word of caution.

I was surprised to read the article and see that Openfiler was used in this setup in combination with ESX's Enterprise features. I therefore assume that it's connected to more than one ESX host and the hosts are sharing the same iScsi targets.

I had a similar setup about a year ago - three ESX hosts and a 4TB shared target on Openfiler. The problem was that ESX would contunally get corrupt vmfs volumes, volume sizes would literally fluctuate during use, VM's woud disappear etc. So I ended up calling VMWare support (which did a great job, btw) - only to find out that Openfiler is not supported as a iScsi solution for the ESX's software redirector.

I said "yeah, so?" - but was informed that it's not only not supported - but that, like with most other non-certified iScsi solutions, it actually lacks the ability to use a few SCSI calls - which ESX DOES use. Hence the problems I was running in to.

In the end I just upgraded the exisiting "real" SAN with more storage (yup, spending MAJOR money). Right now we're using the Openfiler machine as a dedicated backup target - with huge disks and perfect performance. So all is well in the end anyway

But: if anybody has any news about ESX and Openfiler working together WELL in a PRODUCTION environment I'd be happy to hear about it!
  #10  
Old 13th October 2008, 14:22
jasonboche's Avatar
jasonboche jasonboche is offline
Moderator
 
 Join Date: Apr 2006
  6 month star 12 month star
 Location: Minnesota
 Posts: 1,539
 Send a message via ICQ to jasonboche
 Reputation: jasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nicejasonboche is just really nice (409)
Default Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

I have been using Openfiler in my home production environment for 2 years or more with 3 ESX hosts. No problems.

Not many are going to suggest using a free opensource product like Openfiler for your work production environment. Spend a little money on a shared storage solution that you'll at least get support for if you have a problem.

Jas
__________________
VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
boche.net - VMware Virtualization Evangelist
My advice has no warranties. Follow at your own risk.
Closed Thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
VMware ESX Server SAN - Wasabi or Open-E the-waves VMware Virtualization 5 15th March 2008 21:45
NAS/SAN & ISCSI Help COSY Windows Server 2000 / 2003 / 2003 R2 6 18th December 2006 23:19
Vmware ESX 3.0 yuval14 VMware Virtualization 0 19th July 2006 23:32


All times are GMT +3. The time now is 19:06.

Steel Blue 3.5.4 vBulletin Style ©2006 vBEnhanced
Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
 

Valid XHTML 1.0!   Valid CSS!

Copyright 2005 Daniel Petri