Digg this topic Add to my del.icio.us Submit to SlashDot  
Reply to this topicStart new topic
> Poor man's cluster for Windows (DRBD lookalike)
Rating 5 V
Dr. Net!
post Dec 6 2007, 08:33 AM
Post #1


Newbie
*

Group: Members
Posts: 3
Joined: 6-December 07
Member No.: 12,214


Austria


Hi all,

First of all: Thank you Olof - you give me hope ...

Please excuse a longish first post!

A very common usage scenario I have encountered over and over for the quite some years now is "Semi high availability": When a Server dies, it is acceptable for users to be "thrown out" of a server-based application, but after a maximum of 2 minutes or so they need to be able to log in again and continue work - even if the dead server has developed a hardware fault or won't come up again for any other reason.

This forces some failover model between a pair of servers, and while the switchover between two windows servers is not trivial but manageable, there remains the issue of data: There is no way to make sure a standby server has up-to-date data on its storage without really serious storage devices, say: Fibre or NetApp etc., which are more often than not prohibitivly expensive.

In Linux there exists DRBD, which is essentially a Raid 1 over TCP: The disks of two servers are kept in sync on a block level via the network, (at most) one of the machines being the master at any point in time - this master being able to actually use the disk. Reads are satisfied from the master's local disk, while writes are handed to both the master's and via TCP to the slave's disk. When the master dies, the slave can become the new master and now has an up-to-date local disk to use when going from standby to active state.

This eliminates the need for expensive shared storage, cutting costs for a typical installation by 50% or so.

Most of DRBD is about optimization, control logic, authentication etc - things that can more often than not be handled by scripting, if the failover window is not in the 2sec, but in the 2min range.

Our current solution is to set up pairs of Linux servers with DRBD, than run VMware and a single guest Windows instance, the DRBD volumes being presented to Windows as virtual disks. Essentially Linux "miniclustering" and virtualized Windows handles the workload.

Enter ImDisk with the proxy mode!

A quick look a the source makes me quite sure, that besides using a file backend OR establishing TCP connections to devio, it should easily be feasable to read from the file backend, while writing to file AND devio/TCP. A minimal addition to the initialization would have to provide, that the file backend and the devio connection agree in backend size. Connection timeout or socket error would have to give a retry or two, then just drop the TCP part an run as a "normal" file backend.

Would you consider adding such a mode?

I would be happy to provide the integration - i.e. a complete set of scripts and hacks, maybe consolidated into a service:
On the master side it would wait for a slave to appear (honoring a timeout), then set up ImDisk as a master and signal the application to start. On the slave side it would heartbeat the master, and - when the master stops responding - become a master itself and signal the application to now run here.
If the application used a different IP (or better: different NIC) than the heartbeat, a "netsh interface ip name ip" could provide for failover of this "payload address".

Does this make any sense?

Best regards & thanks agian,

Eugen
Go to the top of the page
 
+Quote Post
Olof Lagerkvist
post Dec 8 2007, 09:25 PM
Post #2


Advanced Member
***

Group: Developer
Posts: 120
Joined: 27-April 07
From: Borås, Sweden
Member No.: 6,234


Sweden


I must say this was interesting to read.

One thought that appeared in my mind when I read it was, how do you keep the virtual disk cache synchronized? If there is data written by the application but not yet completely written to disk the filesystem could be in a state where it needs a chkdsk before re-mounted from another system.

Anyway, what you are asking for should be really easy to do as a simple user-mode proxy application for imdisk. Essentially, all it would need to do would be responding to I/O requests and pass them to an image file just like the current devio does but also queue the write operations to devio listening on another machine. That is, a simple modification to the current devio.
Go to the top of the page
 
+Quote Post
Dr. Net!
post Jan 23 2008, 11:10 PM
Post #3


Newbie
*

Group: Members
Posts: 3
Joined: 6-December 07
Member No.: 12,214


Austria


Hi Olof,

sorry for the long delay - quite busy at the moment!

QUOTE (Olof Lagerkvist @ Dec 8 2007, 10:25 PM) *
One thought that appeared in my mind when I read it was, how do you keep the virtual disk cache synchronized? If there is data written by the application but not yet completely written to disk the filesystem could be in a state where it needs a chkdsk before re-mounted from another system.


There are 2 answers to this:

The simple version: We are talking of the block device layer - so whatever comes above is of no interest to us.

The correct(er) version:
A ) Whatever client consumes a high availability storage service is likely to use cache passby. I am not a Windows expert, but i am sure there exists something like the Linux IO_DIRECT on Windows (in fact, I know MSSQL-Server to use it)

B ) The need for chkdsk could easily be satisfied by the scripting part that glues the whole consept together. This might push the time for recovery a bit, but it is easily makeable. Since the original slave activly switches to master because he detects a master failure, the slave definitly knows about the need for chkdsk, so he might as well run it.

QUOTE (Olof Lagerkvist @ Dec 8 2007, 10:25 PM) *
Anyway, what you are asking for should be really easy to do as a simple user-mode proxy application for imdisk. Essentially, all it would need to do would be responding to I/O requests and pass them to an image file just like the current devio does but also queue the write operations to devio listening on another machine. That is, a simple modification to the current devio.


There is one additional issue (good-to-have, not must-have): Both instances must agree on the disk and block size

So I think for a proof of concept we only need a modification to allow local storage AND proxy mode to run parallel for write, with local storage only for read.

I would create some batch files for the handling and set up a testbed, seriously thrashing it.

If it looks promising, further work can be put into making it a self-containing productionready tool

Would you give it a shot?

Best regards,

Eugen
Go to the top of the page
 
+Quote Post
Olof Lagerkvist
post Jan 25 2008, 03:32 PM
Post #4


Advanced Member
***

Group: Developer
Posts: 120
Joined: 27-April 07
From: Borås, Sweden
Member No.: 6,234


Sweden


QUOTE (Dr. Net! @ Jan 24 2008, 12:10 AM) *
...

So I think for a proof of concept we only need a modification to allow local storage AND proxy mode to run parallel for write, with local storage only for read.

I would create some batch files for the handling and set up a testbed, seriously thrashing it.

If it looks promising, further work can be put into making it a self-containing productionready tool

Would you give it a shot?


Yes definitely, this sounds interesting and feels like a "must-try/must-do" thing (IMG:../forums/style_emoticons/default/thumbup.gif)

Problem is that I am (extremely) busy, but I will give it a try a soon as I get some time! (IMG:../forums/style_emoticons/default/cheers.gif)
Go to the top of the page
 
+Quote Post
Dr. Net!
post Feb 12 2008, 11:51 PM
Post #5


Newbie
*

Group: Members
Posts: 3
Joined: 6-December 07
Member No.: 12,214


Austria


QUOTE (Olof Lagerkvist @ Jan 25 2008, 04:32 PM) *
Problem is that I am (extremely) busy, but I will give it a try a soon as I get some time! (IMG:../forums/style_emoticons/default/cheers.gif)


Thanks a lot, I appreciate it!

Best regards, Eugen
Go to the top of the page
 
+Quote Post
« Next Oldest · ImDisk · Next Newest »
 

Fast ReplyReply to this topicStart new topic

Members Who Viewed Topic Today ()

 

Display Mode: Standard · Switch to: Linear+ · Switch to: Outline

Track this topic · Email this topic · Print this topic · Subscribe to this forum

RSS Lo-Fi Version Time is now: 27th February 2008 - 05:14 PM

MKPortal ©2003-2006 mkportal.it