Load balancer as proxy or redirector

Question

The current situation: I have written an c# application server, which communicate with some applications (Computer/Smartphone/Web). Now I have the problem, that the application server has to deal with a lot of requests and it is going to be very slow.

My idea was to change the application server to work in a software cluster. To select the correct application server I want to write a load-balancer who choose the application server with the lowest workload.

My problem is, that I don't know how to write the load-balancer. Should the load-balancer work as a proxy, so that all the traffic goes through the load-balancer or should the load-balancer redirect to the application server and the application communicate directly with the application server.

Answer 1

Actually there are off-the-shelf products which do exactly what you're looking for, one of the most established ones is HAProxy that acts as a HTTP/TCP Load Balancer/ HA proxy, it can select appropriate server based on previous client requests (e.g. by cookie -insertion, it supports other methods), which I believe does exactly what you need.

back to the question,

Should the load balancer work as a proxy, so that all the traffic goes through the load balancer or should the load balancer redirect to the application server

Proxy implementation is a normal route to take, and Redirecting is not such a good idea and cause some disturbing issues on client-side specially browsers (e.g. bookmarks won't work as intended) and I would say it wouldn't have much gain over using proxy (aside from removing load balancer node if balancing is going to be done on client-side)

that i don't know how to write the load balancer

Short answer is you don't need to write your own, as I said before there are well established products in this area, however if you want to write your own HAProxy Architecture manual and Writing a load balancer proxy from ground up would be good start.

Answer 2

Redirecting would change the URL for the end-user, which is usually not a good idea.

What you're attempting to do is possible, but very complicated. There are numerous factors that constitute 'workload', including CPU, drive activity (possibly on multiple drives), network activity (possibly on multiple network cards), and software locking. Being able to effectively monitor all of those things is a very large project (I've never even heard of anyone taking locks into account). Entire companies are dedicated to doing stuff like that.

For your situation, I would recommend Microsoft's built-in Network Load Balancing. It does more of a random load balancing, but it gets the job done, and for the vast majority of applications, random distribution of requests results in a fairly even workload.

If that's not sufficient, get a hardware load balancer, or plan on at least two weeks of hardcore coding to properly balance based on CPU, drive activity, and network activity.

Answer 3

There are ready to use load balancer like Apache + mod_cluster. Configuration can be created like .... Apache+mod_cluster -> Tomcat1 , Tomcat2 , Tomcat3 ,Tomcat4. All request will come to Apache+mod_cluster and if it not static than distributed between Tomcat1, Tomcat2 , Tomcat3 , Tomcat4. If request is static type than it will be handle by Apache only . It is possible and advisable to configure Stick Session.

Main advanteage of mod_cluster is that Server-side load balance. Apache + mod_cluster can handle HTTPS request also.

http://mod-cluster.jboss.org/

Answer 4

Answering in two parts:

1. You need a Proxy functionality, and not a redirect or a router function. A redirect would reveal the IP/URL for your backend server pool to the client, which you certainly do not want. The clients could always bypass your LB once they know the backend IPs. Thus, all the traffic must flow through the proxy.

2. I would not recommend entering the realm of writing a LB. Its a pretty specialized function, and there are many free/commercial baked products that can be deployed for this. You might choose one of HAProxy, Appache HTTPD, Microsoft NLB, NginX. Each one offers a configuration choice of many load balancing algorithms, that you may want to use.