In our office we have relatively poor ADSL connectivity (4-6 Mbps - I know some people here have far worse though). We currently have two ADSL lines linked together via a Draytek router and do OK. However I'd like to think about adding a third line and that goes beyond what the Draytek can do.
So I'd like suggestions for Linux distros which could replace the Draytek and do some half decent load balancing (and possibly caching for eg Linux and Windows updates etc) across multiple connections. It would need to be able to handle a number of VPN connections too (PPTP unfortunately but that's determined by the other end points).
I'm not interested in rolling my own - it needs to "just work" in the way that the Draytek does.
ClearOS and pfSense are on my radar although I have no experience of either. Any other suggestions or advice?
On Wed, Apr 29, 2015 at 10:13:24AM +0100, Mark Rogers wrote:
In our office we have relatively poor ADSL connectivity (4-6 Mbps - I know some people here have far worse though). We currently have two ADSL lines linked together via a Draytek router and do OK. However I'd like to think about adding a third line and that goes beyond what the Draytek can do.
So I'd like suggestions for Linux distros which could replace the Draytek and do some half decent load balancing (and possibly caching for eg Linux and Windows updates etc) across multiple connections. It would need to be able to handle a number of VPN connections too (PPTP unfortunately but that's determined by the other end points).
I'm not interested in rolling my own - it needs to "just work" in the way that the Draytek does.
ClearOS and pfSense are on my radar although I have no experience of either. Any other suggestions or advice?
I have been down a similar route.
I have a Draytek Vigor 2820n load sharing across two ADSL connections.
I'm considering adding a 3G/4G connection, or maybe using it to replace one of the ADSL lines.
The best I have come up with is to use OpenWrt, I have it on a Mikrotik router at the moment. However it's certainly not really a "just works" system.
OpenWrt on other hardware might suit you though, the issue with Mikrotik is that it's at what one might call a beta stage. Other hardware (take a look at the OpenWrt web site and conversations there, plus join the user mailing list to ask questions) is more mature and will work 'out of the box'.
On 29 April 2015 at 10:31, Chris Green cl@isbd.net wrote:
OpenWrt on other hardware might suit you though, the issue with Mikrotik is that it's at what one might call a beta stage. Other hardware (take a look at the OpenWrt web site and conversations there, plus join the user mailing list to ask questions) is more mature and will work 'out of the box'.
Thanks, I'll take a look.
Are there any UK vendors that sell routers pre-installed with OpenWrt? Whenever I have looked at OpenWrt in the past I've spent more time trying to determine what hardware I should run it on (faced with a long list of supported hardware, a lot of which is no longer current, and no idea how to compare the ones that are current), and usually the investigation stalls there.
Related query:
Is there any reason why a router needs multiple hardware interfaces?
I don't see any reason why I can't have multiple ADSL routers all with static IP addresses on (say) 192.168.1.x addresses and the rest of the network on 192.168.2.x addresses, all on the same ethernet network, but that doesn't seem to be how things are expected to be set up and I can't get my head around why it matters.
On Wed, Apr 29, 2015 at 01:34:34PM +0100, Mark Rogers wrote:
On 29 April 2015 at 10:31, Chris Green cl@isbd.net wrote:
OpenWrt on other hardware might suit you though, the issue with Mikrotik is that it's at what one might call a beta stage. Other hardware (take a look at the OpenWrt web site and conversations there, plus join the user mailing list to ask questions) is more mature and will work 'out of the box'.
Thanks, I'll take a look.
Are there any UK vendors that sell routers pre-installed with OpenWrt? Whenever I have looked at OpenWrt in the past I've spent more time trying to determine what hardware I should run it on (faced with a long list of supported hardware, a lot of which is no longer current, and no idea how to compare the ones that are current), and usually the investigation stalls there.
There's one whose name I can't remember, but the OpenWrt community are having problems with them from what I remember.
Some of the modern TP-Link routers seem to go well with OpenWrt if I remember.
On Wed, Apr 29, 2015 at 02:17:56PM +0100, Mark Rogers wrote:
Related query:
Is there any reason why a router needs multiple hardware interfaces?
I don't see any reason why I can't have multiple ADSL routers all with static IP addresses on (say) 192.168.1.x addresses and the rest of the network on 192.168.2.x addresses, all on the same ethernet network, but that doesn't seem to be how things are expected to be set up and I can't get my head around why it matters.
So what good would this do you? All the routers can talk to each other, the rest of the network can communicate with itself but you have no way to communicate between them.
What advantages would using the same physical network confer?
... and what/who decides which router to use?
On 29 April 2015 at 15:09, Chris Green cl@isbd.net wrote:
So what good would this do you? All the routers can talk to each other, the rest of the network can communicate with itself but you have no way to communicate between them.
Why?
Let's say I have three ADSL routers on (respectively) 192.168.1.1, 192.168.2.1, 192.168.3.1, all with 255.255.255.0 subnets and all with DHCP turned off. The LAN ports of all of these could be connected to the same switch and they'd not be able to talk to each other as they're on different subnets. (I've tweaked IP addresses from my last post.)
I then add a new box with one network card (connected to the same switch) but a number of IP addresses on it (eth0=192.168.100.1, eth0:1=192.168.1.2, eth0:2=192.168.2.2, eth0:3=192.168.3.2), so it can talk to everything. Enable DHCP on the 192.168.100.x range.
I then add other PCs and laptops (etc) to the same switch (or via WiFi to the same network) and they all get 192.168.100.x addresses, gateway 192.168.100.1. They can only see the gateway, it can then forward traffic as required to each of the three ADSL routers.
(Aside: My routing knowledge isn't strong enough to answer this, but I can't actually see why this couldn't also work if all the routers and the rest of the network were on the same subnet.)
What advantages would using the same physical network confer?
Reason why this might be useful (and why I was thinking about it): Simpler hardware for the "box" (just one physical interface required so something like a Pi could handle it). Simpler cabling (the ADSL routers don't all need to be in one location and "behind" the "box"). It means that the number of WAN's a router can support is not limited by having to have a separate physical port for each one.
Downside is throughput, although with a gigabit card and several 4Mbps ADSL connections I don't think it's going to be struggling. Other downside might be "but it won't work", which is why I was asking!
I don't really see what is different between having eth0/eth0:1/eth0:2/eth0:3 (on a single cable) vs eth0/eth1/eth2/eth3 (on separate cards but all connected to the same switch) vs eth0-eth3 on separate networks.
... and what/who decides which router to use?
There's only one DHCP server and that would "decide". Of-course any device on the network could be configured to use one of the gateways directly but that's a "feature" rather than a problem in a small network (I accept that there are plenty of environments where you'd want to lock that down though). Even so, managed switches could control that rather than needing separate physical interfaces in the router.
On Wed, 29 Apr 2015 15:39:48 +0100 Mark Rogers mark@more-solutions.co.uk allegedly wrote:
On 29 April 2015 at 15:09, Chris Green cl@isbd.net wrote:
So what good would this do you? All the routers can talk to each other, the rest of the network can communicate with itself but you have no way to communicate between them.
Why?
Let's say I have three ADSL routers on (respectively) 192.168.1.1, 192.168.2.1, 192.168.3.1, all with 255.255.255.0 subnets and all with DHCP turned off. The LAN ports of all of these could be connected to the same switch and they'd not be able to talk to each other as they're on different subnets. (I've tweaked IP addresses from my last post.)
I then add a new box with one network card (connected to the same switch) but a number of IP addresses on it (eth0=192.168.100.1, eth0:1=192.168.1.2, eth0:2=192.168.2.2, eth0:3=192.168.3.2), so it can talk to everything. Enable DHCP on the 192.168.100.x range.
I then add other PCs and laptops (etc) to the same switch (or via WiFi to the same network) and they all get 192.168.100.x addresses, gateway 192.168.100.1. They can only see the gateway, it can then forward traffic as required to each of the three ADSL routers.
(Aside: My routing knowledge isn't strong enough to answer this, but I can't actually see why this couldn't also work if all the routers and the rest of the network were on the same subnet.)
What advantages would using the same physical network confer?
Reason why this might be useful (and why I was thinking about it): Simpler hardware for the "box" (just one physical interface required so something like a Pi could handle it). Simpler cabling (the ADSL routers don't all need to be in one location and "behind" the "box"). It means that the number of WAN's a router can support is not limited by having to have a separate physical port for each one.
Downside is throughput, although with a gigabit card and several 4Mbps ADSL connections I don't think it's going to be struggling. Other downside might be "but it won't work", which is why I was asking!
I don't really see what is different between having eth0/eth0:1/eth0:2/eth0:3 (on a single cable) vs eth0/eth1/eth2/eth3 (on separate cards but all connected to the same switch) vs eth0-eth3 on separate networks.
... and what/who decides which router to use?
There's only one DHCP server and that would "decide". Of-course any device on the network could be configured to use one of the gateways directly but that's a "feature" rather than a problem in a small network (I accept that there are plenty of environments where you'd want to lock that down though). Even so, managed switches could control that rather than needing separate physical interfaces in the router.
Mark
I think I can see what you are trying to do, but I see a problem. What is your default route to the wider internet?
All you internal systems will have a default route set as 192.168.100.1. But which router does that box choose to send traffic out? You could set up your own routing table partitioning the outside world into three bloody great subnets and allocating one to each of the ADSL routers, but that is asking a lot of your little domestic setup. Alternatively, you have one default route allocated, but I don't see how that helps you since the other two routers then become redundant.
Or have I missed something?
Mick
---------------------------------------------------------------------
Mick Morgan gpg fingerprint: FC23 3338 F664 5E66 876B 72C0 0A1F E60B 5BAD D312 http://baldric.net
---------------------------------------------------------------------
On 29 April 2015 at 18:14, mick mbm@rlogin.net wrote:
I think I can see what you are trying to do, but I see a problem. What is your default route to the wider internet?
All you internal systems will have a default route set as 192.168.100.1. But which router does that box choose to send traffic out? You could set up your own routing table partitioning the outside world into three bloody great subnets and allocating one to each of the ADSL routers, but that is asking a lot of your little domestic setup. Alternatively, you have one default route allocated, but I don't see how that helps you since the other two routers then become redundant.
Basic load sharing (and fail-over) is do-able, typically using a round-robin approach (maybe with some weighting). So each outbound connection would pick a different gateway (as far as .100.1 is concerned). How that works is beyond me but I know it can be done, it's what decent routers like the Draytek I have now will do, it's also what some Linux distros will do (I'm now playing with ClearOS which seems to have this functionality), and I have also found some scripts for setting it up (but they're more detailed than I was able to follow).
But they all seem to expect each gateway to reside on its own network interface, so if I want to have 3 WAN options for gateway then I need 3 network interfaces (plus another for the internal side). That's the bit I don't understand, and I feel like there must be a reason (otherwise they'd work the way I describe) but I can't find out what that reason would be.
On Wed, Apr 29, 2015 at 03:39:48PM +0100, Mark Rogers wrote:
On 29 April 2015 at 15:09, Chris Green cl@isbd.net wrote:
So what good would this do you? All the routers can talk to each other, the rest of the network can communicate with itself but you have no way to communicate between them.
Why?
Let's say I have three ADSL routers on (respectively) 192.168.1.1, 192.168.2.1, 192.168.3.1, all with 255.255.255.0 subnets and all with DHCP turned off. The LAN ports of all of these could be connected to the same switch and they'd not be able to talk to each other as they're on different subnets. (I've tweaked IP addresses from my last post.)
Ah, not quite the arrangement you seemed to be suggesting before, maybe I misunderstood.
I then add a new box with one network card (connected to the same switch) but a number of IP addresses on it (eth0=192.168.100.1, eth0:1=192.168.1.2, eth0:2=192.168.2.2, eth0:3=192.168.3.2), so it can talk to everything. Enable DHCP on the 192.168.100.x range.
Can you do this? I.e. have a single network card with multiple subnets? .... yes, you can, IP aliasing.
I then add other PCs and laptops (etc) to the same switch (or via WiFi to the same network) and they all get 192.168.100.x addresses, gateway 192.168.100.1. They can only see the gateway, it can then forward traffic as required to each of the three ADSL routers.
(Aside: My routing knowledge isn't strong enough to answer this, but I can't actually see why this couldn't also work if all the routers and the rest of the network were on the same subnet.)
Not the same subnet, the same physical network, they're on different subnets.
However, at this point I fail to see what difference it makes compared with simply having all the devices on the same subnet. You just have the added complication of having to setup the IP aliases on the single network card.
What advantages would using the same physical network confer?
Reason why this might be useful (and why I was thinking about it): Simpler hardware for the "box" (just one physical interface required so something like a Pi could handle it). Simpler cabling (the ADSL routers don't all need to be in one location and "behind" the "box"). It means that the number of WAN's a router can support is not limited by having to have a separate physical port for each one.
Yes, but, as I say above, what have you gained?
Downside is throughput, although with a gigabit card and several 4Mbps ADSL connections I don't think it's going to be struggling. Other downside might be "but it won't work", which is why I was asking!
I don't really see what is different between having eth0/eth0:1/eth0:2/eth0:3 (on a single cable) vs eth0/eth1/eth2/eth3 (on separate cards but all connected to the same switch) vs eth0-eth3 on separate networks.
... and what/who decides which router to use?
There's only one DHCP server and that would "decide". Of-course any
That's deciding what IP addresses are given to systems whenm they start up, it's not deciding where to send packets.
device on the network could be configured to use one of the gateways directly but that's a "feature" rather than a problem in a small network (I accept that there are plenty of environments where you'd want to lock that down though). Even so, managed switches could control that rather than needing separate physical interfaces in the router.
So there still needs to be some sort of load balancing/decision making in the router which is specified as the gateway for all the connected systems.
I can see no advantage over:-
192.168.1.1 - 'master' router, is the gateway and does DHCP 192.168.1.2 - router A, fixed IP, no DHCP 192.168.1.3 - router B, fixed IP, no DHCP 192.168.1.3 - router C, fixed IP, no DHCP ... ... 192.168.1.x - system(s) assigned IP by master router ...
The routing/load balancing algorithm in the master router sends packets out via its own external interface or off to one of the other
So there still needs to be some sort of load balancing/decision making in the router which is specified as the gateway for all the connected systems.
I can see no advantage over:-
192.168.1.1 - 'master' router, is the gateway and does DHCP 192.168.1.2 - router A, fixed IP, no DHCP 192.168.1.3 - router B, fixed IP, no DHCP 192.168.1.3 - router C, fixed IP, no DHCP ... ... 192.168.1.x - system(s) assigned IP by master router ...
The routing/load balancing algorithm in the master router sends packets out via its own external interface or off to one of the other routers.
On 29/04/15 10:13, Mark Rogers wrote:
In our office we have relatively poor ADSL connectivity (4-6 Mbps - I know some people here have far worse though). We currently have two ADSL lines linked together via a Draytek router and do OK. However I'd like to think about adding a third line and that goes beyond what the Draytek can do.
So I'd like suggestions for Linux distros which could replace the Draytek and do some half decent load balancing (and possibly caching for eg Linux and Windows updates etc) across multiple connections. It would need to be able to handle a number of VPN connections too (PPTP unfortunately but that's determined by the other end points).
I'm not interested in rolling my own - it needs to "just work" in the way that the Draytek does.
ClearOS and pfSense are on my radar although I have no experience of either. Any other suggestions or advice?
I was going to suggest you look at the Linux Router Project, but it's defunct. However, in my travels I found this:
http://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
The must be at least one of those that can help!
Cheers, Laurie.
On 2015-04-30 13:24, Laurie Brown wrote:
On 29/04/15 10:13, Mark Rogers wrote:
In our office we have relatively poor ADSL connectivity (4-6 Mbps - I know some people here have far worse though). We currently have two ADSL lines linked together via a Draytek router and do OK. However I'd like to think about adding a third line and that goes beyond what the Draytek can do.
So I'd like suggestions for Linux distros which could replace the Draytek and do some half decent load balancing (and possibly caching for eg Linux and Windows updates etc) across multiple connections. It would need to be able to handle a number of VPN connections too (PPTP unfortunately but that's determined by the other end points).
I'm not interested in rolling my own - it needs to "just work" in the way that the Draytek does.
ClearOS and pfSense are on my radar although I have no experience of either. Any other suggestions or advice?
I was going to suggest you look at the Linux Router Project, but it's defunct. However, in my travels I found this:
http://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
The must be at least one of those that can help!
Cheers, Laurie.
I've used pfSense a lot, and it has documentation on multiple-wan and wan load balancing [1] and [2].
Alternatively, I've played a little with vyos [3] - its much closer to a router solution (linux-based but aiming to be compatible with Vyatta which is a commercial router OS), though haven't deployed it in anger anywhere yet - might be worth a look for a comparison. My experience so far, however, has been that the documentation is lagging behind the implementation with VyOs.
[1] https://doc.pfsense.org/index.php/Multi-WAN [2] https://forum.pfsense.org/index.php?topic=28121.0 [3] http://vyos.net
Hope this helps,
Jim
On 30 April 2015 at 09:37, Chris Green cl@isbd.net wrote:
However, at this point I fail to see what difference it makes compared with simply having all the devices on the same subnet. You just have the added complication of having to setup the IP aliases on the single network card.
I'm sure we're talking at cross purposes here.
The "standard" option for making use of multiple WAN connections, as implemented in (eg) a Draytek router, is that (a) each WAN connection is a distinct physical port (on my Draytek that's an ADSL port, a standard network port and a USB port for a 3G dongle), (b) each interface is on a different subnet, (c) the LAN is connected to separate LAN port(s), utilising the router as the default gateway, (d) the router routes traffic out of whichever WAN port it decides is most appropriate based on load, failover, any specific defined rules, etc.
The only question I'm trying to answer here is why doea this setup require different physical ports on separate networks, rather than virtual ports on the same network.
What advantages would using the same physical network confer?
Reason why this might be useful (and why I was thinking about it): Simpler hardware for the "box" (just one physical interface required so something like a Pi could handle it). Simpler cabling (the ADSL routers don't all need to be in one location and "behind" the "box"). It means that the number of WAN's a router can support is not limited by having to have a separate physical port for each one.
Yes, but, as I say above, what have you gained?
Er, what I just listed? If I want to set up a router using (eg) a Pi, then requiring a physical interface per network makes it a non-starter. Much simpler cabling is a bonus (although not critical).
So there still needs to be some sort of load balancing/decision making in the router which is specified as the gateway for all the connected systems.
Absolutely, that's what the router is for!
I can see no advantage over:-
192.168.1.1 - 'master' router, is the gateway and does DHCP 192.168.1.2 - router A, fixed IP, no DHCP 192.168.1.3 - router B, fixed IP, no DHCP 192.168.1.3 - router C, fixed IP, no DHCP ... ... 192.168.1.x - system(s) assigned IP by master router ...The routing/load balancing algorithm in the master router sends packets out via its own external interface or off to one of the other
Agreed, that was the secondary part of my question (whether all WAN connections needed to be on different subnets). But either way the point is: can any of this work in reality? There is surely a reason why routers expect to have a separate physical port per WAN, when a relatively simple mod would allow multiple WANs to be supported on a single physical interface if the above is true. But if there is such a reason, what is it?
On 30 April 2015 at 14:28, Jim Rippon jim@rippon.me.uk wrote:
I've used pfSense a lot, and it has documentation on multiple-wan and wan load balancing [1] and [2].
Thanks for the "vote" in its favour! I'll look at it a bit harder. (I did install pfSense in a virtual machine to experiment but got a bit lost.)
Alternatively, I've played a little with vyos [3] - its much closer to a router solution
vyos looks like it might be good too, thanks.
On Thu, Apr 30, 2015 at 03:11:53PM +0100, Mark Rogers wrote:
On 30 April 2015 at 09:37, Chris Green cl@isbd.net wrote:
However, at this point I fail to see what difference it makes compared with simply having all the devices on the same subnet. You just have the added complication of having to setup the IP aliases on the single network card.
I'm sure we're talking at cross purposes here.
The "standard" option for making use of multiple WAN connections, as implemented in (eg) a Draytek router, is that (a) each WAN connection is a distinct physical port (on my Draytek that's an ADSL port, a standard network port and a USB port for a 3G dongle), (b) each interface is on a different subnet, (c) the LAN is connected to separate LAN port(s), utilising the router as the default gateway, (d) the router routes traffic out of whichever WAN port it decides is most appropriate based on load, failover, any specific defined rules, etc.
The only question I'm trying to answer here is why doea this setup require different physical ports on separate networks, rather than virtual ports on the same network.
I don't think it does in general does it? The typical domestic ADSL router doesn't have a real WAN ethernet port at all, it just goes to the ADSL 'modem' bit. Big, expensive, commercial routers (that are just routers) have separate ports because they are trying to maximise throughput. The Draytek is a sort of half-way house.
I can see no advantage over:-
192.168.1.1 - 'master' router, is the gateway and does DHCP 192.168.1.2 - router A, fixed IP, no DHCP 192.168.1.3 - router B, fixed IP, no DHCP 192.168.1.3 - router C, fixed IP, no DHCP ... ... 192.168.1.x - system(s) assigned IP by master router ...The routing/load balancing algorithm in the master router sends packets out via its own external interface or off to one of the other
Agreed, that was the secondary part of my question (whether all WAN connections needed to be on different subnets). But either way the point is: can any of this work in reality? There is surely a reason why routers expect to have a separate physical port per WAN, when a relatively simple mod would allow multiple WANs to be supported on a single physical interface if the above is true. But if there is such a reason, what is it?
The above is perfectly normal isn't it? It's not a 'mod' allowing multiple WANs, it's simply a (fairly complicated) routing table in 192.168.1.1 which sends packets out to different external connections.
Essentially you *do* have separate physical interfaces for each WAN, it's the outward facing ADSL (or 3G, or whatever) port on routers A, B, C etc.
So instead of separate ethernet ports on the master router you have separate routers.
On 30 April 2015 at 15:51, Chris Green cl@isbd.net wrote:
The above is perfectly normal isn't it? It's not a 'mod' allowing multiple WANs, it's simply a (fairly complicated) routing table in 192.168.1.1 which sends packets out to different external connections.
OK, but if so then why do all the router distros assume you have a separate physical network interface for each WAN? Adding in multiple gateways without having them all on different ports should be no more complicated than having them on separate ports (the hard bit is in the load balanced routing, which applies in any case).
At the moment I'm experimenting with virtual machines with multiple virtual network adapters all bridged to the host's single ethernet adapter, but I haven't pushed this to the point of completion yet (doing that would break the rest of the network while I got things working).
On Thu, Apr 30, 2015 at 05:15:44PM +0100, Mark Rogers wrote:
On 30 April 2015 at 15:51, Chris Green cl@isbd.net wrote:
The above is perfectly normal isn't it? It's not a 'mod' allowing multiple WANs, it's simply a (fairly complicated) routing table in 192.168.1.1 which sends packets out to different external connections.
OK, but if so then why do all the router distros assume you have a separate physical network interface for each WAN? Adding in multiple gateways without having them all on different ports should be no more complicated than having them on separate ports (the hard bit is in the load balanced routing, which applies in any case).
Probably on the grounds that it's more secure and a lot of the distros are trying to be decent firewalls out of the box. If you're mixing and matching outbounds on the same interface then one mistake in your routing table and things can be going the wrong way. Or there's the potential for cleverly crafted packets to goes places they shouldn't.
J.
On 30 April 2015 at 17:22, Jonathan McDowell noodles@earth.li wrote:
Probably on the grounds that it's more secure and a lot of the distros are trying to be decent firewalls out of the box. If you're mixing and matching outbounds on the same interface then one mistake in your routing table and things can be going the wrong way. Or there's the potential for cleverly crafted packets to goes places they shouldn't.
If that's all it is that's fine (in which case I understand, which was my main objective!). I would say however that even then separate interfaces for red and green makes sense, but you still shouldn't need multiple red interfaces just to talk to multiple routers on the red side; in many (the majority of?) cases all the red (WAN) interfaces are equivalent from a security POV.
Once I've picked a router distro I'll probably ask the same question on their mailing list and see what they say.
-
Chris Green -
Jim Rippon -
Jonathan McDowell -
Laurie Brown -
Mark Rogers -
mick