My Pi is on my local network (headless and somewhat physically inaccessible) at 192.168.1.200
I have a 3G USB stick connected to my Pi, which presents itself as an ethernet device (eth1); the Pi gets an IP address of 192.168.8.100 and the USB stick has a configuration website on 192.168.8.1
How can I access this website from my laptop (192.168.1.100) via SSH?
I tried SSH tunnelling but it's not working, at least in part because the web page links to resources at http://192.168.8.1 which aren't accessible. I have tried w3m on the Pi itself, but the web page has loads of javascript so doesn't work.
Presumably I can set up a proxy somehow? Other suggestions?
On Wed, Oct 20, 2021 at 09:15:10AM +0100, Mark Rogers wrote:
My Pi is on my local network (headless and somewhat physically inaccessible) at 192.168.1.200
I have a 3G USB stick connected to my Pi, which presents itself as an ethernet device (eth1); the Pi gets an IP address of 192.168.8.100 and the USB stick has a configuration website on 192.168.8.1
How can I access this website from my laptop (192.168.1.100) via SSH?
I tried SSH tunnelling but it's not working, at least in part because the web page links to resources at http://192.168.8.1 which aren't accessible. I have tried w3m on the Pi itself, but the web page has loads of javascript so doesn't work.
Presumably I can set up a proxy somehow? Other suggestions?
Is this all on the same physical LAN even though they're different subnets?
If it's the same physical LAN then surely all you need to do is to set up some sort of routing (i.e. use 'route') to allow you simply to tell your browser to go to 192.168.8.1 from 192.168.1.100.
It should basically say for any address in the 192.168.8.x range route requests to 192.168.1.200. You *might* also need soemthing on the Pi but I suspect that will already have a rule saying where 192.168.8.x requests should go.
On Wed, 20 Oct 2021 at 09:31, Chris Green cl@isbd.net wrote:
Is this all on the same physical LAN even though they're different subnets?
No.
The 192.168.8.x subnet only exists between the USB dongle and the Pi, it is "created by" the USB dongle.
However...
If it's the same physical LAN then surely all you need to do is to set up some sort of routing (i.e. use 'route') to allow you simply to tell your browser to go to 192.168.8.1 from 192.168.1.100.
... routing might actually be a solution here - effectively turning the Pi into a router for the 192.168.8.0/24 network. But I don't think it would work, as that would allow me to get traffic to the web server on the dongle, but wouldn't route responses back to me unless I set the routing up on the dongle (which I probably can't do, and if I can it'll be via that web interface!)
So I need the web server on the dongle to see any connection as being from the Pi, but it needs to actually come from my laptop. (I suppose there are other options, like having Firefox run on the Pi and I access that remotely instead, eg via VNC or X) but that's quite a lot to set up just to access one web page!
Mark
On Wed, Oct 20, 2021 at 09:51:52AM +0100, Mark Rogers wrote:
On Wed, 20 Oct 2021 at 09:31, Chris Green cl@isbd.net wrote:
Is this all on the same physical LAN even though they're different subnets?
No.
The 192.168.8.x subnet only exists between the USB dongle and the Pi, it is "created by" the USB dongle.
Yes, but it's *accessible* from the 192.168.1.x subnet via the pi, as you realise below...
However...
If it's the same physical LAN then surely all you need to do is to set up some sort of routing (i.e. use 'route') to allow you simply to tell your browser to go to 192.168.8.1 from 192.168.1.100.
... routing might actually be a solution here - effectively turning the Pi into a router for the 192.168.8.0/24 network. But I don't think it would work, as that would allow me to get traffic to the web server on the dongle, but wouldn't route responses back to me unless I set the routing up on the dongle (which I probably can't do, and if I can it'll be via that web interface!)
Doesn't the pi 'know' about the 192.168.8.0/24 anyway? Try a 'route' on the pi and see what it tells you.
So I need the web server on the dongle to see any connection as being from the Pi, but it needs to actually come from my laptop. (I suppose there are other options, like having Firefox run on the Pi and I access that remotely instead, eg via VNC or X) but that's quite a lot to set up just to access one web page!
On Wed, 20 Oct 2021 at 10:22, Chris Green cl@isbd.net wrote:
Doesn't the pi 'know' about the 192.168.8.0/24 anyway? Try a 'route' on the pi and see what it tells you.
I'm not sure how a route on the Pi would help here?
I can add a route on my laptop to send traffic for 192.168.8.0/24 to 192.168.1.200 (the Pi). If I enable forwarding on the Pi that should let me send requests from my laptop to the USB dongle's web server.
But the replies to these requests would need the USB dongle to have a route telling it how to reach 192.168.1.100, ie it would also need a route adding (but this time for 192.168.1.0/24). Which as far as I know the dongle can't do, but if it could I'd need to be able to reach it over the network to set it up and thus be in a chicken/egg scenario.
I did however manage to set up a web proxy (apt get privoxy, and edit /etc/privoxy/config to set listen-address to :8118 as the default only listens on localhost). Changing my laptop to use the proxy gave me access to the web pages (albeit sending all my web traffic through the Pi until I turned the proxy off). So a messy solution but one which worked.
(I also tried browsh which uses a headless Firefox to render the pages, captures the output, and presents it in text form over SSH. Very clever, but painfully slow to the point that I had to give up with it.)
Mark
On 20/10/2021 09:51, Mark Rogers wrote:
On Wed, 20 Oct 2021 at 09:31, Chris Green cl@isbd.net wrote:
Is this all on the same physical LAN even though they're different subnets?
No.
The 192.168.8.x subnet only exists between the USB dongle and the Pi, it is "created by" the USB dongle.
However...
If it's the same physical LAN then surely all you need to do is to set up some sort of routing (i.e. use 'route') to allow you simply to tell your browser to go to 192.168.8.1 from 192.168.1.100.
... routing might actually be a solution here - effectively turning the Pi into a router for the 192.168.8.0/24 network. But I don't think it would work, as that would allow me to get traffic to the web server on the dongle, but wouldn't route responses back to me unless I set the routing up on the dongle (which I probably can't do, and if I can it'll be via that web interface!)
So I need the web server on the dongle to see any connection as being from the Pi, but it needs to actually come from my laptop. (I suppose there are other options, like having Firefox run on the Pi and I access that remotely instead, eg via VNC or X) but that's quite a lot to set up just to access one web page!
Presuming the Pi runs a desktop, despite being headless....
Setup X forwarding on ssh (not too hard)
ssh into the pi
ssh -X user@pi
in the ssh session, start firefox (or another browser)
and the firefox session display on the machine you launched ssh on.
Hope that helps
Steve
On Fri, 22 Oct 2021 at 22:18, steve-ALUG@hst.me.uk wrote:
Presuming the Pi runs a desktop, despite being headless....
It runs as a kiosk, so it does have X but no desktop as such, just a web browser in fullscreen mode.
But X is the key so this should work anyway, and initial testing suggests it will so I'll play further.
Thanks!
On 20/10/2021 09:15, Mark Rogers wrote:
My Pi is on my local network (headless and somewhat physically inaccessible) at 192.168.1.200
I have a 3G USB stick connected to my Pi, which presents itself as an ethernet device (eth1); the Pi gets an IP address of 192.168.8.100 and the USB stick has a configuration website on 192.168.8.1
How can I access this website from my laptop (192.168.1.100) via SSH?
I tried SSH tunnelling but it's not working, at least in part because the web page links to resources athttp://192.168.8.1 which aren't accessible. I have tried w3m on the Pi itself, but the web page has loads of javascript so doesn't work.
Presumably I can set up a proxy somehow? Other suggestions?
I do this for pretty much exactly the same reason - accessing the web interface of a router via a name can often cause all sorts of problems for some reason. Anyway, the solution I have used takes me back to the 90s in many ways, as that's when I used the technique most. What you need is a Socks proxy and a browser that supports them (Firefox does, but to be honest I've not even looked at other browser support). You can use a standard ssh install to create the Socks proxy, so no need for extra software installs. The incantation is something along the lines of:
ssh -i ~/.ssh/id_rsa -D 1080 -f -C -q -Nuser@hostname.tld
Where:
-i is defining the ssh key, so no need to provide that if you are using password authentication or have that configured in the ~/.ssh/config file
-D sets the Socks port
-f backgrounds ssh
-C sets compression
-q sets quiet mod
-N stops execution of a remote command
Once that is running (sorry, can't double check things, I'm booted into that other OS with the 11 after it at the moment, a bit of an 'is it up to being a daily driver' challenge; painful so far), you can configure Firefox (or other supporting browser) to use the Socks proxy.
For Firefox just go to the bottom section of the Settings where you have Network Settings and a button to configure things. Go for the Manual proxy configuration and the Socks Host and port, so local host and 1080, Socks 5 is fine. There's probably an extra parameter on the SSH command if you are using a separate machine to run the Socks proxy and not your local one.
With any luck I've not made a blunder in those details, it's later than I thought!
On Fri, Oct 22, 2021 at 12:55:48AM +0100, Paul Tansom wrote:
On 20/10/2021 09:15, Mark Rogers wrote:
My Pi is on my local network (headless and somewhat physically inaccessible) at 192.168.1.200
I have a 3G USB stick connected to my Pi, which presents itself as an ethernet device (eth1); the Pi gets an IP address of 192.168.8.100 and the USB stick has a configuration website on 192.168.8.1
How can I access this website from my laptop (192.168.1.100) via SSH?
I tried SSH tunnelling but it's not working, at least in part because the web page links to resources at [1]http://192.168.8.1 which aren't accessible. I have tried w3m on the Pi itself, but the web page has loads of javascript so doesn't work.
Presumably I can set up a proxy somehow? Other suggestions?
You could run X through the ssh connection:-
ssh -X 192.168.1.200
... and then run a GUI browser on the PI which will display 'locally' on the system you have ssh'ed from. It'll be quite slow but across a fast LAN should be acceptable.
On Fri, 22 Oct 2021 at 11:23, Chris Green cl@isbd.net wrote:
You could run X through the ssh connection:-
ssh -X 192.168.1.200... and then run a GUI browser on the PI which will display 'locally' on the system you have ssh'ed from. It'll be quite slow but across a fast LAN should be acceptable.
For some reason I've never had much luck with this in the past but I just tried it again and it's certainly a viable option. I eve got it working on Windows (sadly my work laptop runs it most of the time) using a combination of PuTTY and VcXsrv.
Yes it is pretty slow, but for what I need to do (and given how rare it needs doing) I think that'll be enough.
On Fri, Oct 29, 2021 at 06:18:10PM +0100, Mark Rogers wrote:
On Fri, 22 Oct 2021 at 11:23, Chris Green cl@isbd.net wrote:
You could run X through the ssh connection:-
ssh -X 192.168.1.200... and then run a GUI browser on the PI which will display 'locally' on the system you have ssh'ed from. It'll be quite slow but across a fast LAN should be acceptable.
For some reason I've never had much luck with this in the past but I just tried it again and it's certainly a viable option. I eve got it working on Windows (sadly my work laptop runs it most of the time) using a combination of PuTTY and VcXsrv.
Yes it is pretty slow, but for what I need to do (and given how rare it needs doing) I think that'll be enough.
If the slowness becomes a nuisance then there are various other ways of doing the same thing that will speed it up - VNC and variants basically.
I use x2go which, across a home LAN, makes things work pretty quickly and transparently.
On Fri, 22 Oct 2021 at 09:34, Paul Tansom paul@aptanet.com wrote:
You can use a standard ssh install to create the Socks proxy, so no need for extra software installs. The incantation is something along the lines of:
ssh -i ~/.ssh/id_rsa -D 1080 -f -C -q -N user@hostname.tld
Where: [...]
Ah, that's pretty much perfect, thank you!
For Firefox just go to the bottom section of the Settings where you have Network Settings and a button to configure things.
Are you aware of a way to enable/require the proxy only for certain addresses? I have no need/desire to send all web traffic via my Pi.
On 29/10/2021 17:39, Mark Rogers wrote:
Are you aware of a way to enable/require the proxy only for certain addresses? I have no need/desire to send all web traffic via my Pi.
One way is via a Proxy Auto Configuration script (PAC).
This link below shows how to use it to switch on/off proxy depending if you are in your home network or not, but I'm fairly sure you could adapt the principles easily to route only certain traffic to a pi - you'd probably just need to work out some sort of "if" function to return true if the route should go to the pi. Look at the listed links for further info.
http://no-sheds.blogspot.com/2020/01/proxy-auto-configuration-script.html
Hope that helps.
Steve
On 29/10/2021 17:39, Mark Rogers wrote:
On Fri, 22 Oct 2021 at 09:34, Paul Tansom paul@aptanet.com wrote:
You can use a standard ssh install to create the Socks proxy, so no need for extra software installs. The incantation is something along the lines of:
ssh -i ~/.ssh/id_rsa -D 1080 -f -C -q -N user@hostname.tld
Where: [...]
Ah, that's pretty much perfect, thank you!
For Firefox just go to the bottom section of the Settings where you have Network Settings and a button to configure things.
Are you aware of a way to enable/require the proxy only for certain addresses? I have no need/desire to send all web traffic via my Pi.
Slow reply as life has become a tad complicated over the past couple of weeks and I'm juggling family, work and other commitments, whilst not actually being at home that much. That has, in fact, meant that I've just come across the same issue, but not actually found a solution yet. I'm wondering whether there's some sort of (regular?) expression that can be used in the area to set sites you don't want to use the proxy for that gets across "any site that isn't in this IP address range" or something like that.
I've had the added issue of having to mess with my hosts file, which I can't remember having done for years, to make up for the fact that I'm not on my local network to access my local Bind DNS server. What I really need is my Wireguard VPN back up and running, but that's another story! [1]
One possibly, but rather clunky, option would be to use different web browsers, and if you want to use the same one (working on Firefox as my example) either use profiles or alternative installs. Firefox profiles requires a close and restart of the browser (so possibly no easier than flipping to the settings and enabling/disabling the proxy; which thankfully remembers the values for the disabled option), but I'm not sure if the profiles in other browsers can handle different proxy setups (you would hope), or a Socks proxy. If you're on Ubuntu, or other Snap supporting distro you can install the deb and Snap packages at the same time, or try an Appimage (not sure how up to date it is kept) or Flatpak (not tried, but would require a few extra bits to get it running).
If I manage to come up with a solution I'll post another reply.
[1] Long story short, I've switched ISPs [2] and my old Wireguard setup on my OpenWrt router has gone and I'm working on getting a new one up and running in Docker, but not quite managinge (not helped by initially also wanting to deploy it with Ansible). I could put it on my EdgeRouter, but it doesn't integrate as smoothly as it should and will be a bit of a cludge.
[2] Accepting that there are people worse off for speed, my FTTC connection peaks at 16Mbps, which is less than a friends ADSL used to peak at (17Mbps, all be it with the cabinet a couple of feet from their garden). As such I've been forced to Virgin for a cable connection, which has forced me to temporarily (hopefully) abandon IPv6; and move from native support to a tunnel - ho hum.
On Wed, Nov 03, 2021 at 03:31:54PM +0000, Paul Tansom wrote:
One possibly, but rather clunky, option would be to use different web browsers, and if you want to use the same one (working on Firefox as my example) either use profiles or alternative installs. Firefox profiles requires a close and restart of the browser (so possibly no easier than flipping to the settings and enabling/disabling the proxy; which thankfully remembers the values for the disabled option), but I'm not sure if the profiles in other browsers can handle different proxy setups (you would hope), or a Socks proxy. If you're on Ubuntu, or other Snap supporting distro you can install the deb and Snap packages at the same time, or try an Appimage (not sure how up to date it is kept) or Flatpak (not tried, but would require a few extra bits to get it running).
If you run Firefox from the command line you can specify the profile to be used, 'man firefox' tells you how.
firefox -P <name of profile>
You also need -no-remote to provent it just latching on to an already running Firefox (see the man page again).
On 03/11/2021 15:31, Paul Tansom wrote:
On 29/10/2021 17:39, Mark Rogers wrote:
Are you aware of a way to enable/require the proxy only for certain addresses? I have no need/desire to send all web traffic via my Pi.
Slow reply as life has become a tad complicated over the past couple of weeks and I'm juggling family, work and other commitments, whilst not actually being at home that much. That has, in fact, meant that I've just come across the same issue, but not actually found a solution yet. I'm wondering whether there's some sort of (regular?) expression that can be used in the area to set sites you don't want to use the proxy for that gets across "any site that isn't in this IP address range" or something like that.
My answer to a previous post should help: One way is via a Proxy Auto Configuration script (PAC).
This link below shows how to use it to switch on/off proxy depending if you are in your home network or not, but I'm fairly sure you could adapt the principles easily to route only certain traffic to a pi - you'd probably just need to work out some sort of "if" function to return true if the route should go to the pi. Look at the listed links for further info.
http://no-sheds.blogspot.com/2020/01/proxy-auto-configuration-script.html
You can use a PAC file on windows to have a system wide auto-changing proxy. I think, but am not sure, that you can do the same on Linux. You can also specify it in the web-browser.
I've had the added issue of having to mess with my hosts file, which I can't remember having done for years, to make up for the fact that I'm not on my local network to access my local Bind DNS server.
I did this initially to have an add-blocking proxy whilst at home, but when away from home, automatically switching the proxy off (because it wasn't accessible). You could use the same techniques as that website uses to achieve this.
Hope that helps
Steve
On 03/11/2021 15:31, Paul Tansom wrote:
On 29/10/2021 17:39, Mark Rogers wrote:
On Fri, 22 Oct 2021 at 09:34, Paul Tansom paul@aptanet.com wrote:
You can use a standard ssh install to create the Socks proxy, so no need for extra software installs. The incantation is something along the lines of:
ssh -i ~/.ssh/id_rsa -D 1080 -f -C -q -N user@hostname.tld
Where: [...]
Ah, that's pretty much perfect, thank you!
For Firefox just go to the bottom section of the Settings where you have Network Settings and a button to configure things.
Are you aware of a way to enable/require the proxy only for certain addresses? I have no need/desire to send all web traffic via my Pi.
Slow reply as life has become a tad complicated over the past couple of weeks and I'm juggling family, work and other commitments, whilst not actually being at home that much. That has, in fact, meant that I've just come across the same issue, but not actually found a solution yet. I'm wondering whether there's some sort of (regular?) expression that can be used in the area to set sites you don't want to use the proxy for that gets across "any site that isn't in this IP address range" or something like that.
I've had the added issue of having to mess with my hosts file, which I can't remember having done for years, to make up for the fact that I'm not on my local network to access my local Bind DNS server. What I really need is my Wireguard VPN back up and running, but that's another story! [1]
One possibly, but rather clunky, option would be to use different web browsers, and if you want to use the same one (working on Firefox as my example) either use profiles or alternative installs. Firefox profiles requires a close and restart of the browser (so possibly no easier than flipping to the settings and enabling/disabling the proxy; which thankfully remembers the values for the disabled option), but I'm not sure if the profiles in other browsers can handle different proxy setups (you would hope), or a Socks proxy. If you're on Ubuntu, or other Snap supporting distro you can install the deb and Snap packages at the same time, or try an Appimage (not sure how up to date it is kept) or Flatpak (not tried, but would require a few extra bits to get it running).
If I manage to come up with a solution I'll post another reply.
... and here it is :-)
There's a Firefox plug-in called FoxyProxy which does exactly what is needed, and seems to have been around and is flagged as recommended as well. Once installed configure a proxy as follows:
Add a new one, with a suitable description, remove the whitelist pattern for all URLs and configure your proxy, so as I run my Socks on port 9080 (to avoid needing to use sudo or root) I have a SOCKS5 proxy type, using localhost and port 9080 (with no user credentials). Save and edit patterns.
The patterns should be empty, so add a white pattern with a suitable name with the pattern matching your IP address range (e.g. 192.168.0.[1-254]) and ensure the type is Reg Exp. I've stuck with all for the HTTP/s bit and obviously it needs to be enabled. I've also got one for accessing my my local DNS (since I have DNS heading across the proxy too), which is a wildcard along the lines of *.home.tld.uk in there (I also have a couple of IPs where I have restricted access to the remote port of a couple of routers to my static IP addresses so need to bounce of my own connection - these seem to work best as Reg Exp even though I'm just quoting the IP).
Once this is setup you can use the FoxyProxy icon in the top bar and set it to "Use enabled proxies by pattern and order". It indicates which proxy you are using over the icon (or no proxy). It seems to work pretty well so far. It is also quite handy for setting up other proxies and being able to switch easily between them. I should have looked before!
-
Chris Green -
Mark Rogers -
Paul Tansom -
steve-ALUG＠hst.me.uk