How can I get mytools.management to work with HAproxy

adam:ONE® pfSense
1

I have set up HAproxy on my pfSense gateway and now I don’t seem to be able to access mytools.management. My HAproxy set up is (more or less):

I did have an http → https redirect on my LAN excluding mytools.management but it did not work

I now have a front-end on my WAN port 80 which does an http→https redirect
I have a front end on all https to redirect https to my internal servers and to add my Lets Encrypt certificate to the traffic.

With this setup, if I go to http://mytools.management, I get redirected to https://gateway-a650fvtn.2my.network/ which fails with a certificate warning as the certificate is only valid for my domain. If I accept the certificate, I then get a 503 Service Unavailable.

I would like to reinstate LAN front-end with some sort of exception for mytools.management and whatever else it takes to allow me to see logs.

If I add an acl to the LAN frontend to exclude mytools.management, I get a 503 for mytools.management

If I stop HAproxy and restart anmuscle.sh so it binds to port 80, mytools.management redirects to https://gateway-a650fvtn.2my.network/ and, after a while, starts working.

I have also tried setting up a backend for anmuscle listening on LAN:853 and on 127.0.0.2:853 but this was shooting in the dark and both failed.

How can I get this all to work?

2

@NickH normally HAProxy is used when exposing services to the Internet, is there a reason you’re running it on the LAN interface? I’ve set it up before and as long as you use it on non-LAN interfaces it’s fine.

Otherwise you’ll need to either setup an alias IP address to run it on, or setup an alias IP for adam:ONE to run on.

3

It is also running on the LAN interface because it is looking after the LE certification. I can then use the same url from inside and outside my LAN, avoiding switching between http and https depending on where I am.

If I set up an alias IP, how do I get mytools.management to listen on it? Is it just a question of of editing /etc/adamone/anmuscle.conf? If so, which entry? I’d guess http-listener, but what about dot-listener?

4

You would normally keep the IPs for http-listener and dot-listener the same. And you’re right just edit the conf file and run service anmuscle.sh restart

If you edit dns-listener then your endpoints/DHCP will also need to be updated to point to the new IP for DNS.

5

This hasn’t totally worked. I now have:

lan-interfaces=bridge0
dns-listener=172.17.2.254@53,127.0.0.2@53
http-listener=172.17.2.253@80,127.0.0.2@80
dot-listener=172.17.2.253@853,127.0.0.2@853
http-fallback-interface=bridge0

But doing an nslookup, in Windoze I get:

C:\Users\nick>nslookup mytools.management.
Server: pfSense.howitts.co.uk
Address: 172.17.2.254

Non-authoritative answer:
Name: mytools.management
Addresses: 172.17.2.254
172.17.2.253

and in Linux I get:

root@hostd-test:~# nslookup mytools.management
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: mytools.management
Address: 172.17.2.254
Name: mytools.management
Address: 172.17.2.253

Somehow I need to stop adamOne from resolving to 172.17.2.254.

At the same time, I see in pfSense:

[2.8.0-RELEASE][root@pfSense.howitts.co.uk]/root: sockstat | grep anmuscle
root anmuscle 21015 21 udp4 172.17.2.254:53 :
root anmuscle 21015 22 tcp4 172.17.2.254:53 :
root anmuscle 21015 23 udp4 127.0.0.2:53 :
root anmuscle 21015 24 tcp4 127.0.0.2:53 :
root anmuscle 21015 25 tcp4 82.5.79.245:9317 34.120.84.240:1883
root anmuscle 21015 26 tcp4 172.17.2.254:853 :
root anmuscle 21015 27 tcp4 127.0.0.2:80 :
root anmuscle 21015 28 udp4 172.17.2.254:137 :
root anmuscle 21015 29 udp4 172.17.2.253:137 :
root anmuscle 21015 31 tcp4 127.0.0.2:853 :
root anmuscle 21015 32 tcp4 82.5.79.245:49153 34.120.84.240:443
root anmuscle 21015 34 udp4 *:27378 :
root anmuscle 21015 35 tcp4 172.17.2.254:443 :
root anmuscle 21015 36 tcp4 127.0.0.2:443 :

So anmuscle does not appear to be listening on 172.17.2.253:80, my alias, but the IP does respond to pings.

6

Alright I think you would need to move the dns-listener as well then unfortunately. I’ll tag @tom here in case he has any other ideas.

7

I have a feeling that would be bad news as it would move the DNS server to to .253? Some of that could be handled by updating the DHCP server, but not the static hosts.

I still don’t see it would work as I have nothing listening on .253:80. At my end, I could try setting the hosts file on my PC to override mytools.management and gateway-a650fvtn.2my.networkgateway-a650fvtn.2my.network to point to .253, but, if nothing is listening there, I can’t see it will help.

BTW another use case for HAProxy on your LAN is if you have multiple web servers running on different ports on the same machine. You can then use HAProxy to map the external URL to the internal server:port.

8

I have found a workaround.

Remove the split DNS for the web server FQDNs, so they all resolve to the external IP. You can then bind HAProxy only to the external IP for both HTTP→HTTPS and the HTTPS rules, including adding the SSL cert. This means HAProxy is not listening on the LAN interface at all allowing mytools.management and https://gateway-a650fvtn.2my.network/log/traffic to work.

You shouldn’t have to do this. Really there should be a neat way to allow the proxy to bind to the pfSense LAN interface.

9

If I remember correctly, when I was using HAProxy, I had a virtual IP setup on lan interface with an authoritative entry setup on adam dashboard that pointed the domain to virtual IP. HAProxy was listening on the virtual IP.

I have moved over to GitHub - fosrl/pangolin: Identity-Aware Tunneled Reverse Proxy Server with Dashboard UI . Would highly recommend looking into it. You can run it on VM behind firewall for local auth + certs. I then use cloudflare zero trust tunnel that points directly to the pangolin setup.

1 Like
10

How did you get anmuscle to bind to your Virtual IP? Also, how did you set up the authoritative entry in the dashboard?

11

There is no need to bind anmuscle to virtual ip.

Under my rules, on dashboard.adamnet.works, create a new list. From drop down, select authoritative entry.

Name can be anything, authoritative ip is the virtual ip and in domains you can add “domain.com" and tick include subdomains.

12

I am afraid I am feeling stupid. Do you mean all subdomains of my domain are served from your virtual IP and you get HAProxy to bind to the virtual IP. Then adamOne can respond on the principle LAN IP? Or have I missed something?

13

Sounds about right.

HAProxy listens on the virtual IP for the ports 80 and 443.

When you access “sub.domain.com", it will resolve to the virtual IP(via authoritative entry).

mytools.management will work correctly on lan ip.

Hope it makes sense.

14

@atw Perhaps there should be a howto around this. I think you can use the pfSense hosts file instead of an authoritative list but the end result is the same. I would guess the solution would work with any reverse proxy running on the router on any of the O/Ss you support.

15

@David From Technical Features of adam:ONE (v4) anmuscle , could I have used either –brain-address or –cerebrum-address to move the IP that mytools.management listens on?

Can we have a fuller explanation of how these parameters work and what their defaults are? What are the brain and cerebum services?