Topics

Node Loosing Connectivity (pi)


James McBride
 

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




David Cameron - IRLP
 

It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




James McBride
 

It is setup as static IP (no other interfaces) - but with the pi os install I did the default was to use dhcpcd5 and not the interfaces file (dhcpcd5 ignores this - if you use interfaces file also it creates multiple instances of interface) - I could just remove dhcpcd5 completely also but I've left it for now... it's another option to fix if the below doesn't work.

James VK6FJA

On 8/04/2020 12:37 am, David Cameron - IRLP wrote:
It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W





David Cameron - IRLP
 

Well, you have something going on there. If you want to use dhcpcd5, you are going to have to figure it out on your own :)

Just make sure wicd-curses is not installed. 

David Cameron 

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:48 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

It is setup as static IP (no other interfaces) - but with the pi os install I did the default was to use dhcpcd5 and not the interfaces file (dhcpcd5 ignores this - if you use interfaces file also it creates multiple instances of interface) - I could just remove dhcpcd5 completely also but I've left it for now... it's another option to fix if the below doesn't work.

James VK6FJA

On 8/04/2020 12:37 am, David Cameron - IRLP wrote:
It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W





k9dc
 

Openvpn should never lose the default route, as it does not remove anything. It adds a 32-bit single host route to the VPN server, then it adds two /1 (netmask 128.0.0.0) routes to the internet over the tunnel. Basically the longer mask (/1) overrides the default route, but does not remove it. Routes to the local network, and the default route via the local router remain in the table.

There is a push statement in the OpenVPN server configuration that makes it happen. IRLP VPN results in a routing table like this.

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 44.48.26.1 128.0.0.0 UG 0 0 0 tun0
0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 eth0
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
44.48.26.0 44.48.26.1 255.255.254.0 UG 0 0 0 tun0
44.48.26.1 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
128.0.0.0 44.48.26.1 128.0.0.0 UG 0 0 0 tun0
149.28.114.197 10.0.0.1 255.255.255.255 UGH 0 0 0 eth0

One thing the CAN happen is loss of DNS. In the case of Openvpn, if you are configured to use DNS outside your local network, it may fail when the vpn dies. So you might be better off using your local router as the Nameserver, if your router supports the feature (most do).

-k9dc

On Apr 7, 2020, at 12:34, James McBride <fjamesa@...> wrote:

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn.. but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr 7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr 7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr 7 07:52:17 kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr 7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it... myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


Klaus Rung
 

Dave, how about if you are using dhcp, do you still remove wicd?


On Tuesday, April 7, 2020, 12:38:09 p.m. EDT, David Cameron - IRLP <dcameron@...> wrote:


It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




k9dc
 

Yes. wicd-curses is not necessary to use DHCP. It is simply included on the line in /etc/network/interfaces that defines the interface

iface eth0 inet dhcp (instead of iface eth0 inet static)

-k9dc

On Apr 7, 2020, at 15:29, Klaus Rung via groups.io <k_rung=yahoo.com@groups.io> wrote:

Dave, how about if you are using dhcp, do you still remove wicd?


On Tuesday, April 7, 2020, 12:38:09 p.m. EDT, David Cameron - IRLP <dcameron@...> wrote:


It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn.. but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr 7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr 7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr 7 07:52:17 kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr 7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it... myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA



On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Klaus Rung
 

Thank you .

On Tuesday, April 7, 2020, 3:37:56 p.m. EDT, k9dc <dave@...> wrote:



Yes. wicd-curses is not necessary to use DHCP. It is simply included on the line in /etc/network/interfaces that defines the interface

iface eth0 inet dhcp  (instead of iface eth0 inet static)

-k9dc


> On Apr 7, 2020, at 15:29, Klaus Rung via groups.io <k_rung=yahoo.com@groups.io> wrote:
>
> Dave, how about if you are using dhcp, do you still remove wicd?
>
>
> On Tuesday, April 7, 2020, 12:38:09 p.m. EDT, David Cameron - IRLP <dcameron@...> wrote:
>
>
> It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.
>
> -------- Original message --------
> From: James McBride <fjamesa@...>
> Date: 4/7/20 9:34 AM (GMT-08:00)
> To: IRLP@irlp.groups.io
> Subject: Re: [IRLP] Node Loosing Connectivity (pi)
>
> Hi all,
>
> I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.
>
> By default I have been starting openvpn with /etc/default/openvpn..  but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.
>
> Here's the syslog:
>
> Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
> Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost
>
> ...
>
> Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
> Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired
>
> In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.
>
> It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.
>
> I removed /etc/default/openvpn  (moved it elsewhere) and made a new
>
> /etc/systemd/openvpn.conf file with:
>
> [Unit]
> Description=OpenVPN
> After=network.target
> [Service]
> Type=forking
> ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
> Restart=always
> RestartSec=30
> [Install]
> WantedBy=multi-user.target
>
> in it...  myClient.conf being the configuration for the client in /etc/openvpn/
>
> Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.
>
> 73
>
> James VK6FJA
>
>
>
> On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
> Hi all,
>
> I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.
>
> The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.
>
> Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?
>
> Has anyone else had this problem and what solution have they found?
>
> Klaus
> ve3kr
> node 7371 J73W
>
>
>
>




James McBride
 

Hi David,

Thanks for the feedback.

Yes I know - I'm on my own with this one (and so I should be), but no problem. Just putting it out there so others can sort out a fix if they need to or aren't in support scope.

I reinstalled the node from Debian 9.4 a while back - only just got back to trying to fix the issue, which arose when I had to install openvpn to tunnel out via CG-NAT to get a public IP (yep, this one differs, so is out of scope for normal irlp support too). Actually probably due for an OS upgrade here soon :)

I don't know the details, but recent versions of Rasbian have moved the network configuration from /etc/network/interfaces to /etc/dhcpcd.conf...  and so static IP configuration in this case has changed - I didn't realise for a while until I tried to setup a static IP on the network and ran into trouble. I was just running a static DHCP allocation so when I moved the IRLP node from my home network (testing and config) to our radio club network (it's home) I didn't have to keep editing things.

It's a strange problem that the eth0 interface does a 2 second link down - link up on odd occasions, no idea why - it's not power, seems to be a kernel or some usb/reset thing that happens on the Pi (which is an ancient model B) - could even be related to the sound device.

My issue was that openvpn didn't seem to want to restart after the eth0 glitch. Moving it's startup from init to systemd was a modernisation issue but not the fix in this case.

The problem is that reseting eth0 causes the routes to be removed from the table for eth0 (INCLUDING the VPN route) and when it returns the VPN static route does not recover (openvpn would normally add this on establishment) - this was easy to see when it failed again as openvpn left it's default route for tun0 in place (0.0.0.0/1) so the vpn itself would just timeout because there was no longer any path to get to it via eth0 anymore but technically it was still up just trying to get out via itself... ie. recursion (and likely because I'm using a non-standard config... I know).

Here's the openvpn client log after a eth0 bounce, the server log side just never sees traffic from the node:

Mon Apr  6 23:06:47 2020 us=162290 Re-using SSL/TLS context

...
Mon Apr  6 23:06:47 2020 us=165388 TCP/UDP: Preserving recently used remote address: [AF_INET]<IP address of VPN>:<port>
Mon Apr  6 23:06:47 2020 us=165909 Socket Buffers: R=[163840->163840] S=[163840->163840]
Mon Apr  6 23:06:47 2020 us=166175 UDP link local: (not bound)
Mon Apr  6 23:06:47 2020 us=166442 UDP link remote: [AF_INET]<IP address of VPN>:<port>
Mon Apr  6 23:07:47 2020 us=447645 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Mon Apr  6 23:07:47 2020 us=448019 TLS Error: TLS handshake failed
Mon Apr  6 23:07:47 2020 us=449092 TCP/UDP: Closing socket
Mon Apr  6 23:07:47 2020 us=449654 SIGUSR1[soft,tls-error] received, process restarting

Mon Apr  6 23:07:47 2020 us=161869 Restart pause, 300 second(s)

....


I've resolved that now by setting a permanent static route to the VPN endpoint via the physical default gw that openvpn would normally establish so that it's always in place.

To add a persistent static route with dhcpcd being used, create or edit the following file:
/lib/dhcpcd/dhcpcd-hooks/40-route with

ip route add <vpn endpoint>/32 via <default gw>

This way the connection can always re-establish when tun0 is up, even after bouncing eth0.

I've done it before with Cisco GRE/IPSEC but recursion didn't occur to me until looking into the IRLP/OpenVPN situation recently.

Dave k9dc was on track also with the openvpn route table print. Likely DNS or NTP will fail for the same reason if ethernet restarts and openvpn process doesn't. You would need a static route and/or host ip entry put in for a DNS that isn't via the tunnel itself so the tunnel endpoint can be resolved again for openvpn to re-establish whilst tun0 route is still up (ie. without a full process restart).

The only alternatives were to script up some sort of openvpn detection / full process restart or dtmf reboot ... always many solutions to a problem.

73

James VK6FJA

Node 6438 & 6100..

BTW - 6438 is a pi node we are using in place of the original 6100 on VK6RNC.  Radio is a FT7800R, was an Alinco DR135 but I switched them around when 6100 was pulled from service for repair, I actually want to put the Alinco back in.  6100 is on my desk (been offline for a long while - may need a manual key update when I do rebuild it's disk) and 6200 has also not been used since it failed a while back (not mine, but I believe it hasn't been brought back to life yet).



On 8/04/2020 12:57 am, David Cameron - IRLP wrote:
Well, you have something going on there. If you want to use dhcpcd5, you are going to have to figure it out on your own :)

Just make sure wicd-curses is not installed. 

David Cameron 

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:48 AM (GMT-08:00)
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

It is setup as static IP (no other interfaces) - but with the pi os install I did the default was to use dhcpcd5 and not the interfaces file (dhcpcd5 ignores this - if you use interfaces file also it creates multiple instances of interface) - I could just remove dhcpcd5 completely also but I've left it for now... it's another option to fix if the below doesn't work.

James VK6FJA

On 8/04/2020 12:37 am, David Cameron - IRLP wrote:
It sounds to me like he may have wicd and entries in your interfaces file. You can only use one. If you're using a static IP, you should remove the wicd program and set the static IP in the interfaces file.

-------- Original message --------
From: James McBride <fjamesa@...>
Date: 4/7/20 9:34 AM (GMT-08:00)
Subject: Re: [IRLP] Node Loosing Connectivity (pi)

Hi all,

I don't know if it will help but this is what I've found ona e pi node here. It's been annoying me for ages and hopefully I have a solution that might work. Perhaps others have had the same issue and have a working solution, however even I will be implementing the DTMF restart solution also as it is handy to reboot via radio when the network can't be reached.

By default I have been starting openvpn with /etc/default/openvpn..   but this failed to restart openvpn properly when the networking failed - which I have found it does randomly when the dhcpcd5 client kicks in on the pi and (even with a static IP) it causes the phyical pi ethernet to die and restart resulting in a recursive route situation (trying to resend packets to the default gateway rather than the tunnel interface default) until openvpn process can be manually restarted also.

Here's the syslog:

Apr  7 07:52:15 kernel: [141931.184700] smsc95xx 1-1.1:1.0 eth0: link down
Apr  7 07:52:15 dhcpcd[365]: eth0: carrier lost

...

Apr  7 07:52:17  kernel: [141932.744437] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Apr  7 07:52:17 dhcpcd[365]: eth0: carrier acquired

In the above case - openvpn just fails to start as the default route has been removed by the eth0 interface restart - which results in a whole lot of timeouts for the node which now can't access the network via the VPN, only the local network - so inbound ports are blocked.

It seems the solution is to not start openvpn from init, and instead start it from systemd with a restart=true and dependency on the network being up first, this should then make openvpn restart when the network restarts.

I removed /etc/default/openvpn  (moved it elsewhere) and made a new

/etc/systemd/openvpn.conf file with:

[Unit]
Description=OpenVPN
After=network.target
[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon --cd /etc/openvpn/ --config myClient.conf
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

in it...  myClient.conf being the configuration for the client in /etc/openvpn/

Seems to function better. I'm going to see if this finally stops the node from needing manual repairs every so often. Failing this I'll have to look into if I need another static route to stop the deleted tun0 default from recursive routing when the eth0 interface restarts and the gateway changes.

73

James VK6FJA


On 29/03/2020 10:19 pm, Klaus Rung via Groups.Io wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W





k9dc
 

WRT to setting up additional detection scripts, don’t forget I added a pair of VPN test addresses you can use. Either 192.168.168.1 or 172.23.168.1 will ping if and only if the VPN tunnel is working (you only need to use one of them). The addresses are the same on Chicago and Sydney.

-k9dc

On Apr 8, 2020, at 11:30, James McBride <fjamesa@...> wrote:

The only alternatives were to script up some sort of openvpn detection / full process restart or dtmf reboot ... always many solutions to a problem.
73
James VK6FJA