Topics

Node Loosing Connectivity


Klaus Rung
 

Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Herman Hoffman
 

Modems and router settings must be scrutinized.  Despite using a VPN connection.  Once that has been done involve the ISP and do not let up until they provide the information regarding changes they may have made.  They love to fingerprint, Especially at people who know what they are doing.  Get grumpy and threatening if necessary, because it is.  Document your results, you will need that information to get results and persevere.


On Sun, Mar 29, 2020 at 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




John Lorenc
 

The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Klaus Rung
 

Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




David Cameron - IRLP
 

I would determine what the problem is. You can run openvpn in debug mode. You can also run in pings to something on the outside, something on the inside, and the router. Figure out what the failure is.

Next time, try a restart of the VPN by running

/etc/init.d/openvpn restart

I have not looked, but there likely already scripts to do this as openvpn is very common. 

Dave Cameron 

-------- Original message --------
From: "Klaus Rung via Groups.Io" <k_rung@...>
Date: 3/29/20 7:55 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity

Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Klaus Rung
 

Thanks Dave for the reply.

The node is in Dominica so I only have ssh access from here.

I will try your suggestions and anyone else's that can come up with a solution. A node cannot function behind a network like that as there is no way to port forward unless it is on a vpn but even that seems to be breaking. I suspect something is going to sleep and loosing the path back in in the providers network and it needs to be refreshed before it goes away. Maybe they have a timer to drop connections after a while. Talking to the provider would be fruitless there. This is not the only node with this issue. I have another in St. Martin doing the same thing and is behind the same type of network. Once rebooted all is fine for a while. We have found that you can ping the local router ip just fine so there is nothing wrong with the node itself or the connection to the local router. The problem lies with connection to the web.

Does openvpn not keep pinging something on the outside to keep the connection active? I don't have enough knowledge on this stuff.

Klaus
ve3kr

On Sunday, March 29, 2020, 11:19:14 a.m. EDT, David Cameron - IRLP <dcameron@...> wrote:


I would determine what the problem is. You can run openvpn in debug mode. You can also run in pings to something on the outside, something on the inside, and the router. Figure out what the failure is.

Next time, try a restart of the VPN by running

/etc/init.d/openvpn restart

I have not looked, but there likely already scripts to do this as openvpn is very common. 

Dave Cameron 

-------- Original message --------
From: "Klaus Rung via Groups.Io" <k_rung@...>
Date: 3/29/20 7:55 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity

Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




John Lorenc
 


Klaus, found an example of editing cron at https://www.qsl.net/w2ymm/howto.html.

Disable and Enable with custom_cron

Question ?

Can I have cron do a shut down and restart echolink over night ?

Answer..
Adding the following two lines to custom.crons should do it:

0 23 * * * (/home/EchoIRLP/ scripts/echo_ disable >/dev/null 2>&1)
0 06 * * * (/home/EchoIRLP/ scripts/echo_ enable >/dev/null 2>&1)

The 23 above will disable it at 11 pm and the 06 will re-enable it at 6 am.

Google will help you with figuring out the asterisks to do the reboot when you like. Change /home/EchoIRLP/scripts/echo_enable to /usr/bin/sudo reboot and add the line to custom.crons.

After custom.cons is edited, you need to do an

update files

Note this is a band aid to at least allow you some level of ssh access while you are figuring out root cause.

John va3wm
IRLP 2248 2931



On Mar 29, 2020, at 10:55 AM, Klaus Rung via Groups.Io <k_rung@...> wrote:


Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Klaus Rung
 

Ok, thanks, that would fix the problem in the short term sort of but it seems the node is loosing connectivity about every 4 hrs so a cron probably will not work with this.

On Sunday, March 29, 2020, 11:43:15 a.m. EDT, John Lorenc <va3xjl@...> wrote:



Klaus, found an example of editing cron at https://www.qsl.net/w2ymm/howto.html.

Disable and Enable with custom_cron

Question ?

Can I have cron do a shut down and restart echolink over night ?

Answer..
Adding the following two lines to custom.crons should do it:

0 23 * * * (/home/EchoIRLP/ scripts/echo_ disable >/dev/null 2>&1)
0 06 * * * (/home/EchoIRLP/ scripts/echo_ enable >/dev/null 2>&1)

The 23 above will disable it at 11 pm and the 06 will re-enable it at 6 am.

Google will help you with figuring out the asterisks to do the reboot when you like. Change /home/EchoIRLP/scripts/echo_enable to /usr/bin/sudo reboot and add the line to custom.crons.

After custom.cons is edited, you need to do an

update files

Note this is a band aid to at least allow you some level of ssh access while you are figuring out root cause.

John va3wm
IRLP 2248 2931



On Mar 29, 2020, at 10:55 AM, Klaus Rung via Groups.Io <k_rung@...> wrote:


Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Klaus Rung
 

Here is a part of the log of the 7371 Node

Mar 28 2020 14:44:25 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 14:50:05 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:01:49 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:04:34 -0400 decode: DTMF = A
Mar 28 2020 15:04:34 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 15:08:37 -0400 decode: DTMF = A
Mar 28 2020 15:08:37 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 15:18:48 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:20:53 -0400 EchoIRLP: starting 2173 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:21:01 -0400 Node Enabled
Mar 28 2020 15:21:05 -0400 EchoIRLP: shutdown
Mar 28 2020 15:21:07 -0400 EchoIRLP: starting 2553 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:22:15 -0400 EchoIRLP: shutdown
Mar 28 2020 15:23:14 -0400 EchoIRLP: starting 2175 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:23:22 -0400 Node Enabled
Mar 28 2020 15:23:26 -0400 EchoIRLP: shutdown
Mar 28 2020 15:23:27 -0400 EchoIRLP: starting 2559 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:25:17 -0400 EchoIRLP: connected echolink J73HGL 1
Mar 28 2020 15:25:24 -0400 EchoIRLP: connected speakfreely J73W-R 2
Mar 28 2020 15:25:52 -0400 EchoIRLP: disconnected bye J73HGL 1
Mar 28 2020 15:25:53 -0400 EchoIRLP: Node Disconnect from EchoLink 877742 J73HGL
Mar 28 2020 15:26:00 -0400 EchoIRLP: disconnected bye J73W-R 0
Mar 28 2020 15:40:40 -0400 EchoIRLP: connected echolink J73HGL 1
Mar 28 2020 15:40:47 -0400 EchoIRLP: connected speakfreely J73W-R 2
Mar 28 2020 16:07:40 -0400 EchoIRLP: disconnected rtcp_timeout J73HGL 1
Mar 28 2020 16:07:42 -0400 EchoIRLP: Node Disconnect from EchoLink 7019 J73HGL
Mar 28 2020 16:07:55 -0400 EchoIRLP: disconnected bye J73W-R 0
Mar 28 2020 16:35:27 -0400 decode: DTMF = A
Mar 28 2020 16:35:27 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 19:33:19 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 19:46:45 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 19:54:27 -0400 WARNING - DNS is not setup correctly
Mar 28 2020 20:03:40 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.



On Sunday, March 29, 2020, 11:19:14 a.m. EDT, David Cameron - IRLP <dcameron@...> wrote:


I would determine what the problem is. You can run openvpn in debug mode. You can also run in pings to something on the outside, something on the inside, and the router. Figure out what the failure is.

Next time, try a restart of the VPN by running

/etc/init.d/openvpn restart

I have not looked, but there likely already scripts to do this as openvpn is very common. 

Dave Cameron 

-------- Original message --------
From: "Klaus Rung via Groups.Io" <k_rung@...>
Date: 3/29/20 7:55 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity

Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




David Cameron - IRLP
 

Not the IRLP log, but the openvpn log. You will have to increase the verbosity in the configuration file, then look at the logs. I am not sure where you view that, but it is a start. 

You could run a simple script that does a ping every 10 seconds to the vpn server, and if it fails three times in a row, restart the vpn service. 

Not a cron, but one that runs all the time in the background. 

Dave Cameron 



-------- Original message --------
From: "Klaus Rung via Groups.Io" <k_rung@...>
Date: 3/29/20 9:29 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity

Here is a part of the log of the 7371 Node

Mar 28 2020 14:44:25 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 14:50:05 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:01:49 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:04:34 -0400 decode: DTMF = A
Mar 28 2020 15:04:34 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 15:08:37 -0400 decode: DTMF = A
Mar 28 2020 15:08:37 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 15:18:48 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 15:20:53 -0400 EchoIRLP: starting 2173 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:21:01 -0400 Node Enabled
Mar 28 2020 15:21:05 -0400 EchoIRLP: shutdown
Mar 28 2020 15:21:07 -0400 EchoIRLP: starting 2553 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:22:15 -0400 EchoIRLP: shutdown
Mar 28 2020 15:23:14 -0400 EchoIRLP: starting 2175 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:23:22 -0400 Node Enabled
Mar 28 2020 15:23:26 -0400 EchoIRLP: shutdown
Mar 28 2020 15:23:27 -0400 EchoIRLP: starting 2559 thebridge 1.09 Mar 13 2019 18:11:33
Mar 28 2020 15:25:17 -0400 EchoIRLP: connected echolink J73HGL 1
Mar 28 2020 15:25:24 -0400 EchoIRLP: connected speakfreely J73W-R 2
Mar 28 2020 15:25:52 -0400 EchoIRLP: disconnected bye J73HGL 1
Mar 28 2020 15:25:53 -0400 EchoIRLP: Node Disconnect from EchoLink 877742 J73HGL
Mar 28 2020 15:26:00 -0400 EchoIRLP: disconnected bye J73W-R 0
Mar 28 2020 15:40:40 -0400 EchoIRLP: connected echolink J73HGL 1
Mar 28 2020 15:40:47 -0400 EchoIRLP: connected speakfreely J73W-R 2
Mar 28 2020 16:07:40 -0400 EchoIRLP: disconnected rtcp_timeout J73HGL 1
Mar 28 2020 16:07:42 -0400 EchoIRLP: Node Disconnect from EchoLink 7019 J73HGL
Mar 28 2020 16:07:55 -0400 EchoIRLP: disconnected bye J73W-R 0
Mar 28 2020 16:35:27 -0400 decode: DTMF = A
Mar 28 2020 16:35:27 -0400 decode: The command contains a bad digit (S,P,A,B,C,D)
Mar 28 2020 19:33:19 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 19:46:45 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.
Mar 28 2020 19:54:27 -0400 WARNING - DNS is not setup correctly
Mar 28 2020 20:03:40 -0400 ipupdate: WARNING - Comms error to server 142.103.194.4. Cant obtain current IP.



On Sunday, March 29, 2020, 11:19:14 a.m. EDT, David Cameron - IRLP <dcameron@...> wrote:


I would determine what the problem is. You can run openvpn in debug mode. You can also run in pings to something on the outside, something on the inside, and the router. Figure out what the failure is.

Next time, try a restart of the VPN by running

/etc/init.d/openvpn restart

I have not looked, but there likely already scripts to do this as openvpn is very common. 

Dave Cameron 

-------- Original message --------
From: "Klaus Rung via Groups.Io" <k_rung@...>
Date: 3/29/20 7:55 AM (GMT-08:00)
To: IRLP@irlp.groups.io
Subject: Re: [IRLP] Node Loosing Connectivity

Thanks John,

Any help in how to craft the cron job?

Klaus


On Sunday, March 29, 2020, 10:44:10 a.m. EDT, John Lorenc <va3xjl@...> wrote:


The rebooting can be automated with a cron job sudo reboot.

The Ethernet interface may be failing. This happened to node 2998 before hw replacement.

John va3wm
Nodes 2248 2931

On Sun, Mar 29, 2020, 10:19 AM Klaus Rung via Groups.Io <k_rung=yahoo.com@groups.io> wrote:
Hi all,

I am having a problem with a node that is using the vpn service from irlp that is connected to a Carrier Grade Nat connection. The node is set to a static ip address and connected to the internet providers router.

The node works fine for a number of hours once booted and then by next morning looses connectivity with the network. Once rebooted it starts to work again normally with in and out calls and ssh remote login working just fine.

Is there a way to use a script to keep the connection alive from the node end so we don't have to have daily manual restarts of the node to wake up the connection?

Has anyone else had this problem and what solution have they found?

Klaus
ve3kr
node 7371 J73W




Kevin B
 

Dave, I like what you said about pinging the server and restarting if it fails 3 times. How would I go about doing that? I looked it up a long time ago but never could find anything that worked. Im left with doing a DTMF reboot but having it ping and restart only the openvpn would be much better.

Kevin


John
 

Robert Pectol wrote a connectivity_check script.
The header of the script read:
# connectivity_check v0.3 Oct. 18, 2012
# Written by Robert Pectol - http://rob.pectol.com/irlp/
#
# Running this script will test the connectivity to your ISP's services (the Internet)
# and can be called manually or from another script. It returns an exit status of 0
# if the connection is detected as being, 'good', else it returns an exit status of 1.

It might be in the archives..

73 John @ 6163 FNQ'land

On 1/04/2020 8:54 am, Kevin B wrote:
Dave, I like what you said about pinging the server and restarting if it fails 3 times. How would I go about doing that? I looked it up a long time ago but never could find anything that worked. Im left with doing a DTMF reboot but having it ping and restart only the openvpn would be much better.
Kevin