Topics

Node 7490 not connecting to reflectors

ai6bx <ai6bx@...>
 
Edited

7490, an embedded rack mount node with SSD, has been working flawlessly for several years now. It has programming files issuing commands to link and drop links with various reflectors during the course of the day and until early Sunday, all has worked great. I noticed last night that the node showed connected to 9667 however signal was not passing. I also noticed that it did not show up on the reflector page and when looking at active nodes at IRLP.net, it shows as idle.

I disconnected the node and then issued commands to establish the link again with the same results. Using stn7490.ip.irlp.net, the node panel comes up. I rebooted the node with no changes. The node has been cycling it’s programmed changes today and continues to show an established link. Any ideas?

thank you,

keith

k9dc
 

Stay with me Keith, this is kind of long.

One of the characteristics of a working IRLP node is you can telnet to the the authentication port (15425) from the outside. If the node is idle, it will connect, identify itself by node number (actually the PGP UserID) and display the foreign IP that is connecting. (we use this characteristic as a quick test of port forwarding on a remote site). For example, if I telnet to port 15425 on one of mine, I get the following response…

repeater@tigger:~$ telnet stn4730.ip.irlp.net 15425
Trying 166.139.38.213...
Connected to stn4730.ip.irlp.net.
Escape character is '^]'.
stn4730 - k9ip : Welcome 75.48.51.3

I connected to node number stn4730 at IP 166.139.38.213 (public IP of 4730), from the IP 75.48.51.3, which is the Public IP of my home network. All good.

But when I connect to 7490 which is at 47.156.178.150, I get a totally bogus IP address returned. Further the address it returns (10.224.137.213) is always the same. Regardless of where I come from, your node always thinks I am coming from 10.224.137.213. This is impossible, because 10.224.137.213 is a private address and not routed over the public Internet.

repeater@tigger:~$ telnet stn7490.ip.irlp.net 15425
Trying 47.156.178.150… <— current IP of stn7490 (wrong if you have a VPN running)
Connected to stn7490.ip.irlp.net.
Escape character is '^]'.
stn7490 - ai6bx : Welcome 10.224.137.213 <— not MY address and is always the same

Further, the IP address currently used by stn7490 is 47.156.178.150. But isn't stn7490 supposed to be using an IRLP VPN address? You should be on an address of 44.48.26.something. You just sent us a ticket advising us that IRLP VPN was working great for you on 7490. Yet looking through the logs on the VPN hub, you have never connected, not even once.

So I have a couple of questions:
1. Do you have multiple routers between your node and the public Internet? Please describe your ISP connection and how you are delivering Internet to your node;
2. Where did you place the IRLP VPN configuration file we sent you? As far as I can tell, you have never had a VPN up and running. Yet you told us it was working great;
3. What IP address is returned when, from the node command line, you issue the command ‘telnet irlp.net 10000’

-puzzled at the help desk.

--
Dave K9DC, IRLP Installation Team

On Nov 5, 2019, at 01:06, ai6bx <ai6bx@...
wrote:

7490, an embedded rack Lund node with sad, has been working flawlessly for several years now. It has programming files issuing commands to link and drop links with various reflectors during the course of the day and until early Sunday, all has worked great. I noticed last night that the node showed connected to 9667 however signal was not passing. I also noticed that it did not show up on the reflector page and when looking at active nodes at IRLP.net, it shows as idle.

I disconnected the node and then issued commands to establish the link again with the same results. Using stn7490.ip.irlp.net, the node panel comes up. I rebooted the node with no changes. The node has been cycling it’s programmed changes today and continues to show an established link. Any ideas?

thank you,

keith

ai6bx <ai6bx@...>
 

On Tue, Nov 5, 2019 at 04:23 AM, k9dc wrote:
Dave,
Thank you for your thoughtful feedback. 7490 is not operating with a VPN so 47.156.178.150 is the appropriate public IP. My other node number 7482 is working through the AMPR VPN IP at his time.

Yes, I do have a router between my Internet connection and the node, a configuration that has been in place for the better part of three years. I have a Netgear router for my home internet with an ARDEN gateway node carrying signal to my repeater site. The Netgear has a DMZ set for the IP of the AREDN gateway and the AREDN node has all the appropriate port forwards built into it to direct two way traffic to the repeater site supporting the IRLP/Echolink node, DMR repeater, remote control for other repeaters, Winlink server, mapping services, and a few other pieces. I think I may know what is happening as I have watched and seen the IRLP come and go today. I note that another AREDN gateway has been opened up on the network with near equal link quality as the path I am using to my QTH. the TCP port forwards are following the path back to my QTH and I suspect the UDP ports going back to the Internet may be taking the alternate path as it is presenting as the most efficient path for exit traffic. Does this make sense as the possible culprit? I think it does as the telnet irlp.net 10000 yields IP 72.132.11.23 which is the alternate gateway path I referenced. It may be time to make some hard code changes in my OLSR tables in the AREDN node to force the return path or get the second gateway shut down.

Thoughts?

Keith

John
 

Keith et al,

You didn't answer Dave's 3rd question
What IP address is returned when, from the node command line, you issue the command ‘telnet irlp.net 10000'

A telnet query to your 7490 node tells me
C:\Windows\System32>telnet stn7490.ip.irlp.net 15425
Connecting To stn7490.ip.irlp.net...Could not open connection to the host, on port 15425: Connect failed

Earlier this arvo, I got this IP from my irlp Hosts file - 47.156.178.150
A ping test timed out

I updated my host file and node 7490 IP address changed to 72.132.16.199
Another ping test and tracert to that IP proved positive.

I would suggest you run the troubleshoot-irlp test from the console
after you have disconnected from the reflector.

73 John @ 6163
(currently off air due to technical issue)

On 5/11/2019 4:06:43 PM, ai6bx (ai6bx@...) wrote:
[Edited Message Follows]
7490, an embedded rack mount node with SSD, has been working flawlessly for several years now. It has programming files issuing commands to link and drop links with various reflectors during the course of the day and until early Sunday, all has worked great. I noticed last night that the node showed connected to 9667 however signal was not passing. I also noticed that it did not show up on the reflector page and when looking at active nodes at IRLP.net, it shows as idle.
I disconnected the node and then issued commands to establish the link again with the same results. Using stn7490.ip.irlp.net, the node panel comes up. I rebooted the node with no changes. The node has been cycling it’s programmed changes today and continues to show an established link. Any ideas?
thank you,
keith

k9dc
 

As John suggested, run the 'troubleshoot-irlp’ command. All tests must pass. The ip address of 7490 has changed to 72.132.11.23, but there is no port forwarding in place for that address. It is very important that the inbound path and the outbound path ALWAYS cross the same gateways on the way in or out. Multiple internet connections will screw with that and very likely break things.

I just checked the server log and found the IP address for 7490 has been changing between 72.132.16.199 and 47.156.178.150, probably as many as 100 times per day for at least the last year. This creates considerable load on the servers to keep up with these changes. Your node also is not working when the IP is so wildly unstable.

Please turn 7490 off until you can stabilize your routing. You will most likely have to turn one of your gateways off permanently, or add static routes to your configuration so IRLP always takes the same path out. Two or more active gateways almost never work.

Here are the changes from just yesterday. This (or worse) has been happening at least since the first of this year

Nov 05 01:23:13 nsupdate stn7490 47.156.178.150 VALID
Nov 05 01:33:30 nsupdate stn7490 72.132.16.199 VALID
Nov 05 02:50:57 nsupdate stn7490 47.156.178.150 VALID
Nov 05 03:26:38 nsupdate stn7490 72.132.16.199 VALID
Nov 05 05:02:47 nsupdate stn7490 47.156.178.150 VALID
Nov 05 06:10:56 nsupdate stn7490 47.156.178.150 SAMEIP
Nov 05 06:47:05 nsupdate stn7490 72.132.16.199 VALID
Nov 05 07:18:02 nsupdate stn7490 47.156.178.150 VALID
Nov 05 07:52:26 nsupdate stn7490 72.132.16.199 VALID
Nov 05 08:22:09 nsupdate stn7490 72.132.16.199 SAMEIP
Nov 05 09:10:22 nsupdate stn7490 72.132.16.199 SAMEIP
Nov 05 09:22:19 nsupdate stn7490 47.156.178.150 VALID
Nov 05 09:58:36 nsupdate stn7490 72.132.16.199 VALID
Nov 05 10:34:54 nsupdate stn7490 72.132.16.199 SAMEIP
Nov 05 10:55:38 nsupdate stn7490 47.156.178.150 VALID
Nov 05 11:10:05 nsupdate stn7490 72.132.16.199 VALID
Nov 05 12:04:39 nsupdate stn7490 47.156.178.150 VALID
Nov 05 13:43:52 nsupdate stn7490 72.132.16.199 VALID
Nov 05 14:21:43 nsupdate stn7490 72.132.16.199 SAMEIP
Nov 05 14:57:00 nsupdate stn7490 47.156.178.150 VALID
Nov 05 16:43:30 nsupdate stn7490 72.132.16.199 VALID
Nov 05 16:50:59 nsupdate stn7490 47.156.178.150 VALID
Nov 05 17:06:17 nsupdate stn7490 72.132.16.199 VALID
Nov 05 17:50:55 nsupdate stn7490 47.156.178.150 VALID
Nov 05 19:14:29 nsupdate stn7490 47.156.178.150 SAMEIP
Nov 05 19:59:14 nsupdate stn7490 72.132.16.199 VALID
Nov 05 20:19:56 nsupdate stn7490 47.156.178.150 VALID
Nov 05 20:50:05 nsupdate stn7490 72.132.16.199 VALID
Nov 05 21:11:18 nsupdate stn7490 47.156.178.150 VALID
Nov 05 21:55:37 nsupdate stn7490 72.132.16.199 VALID
Nov 05 22:16:40 nsupdate stn7490 47.156.178.150 VALID
Nov 05 22:38:05 nsupdate stn7490 72.132.16.199 VALID
Nov 05 22:52:28 nsupdate stn7490 47.156.178.150 VALID
Nov 05 23:06:37 nsupdate stn7490 72.132.16.199 VALID
Nov 05 23:21:49 nsupdate stn7490 47.156.178.150 VALID
Nov 05 23:56:15 nsupdate stn7490 72.132.16.199 VALID

On Nov 6, 2019, at 01:01, ai6bx <ai6bx@...> wrote:

On Tue, Nov 5, 2019 at 04:23 AM, k9dc wrote:
Dave,
Thank you for your thoughtful feedback. 7490 is not operating with a VPN so 47.156.178.150 is the appropriate public IP. My other node number 7482 is working through the AMPR VPN IP at his time.

Yes, I do have a router between my Internet connection and the node, a configuration that has been in place for the better part of three years. I have a Netgear router for my home internet with an ARDEN gateway node carrying signal to my repeater site. The Netgear has a DMZ set for the IP of the AREDN gateway and the AREDN node has all the appropriate port forwards built into it to direct two way traffic to the repeater site supporting the IRLP/Echolink node, DMR repeater, remote control for other repeaters, Winlink server, mapping services, and a few other pieces. I think I may know what is happening as I have watched and seen the IRLP come and go today. I note that another AREDN gateway has been opened up on the network with near equal link quality as the path I am using to my QTH. the TCP port forwards are following the path back to my QTH and I suspect the UDP ports going back to the Internet may be taking the alternate path as it is presenting as the most efficient path for exit traffic. Does this make sense as the possible culprit? I think it does as the telnet irlp.net 10000 yields IP 72.132.11.23 which is the alternate gateway path I referenced. It may be time to make some hard code changes in my OLSR tables in the AREDN node to force the return path or get the second gateway shut down.

Thoughts?

Keith

ai6bx <ai6bx@...>
 

IP for question 3 was above as 72.132.11.23. I found the source of the problem with another gateway on the mesh network at very close to the same link quality. I made changes on the network routing to resolve and all seems good now,

k9dc
 

I am pretty sure your node is still not working. You are now (0800 eastern, Sunday) flipping between 72.132.16.199 and 47.156.178.150, neither one works.

-k9dc

On Nov 10, 2019, at 01:25, ai6bx <ai6bx@...> wrote:

IP for question 3 was above as 72.132.11.23. I found the source of the problem with another gateway on the mesh network at very close to the same link quality. I made changes on the network routing to resolve and all seems good now,