Topics

Node 3288 moribund


Ramon Gandia
 

My node 3288 is in a remote Alaskan village. Internet connection is by
way of a satellite dish on VIASAT (formerly EXEDE). I have a block of
IP's from them.

Ping time is now about 2.5 seconds; and connections are often dropped.
The aiming point of the dish is barely above the horizon, and well
beyond the coverage area of VIASAT.

On top of that, looking at the log files in /var/log/syslog shows a
breakin attempt on ssh at the rate of 4 attacks PER SECOND.

Finally, the command "last" normally fails, and the wtmp logfile
resets every few minutes. This leads me to believe that the
computer has been hacked. Hackers always try to erase all tracks
of themselves.

This is PiRLP node running Wheezy/Raspbian linux 7. 7.6.

My plan is two fold.

1. To move service to the local Cell tower. They now have 4G/LTE.

That service sends out a dynamic IP in the 10.x.y.z range, which
is private. If I run a router on my side, it will be double-NATed.

But I can tie an IRLP node directly to the cell modem, and at least
pick up the 10.x.y/z address directly.

There is no hope whatsoever of having the ISP give me a fixed IP.

So, that would leave me using VPN.

I tried my 3289 node on the cell modem, and worked fine on VPN.
3289 is here on my bench, testing before deploying to another
village.

2. Will download the raspbian, burn the sd card, and test it all
using one of several Pi's I have in stock. Including openvpn.

Once it runs, I'd request the vpn config from Dave.

Should I use a Pi4, or should I go with a Pi3B+?
I have several of each. I think the 3 should be ample.

Next question is: Does the old PiRLP have any modifications to
the Pi board? I believe not, that the only changes are in the
IRLP v3 board on it.

This little Pi is about 3 hours from here by Jeep, and I am
trying to save a trip there until the last minute. Shut down
the node up there, drive, change out the Pi -- or at least the
SD card, and touch up the audio levels.

I think I an run the backup_for_reinstall script from here and
download the file.gz. Might take a few tries what with the
bad internet.

Any ideas or suggestions, anyone?

--
Ramon Gandia AL7X 3288 3289 7254


larry_n7fm
 

Ramon,

If I were choosing I would go with a Pi3+.
Less power required and runs much cooler than Pi4. Still plenty of Horsepower for IRLP.

Larry -N7FM

On 6/12/20 1:10 AM, Ramon Gandia wrote:
My node 3288 is in a remote Alaskan village. Internet connection is
by way of a satellite dish on VIASAT (formerly EXEDE). I have a
block of IP's from them.
Ping time is now about 2.5 seconds; and connections are often
dropped. The aiming point of the dish is barely above the horizon,
and well beyond the coverage area of VIASAT.
On top of that, looking at the log files in /var/log/syslog shows a breakin attempt on ssh at the rate of 4 attacks PER SECOND.
Finally, the command "last" normally fails, and the wtmp logfile resets every few minutes. This leads me to believe that the computer
has been hacked. Hackers always try to erase all tracks of
themselves.
This is PiRLP node running Wheezy/Raspbian linux 7. 7.6.
My plan is two fold.
1. To move service to the local Cell tower. They now have 4G/LTE.
That service sends out a dynamic IP in the 10.x.y.z range, which is
private. If I run a router on my side, it will be double-NATed.
But I can tie an IRLP node directly to the cell modem, and at least pick up the 10.x.y/z address directly.
There is no hope whatsoever of having the ISP give me a fixed IP.
So, that would leave me using VPN.
I tried my 3289 node on the cell modem, and worked fine on VPN. 3289
is here on my bench, testing before deploying to another village.
2. Will download the raspbian, burn the sd card, and test it all using one of several Pi's I have in stock. Including openvpn.
Once it runs, I'd request the vpn config from Dave.
Should I use a Pi4, or should I go with a Pi3B+? I have several of
each. I think the 3 should be ample.
Next question is: Does the old PiRLP have any modifications to the Pi
board? I believe not, that the only changes are in the IRLP v3 board
on it.
This little Pi is about 3 hours from here by Jeep, and I am trying to
save a trip there until the last minute. Shut down the node up
there, drive, change out the Pi -- or at least the SD card, and touch
up the audio levels.
I think I an run the backup_for_reinstall script from here and download the file.gz. Might take a few tries what with the bad
internet.
Any ideas or suggestions, anyone?
-- Ramon Gandia AL7X 3288 3289 7254
uns much cooler and has enough h


k9dc
 

I have three suggestions Ramon.

1. I wouldn’t worry about running the node behind another router. You are going to have use IRLP VPN anyway, an extra router will have no impact, assuming it is reasonably reliable. Cellular routers generally give you a degree of visibility over the cell connection that you would not have if you did not use the router, plus the ability to connect another device behind the router over the same connection (laptop when you are on site etc).

2. You should upgrade the node PiRLP to at least Debian 9 (Stretch). Debian 7 is long obsolete and is no longer supported (by Debian).

3. You can use either the Pi3B+ or Pi4. Personally, I would go with the Pi4, because it is the real fire breather of the two. No one ever complains about having a computer that is too fast. But either one will work.

I really doubt your computer has been hacked. Technically possible of course, especially if you left SSH on port TCP 22. But there simply is no attraction to a machine with a ping time of 2.5 seconds. Hackers likely will not wait 2.5 seconds for a response. But you may see thousands of probes per second on port TCP 22. The better the connection, is, the more likely your system might seem attractive. I would suggest choosing a different port (and of course a very strong password).

-k9dc

On Jun 12, 2020, at 04:10, Ramon Gandia <rfg8io@...> wrote:

My node 3288 is in a remote Alaskan village. Internet connection is by
way of a satellite dish on VIASAT (formerly EXEDE). I have a block of
IP's from them.

Ping time is now about 2.5 seconds; and connections are often dropped.
The aiming point of the dish is barely above the horizon, and well
beyond the coverage area of VIASAT.

On top of that, looking at the log files in /var/log/syslog shows a
breakin attempt on ssh at the rate of 4 attacks PER SECOND.

Finally, the command "last" normally fails, and the wtmp logfile
resets every few minutes. This leads me to believe that the
computer has been hacked. Hackers always try to erase all tracks
of themselves.

This is PiRLP node running Wheezy/Raspbian linux 7. 7.6.

My plan is two fold.

1. To move service to the local Cell tower. They now have 4G/LTE.

That service sends out a dynamic IP in the 10.x.y.z range, which
is private. If I run a router on my side, it will be double-NATed.

But I can tie an IRLP node directly to the cell modem, and at least
pick up the 10.x.y/z address directly.

There is no hope whatsoever of having the ISP give me a fixed IP.

So, that would leave me using VPN.

I tried my 3289 node on the cell modem, and worked fine on VPN.
3289 is here on my bench, testing before deploying to another
village.

2. Will download the raspbian, burn the sd card, and test it all
using one of several Pi's I have in stock. Including openvpn.

Once it runs, I'd request the vpn config from Dave.

Should I use a Pi4, or should I go with a Pi3B+?
I have several of each. I think the 3 should be ample.

Next question is: Does the old PiRLP have any modifications to
the Pi board? I believe not, that the only changes are in the
IRLP v3 board on it.

This little Pi is about 3 hours from here by Jeep, and I am
trying to save a trip there until the last minute. Shut down
the node up there, drive, change out the Pi -- or at least the
SD card, and touch up the audio levels.

I think I an run the backup_for_reinstall script from here and
download the file.gz. Might take a few tries what with the
bad internet.

Any ideas or suggestions, anyone?

--
Ramon Gandia AL7X 3288 3289 7254


Lonney [K1LH]
 

If you have port 22 open to the world and watch the auth log, you'll see a constant steam of drive-bys attempting various username and password combinations.

There are various ways to mitigate this, much like any form of security physical or virtual, its about raising the bar to a point where most opportunists will move on to easier targets.

  • The simplest is to change the SSH port to something like 22000. But that doesn't stop those doing port scans from finding it, most don't bother with this as its time consuming.
  • Create a public/private key-pair for SSH and also use that, with out the private key no one is getting in unless they exploit an unpatched vulnerability in SSH or another exposed service. Combine this with moving the port, is probably good enough and fairly simple to get working.
  • iptables can be configured to throttle connections from a given IP address to port 22 for example - I used to implement this so that 3 connection attempts with-in a minute would then drop connections from that IP for 5 minutes - this worked pretty well, often the same "callers" wouldn't come back once their connection attempts were ignored for a few minutes.
  • Geo-blocking - ignore connections from countries outside of your own.
  • Run a VPN server on the IRLP node or network its on, then no need to open port 22 to the world.

I had a realization yesterday that anyone port scanning and probing port 15425 can see a call-sign, if they know enough to recognize that and go look up the public FCC records they can quickly get a name and address of what they just found. Security conscious folks might have a problem with that these days, it doesn't make much imagination to understand this could be used to unmask your identity and physical address of your IP address (if you're running an IRLP node from the same network) anywhere it gets logged on the internet - e.g. message boards, websites, anywhere someone has access to the logs of places you or anyone else on your network is going, the list is endless. Bit off topic, but the port 22 thing got me thinking :-)

--
Lonney


k9dc
 

Thank you for opening these topics up!

On Jun 12, 2020, at 10:17, Lonney [K1LH] via groups.io <lonney.harper=icloud.com@groups.io> wrote:

If you have port 22 open to the world and watch the auth log, you'll see a constant steam of drive-bys attempting various username and password combinations.

• The simplest is to change the SSH port to something like 22000. But that doesn't stop those doing port scans from finding it, most don't bother with this as its time consuming.
I typically use port 15427 for this. Easy to bracket with 15425 and 15426

• Create a public/private key-pair for SSH and also use that, with out the private key no one is getting in unless they exploit an unpatched vulnerability in SSH or another exposed service.
I submit, that operating a node using SSH with keys, is significantly easier than any of the web based utilities, (remote admin or IRLPvCon), especially from a cell phone. To drop a call, SSH connect, type “end” and the connection is down. No username, no password, and completely secure.

• Run a VPN server on the IRLP node or network its on, then no need to open port 22 to the world.
You need something open to the world to manage it. (change your port)

Also IRLP VPN (not what you mean here, I know), the node is open to the world just from a different IP address.

I had a realization yesterday that anyone port scanning and probing port 15425 can see a call-sign, if they know enough to recognize that and go look up the public FCC records they can quickly get a name and address of what they just found.
Yes it has been that way for over 20 years. Don’t forget, all IRLP nodes are also visible in global DNS.

;; QUESTION SECTION:
;stn4730.ip.irlp.net. IN A

;; ANSWER SECTION:
stn4730.ip.irlp.net. 60 IN A 44.48.26.9

-k9dc


Nosey Nick VA3NNW
 

Ramon Gandia wrote:
Should I use a Pi4, or should I go with a Pi3B+?
I have several of each.  I think the 3 should be ample.
For a media centre I'd be picky, but honestly doesn't matter much for
IRLP, Pi3, 3B+, 4... I'm running IRLP quite nicely on a Pi Zero W, which
might not match the official supported hardware but it works great and
I've never seen it use more than 5-10% CPU when doing normal IRLP stuff,
usually  <3%    :-)

Next question is: Does the old PiRLP have any modifications to
the Pi board?  I believe not, that the only changes are in the
IRLP v3 board on it.
I think OFFICIALLY you snip one of the GPIO pins and glue the
corresponding hole on the ribbon cable so you can't accidentally insert
it the wrong way round:

http://www.irlp.net/R_Pi/P2010014.JPG
http://www.irlp.net/R_Pi/P2010016.JPG
rest of http://www.irlp.net/R_Pi/

Personally I didn't dare cut up a perfect Pi, just used a regular ribbon
connector, Wire 0 is already red, and I just make sure I never plug it
in wrong. It's not something you're going to be plugging/unplugging
often anyway.

Dave said:

I really doubt your computer has been hacked. Technically possible of
course, especially if you left SSH on port TCP 22.
I'm paid to do IT/Internet Security and I'd say it's ABSOLUTELY
INEVITABLE you're hacked if you've got a weak-enough root password. On
occasion I've put a honeypot machine on the internet with a 10-char
dictionary word for root password, and seen first probes within seconds
and hacked within 4 hours. Add a few digits and it might be months
instead of hours. Really you need a decent, long, sufficiently random
password of several words, digits, and symbols, or a "pass phrase" like
https://correcthorsebatterystaple.net/ , for ALL user accounts on any
internet-exposed machine. Better still SSH keys, see below...

But there simply is no attraction to a machine with a ping time of 2.5
seconds. Hackers likely will not wait 2.5 seconds for a response
Oh I assumed Ramon meant AFTER it was hacked. At which point it's
probably busy sending other people's spam, serving parts of an illegal
website, being a scanner / running the tools to attempt to hack MORE
insecure machines, inefficiently crypto-mining, or some combination
thereof - and 2.5second ping time could easily be because the poor thing
is overloaded.

But you may see thousands of probes per second on port TCP 22. The
better the connection, is, the more likely your system might seem
attractive. I would suggest choosing a different port (and of course a
very strong password).
Different port helps a bit, but not as much as you might expect. They
WILL scan all ports, they WILL still find it, quickly determine it's SSH
(SSH does announce itself after all), and start the dictionary attacks.
This isn't HUMANS who will get bored and go away, this is automated
tools attacking you. The above "4 hours" might have been a day if it was
on an obscure port - I'll test sometime if you like.

Very strong password for sure.

Lonney wrote:

Create a public/private key-pair for SSH and also use that, with out
the private key no one is getting in unless they exploit an unpatched
vulnerability in SSH or another exposed service. Combine this with
moving the port, is probably good enough and fairly simple to get working.
THAT's good advice too, and "PermitRootLogin no" in your sshd_config
(always log in as "repeater" or other named user, and "sudo" to root
instead).

To clarify though, you can log in with a key OR a password unless you
also disable passwords - Seriously consider "PasswordAuthentication no"
(to disable all password logins and use keys only) if you have at least
2 SSH keys that can get back in, OR physical access to be able to regain
access in if you lose all copies of your primary SSH key.

.... but SSH itself is kinda complicated for most users, especially
non-technical users, it's hard enough to get them to even USE PuTTY
never mind use it PROPERLY with keys and disabled passwords, so best
advice is probably "set a password of at least 12 characters, a short
phrase is better than a word, and definitely throw some digits AND
symbols in there", see https://correcthorsebatterystaple.net/ for
example. Even if you have to write it down to remember it - YES, the IT
Security guy just told you it's OK to WRITE IT DOWN and store it
somewhere safe - because nobody's going to be able to steal the paper
copy from halfway around the world, but if it's long enough that you HAD
to write it down, then it's probably long enough that their tools won't
guess it even if they try 315569267 passwords a year (that's a guess,
but not an unrealistic one)

Nick VA3NNW

--
"Nosey" Nick Waterman, VA3NNW/G7RZQ, K2 #5209.
use Std::Disclaimer; sig@...
The optimum committee has no members. -- Norman Augustine


Ramon Gandia
 

I'll go with a Pi3B+ Already downloaded Latest 5/27 Raspbian Lite.
And it sha256sum ok.

Here is a morsel on the existing one:

repeater@stn3288:~/$ uptime
23:49:54 up 650 days, 22:18, 1 user, load average: 0.39, 0.40, 0.43
repeater@stn3288:~/$

Very venerable indeed.

--
Ramon AL7X

On 6/12/20 6:57 PM, Nosey Nick VA3NNW wrote:
Ramon Gandia wrote:
Should I use a Pi4, or should I go with a Pi3B+?
I have several of each.  I think the 3 should be ample.
For a media centre I'd be picky, but honestly doesn't matter much for
IRLP, Pi3, 3B+, 4... I'm running IRLP quite nicely on a Pi Zero W, which
might not match the official supported hardware but it works great and
I've never seen it use more than 5-10% CPU when doing normal IRLP stuff,
usually  <3%    :-)

Next question is: Does the old PiRLP have any modifications to
the Pi board?  I believe not, that the only changes are in the
IRLP v3 board on it.
I think OFFICIALLY you snip one of the GPIO pins and glue the
corresponding hole on the ribbon cable so you can't accidentally insert
it the wrong way round:

http://www.irlp.net/R_Pi/P2010014.JPG
http://www.irlp.net/R_Pi/P2010016.JPG
rest of http://www.irlp.net/R_Pi/

Personally I didn't dare cut up a perfect Pi, just used a regular ribbon
connector, Wire 0 is already red, and I just make sure I never plug it
in wrong. It's not something you're going to be plugging/unplugging
often anyway.

Dave said:

I really doubt your computer has been hacked. Technically possible of
course, especially if you left SSH on port TCP 22.
I'm paid to do IT/Internet Security and I'd say it's ABSOLUTELY
INEVITABLE you're hacked if you've got a weak-enough root password. On
occasion I've put a honeypot machine on the internet with a 10-char
dictionary word for root password, and seen first probes within seconds
and hacked within 4 hours. Add a few digits and it might be months
instead of hours. Really you need a decent, long, sufficiently random
password of several words, digits, and symbols, or a "pass phrase" like
https://correcthorsebatterystaple.net/ , for ALL user accounts on any
internet-exposed machine. Better still SSH keys, see below...

But there simply is no attraction to a machine with a ping time of 2.5
seconds. Hackers likely will not wait 2.5 seconds for a response
Oh I assumed Ramon meant AFTER it was hacked. At which point it's
probably busy sending other people's spam, serving parts of an illegal
website, being a scanner / running the tools to attempt to hack MORE
insecure machines, inefficiently crypto-mining, or some combination
thereof - and 2.5second ping time could easily be because the poor thing
is overloaded.

But you may see thousands of probes per second on port TCP 22. The
better the connection, is, the more likely your system might seem
attractive. I would suggest choosing a different port (and of course a
very strong password).
Different port helps a bit, but not as much as you might expect. They
WILL scan all ports, they WILL still find it, quickly determine it's SSH
(SSH does announce itself after all), and start the dictionary attacks.
This isn't HUMANS who will get bored and go away, this is automated
tools attacking you. The above "4 hours" might have been a day if it was
on an obscure port - I'll test sometime if you like.

Very strong password for sure.

Lonney wrote:

Create a public/private key-pair for SSH and also use that, with out
the private key no one is getting in unless they exploit an unpatched
vulnerability in SSH or another exposed service. Combine this with
moving the port, is probably good enough and fairly simple to get working.
THAT's good advice too, and "PermitRootLogin no" in your sshd_config
(always log in as "repeater" or other named user, and "sudo" to root
instead).

To clarify though, you can log in with a key OR a password unless you
also disable passwords - Seriously consider "PasswordAuthentication no"
(to disable all password logins and use keys only) if you have at least
2 SSH keys that can get back in, OR physical access to be able to regain
access in if you lose all copies of your primary SSH key.

.... but SSH itself is kinda complicated for most users, especially
non-technical users, it's hard enough to get them to even USE PuTTY
never mind use it PROPERLY with keys and disabled passwords, so best
advice is probably "set a password of at least 12 characters, a short
phrase is better than a word, and definitely throw some digits AND
symbols in there", see https://correcthorsebatterystaple.net/ for
example. Even if you have to write it down to remember it - YES, the IT
Security guy just told you it's OK to WRITE IT DOWN and store it
somewhere safe - because nobody's going to be able to steal the paper
copy from halfway around the world, but if it's long enough that you HAD
to write it down, then it's probably long enough that their tools won't
guess it even if they try 315569267 passwords a year (that's a guess,
but not an unrealistic one)

Nick VA3NNW


David Cameron - IRLP
 

What is loading that processor? It would usually be about 0.03 if just running IRLP.

Dave Cameorn

On 13/06/2020 12:53 a.m., Ramon Gandia wrote:
I'll go with a Pi3B+   Already downloaded Latest 5/27 Raspbian Lite.
And it sha256sum ok.

Here is a morsel on the existing one:

repeater@stn3288:~/$ uptime
 23:49:54 up 650 days, 22:18,  1 user,  load average: 0.39, 0.40, 0.43
repeater@stn3288:~/$

Very venerable indeed.

--
Ramon AL7X

On 6/12/20 6:57 PM, Nosey Nick VA3NNW wrote:
Ramon Gandia wrote:
Should I use a Pi4, or should I go with a Pi3B+?
I have several of each.  I think the 3 should be ample.
For a media centre I'd be picky, but honestly doesn't matter much for
IRLP, Pi3, 3B+, 4... I'm running IRLP quite nicely on a Pi Zero W, which
might not match the official supported hardware but it works great and
I've never seen it use more than 5-10% CPU when doing normal IRLP stuff,
usually  <3%    :-)

Next question is: Does the old PiRLP have any modifications to
the Pi board?  I believe not, that the only changes are in the
IRLP v3 board on it.
I think OFFICIALLY you snip one of the GPIO pins and glue the
corresponding hole on the ribbon cable so you can't accidentally insert
it the wrong way round:

http://www.irlp.net/R_Pi/P2010014.JPG
http://www.irlp.net/R_Pi/P2010016.JPG
rest of http://www.irlp.net/R_Pi/

Personally I didn't dare cut up a perfect Pi, just used a regular ribbon
connector, Wire 0 is already red, and I just make sure I never plug it
in wrong. It's not something you're going to be plugging/unplugging
often anyway.

Dave said:

I really doubt your computer has been hacked. Technically possible of
course, especially if you left SSH on port TCP 22.
I'm paid to do IT/Internet Security and I'd say it's ABSOLUTELY
INEVITABLE you're hacked if you've got a weak-enough root password. On
occasion I've put a honeypot machine on the internet with a 10-char
dictionary word for root password, and seen first probes within seconds
and hacked within 4 hours. Add a few digits and it might be months
instead of hours. Really you need a decent, long, sufficiently random
password of several words, digits, and symbols, or a "pass phrase" like
https://correcthorsebatterystaple.net/ , for ALL user accounts on any
internet-exposed machine. Better still SSH keys, see below...

But there simply is no attraction to a machine with a ping time of 2.5
seconds. Hackers likely will not wait 2.5 seconds for a response
Oh I assumed Ramon meant AFTER it was hacked. At which point it's
probably busy sending other people's spam, serving parts of an illegal
website, being a scanner / running the tools to attempt to hack MORE
insecure machines, inefficiently crypto-mining, or some combination
thereof - and 2.5second ping time could easily be because the poor thing
is overloaded.

But you may see thousands of probes per second on port TCP 22. The
better the connection, is, the more likely your system might seem
attractive. I would suggest choosing a different port (and of course a
very strong password).
Different port helps a bit, but not as much as you might expect. They
WILL scan all ports, they WILL still find it, quickly determine it's SSH
(SSH does announce itself after all), and start the dictionary attacks.
This isn't HUMANS who will get bored and go away, this is automated
tools attacking you. The above "4 hours" might have been a day if it was
on an obscure port - I'll test sometime if you like.

Very strong password for sure.

Lonney wrote:

Create a public/private key-pair for SSH and also use that, with out
the private key no one is getting in unless they exploit an unpatched
vulnerability in SSH or another exposed service. Combine this with
moving the port, is probably good enough and fairly simple to get working.
THAT's good advice too, and "PermitRootLogin no" in your sshd_config
(always log in as "repeater" or other named user, and "sudo" to root
instead).

To clarify though, you can log in with a key OR a password unless you
also disable passwords - Seriously consider "PasswordAuthentication no"
(to disable all password logins and use keys only) if you have at least
2 SSH keys that can get back in, OR physical access to be able to regain
access in if you lose all copies of your primary SSH key.

.... but SSH itself is kinda complicated for most users, especially
non-technical users, it's hard enough to get them to even USE PuTTY
never mind use it PROPERLY with keys and disabled passwords, so best
advice is probably "set a password of at least 12 characters, a short
phrase is better than a word, and definitely throw some digits AND
symbols in there", see https://correcthorsebatterystaple.net/ for
example. Even if you have to write it down to remember it - YES, the IT
Security guy just told you it's OK to WRITE IT DOWN and store it
somewhere safe - because nobody's going to be able to steal the paper
copy from halfway around the world, but if it's long enough that you HAD
to write it down, then it's probably long enough that their tools won't
guess it even if they try 315569267 passwords a year (that's a guess,
but not an unrealistic one)

Nick VA3NNW