ARGH! Hacked twice in a month.

[BigMoneyJim]

Man, now my home server got pwned. Less than a month ago my main web server got pwned. Both were enlisted to send out spam email after getting hacked.

Both of these have been on the 'net for months or years without getting compromised. If you have a server, update it and check it out. And check your outgoing mail logs. My first server crashed...possibly as a result of my host's antispam guards. My home server was using the hard drive more than normal, and then I discovered it's sending out email as fast as it can.

I just rebuilt a server, now I get to do it again. (Once one is compromised you can't trust it.)

Techie info: I think my first server was compromised via an older version of xmlrpc.php in the web root. (I don't even use that thing...well at least somebody made good use of it. ) An early guess for my second hacked server is that somebody spoofed the DNS of it and intercepted an unencrypted mail password from my mail client. (The imap server only talks to the local LAN, so I didn't have it encrypted...didn't think about spoofing the DNS so my client volunteers my password.)

Oh well, live and learn. Time to change all my passwords and make sure my mail password doesn't match my ssh password and root password. And I'll encrypt even the local traffic from now on.

 

[BigMoneyJim]

I should clarify and emphasize that these were my personal, nonproduction servers. I'm far more paranoid and careful about work stuff.

The first server was wide open to the internet. I was a little less careful with the second one because it is mostly firewalled off.

 

[cute fuzzy bunny]

That'll teach you to change my thread titles!

 

[Marshac]

if your mail server only communicates with clients on the local subnet, then why would spoofing a DNS entry matter? Maybe Laurence can fill me in

So far my server ('kitchen'... what are you cooking?) hasn't been hax0red yet.... it's running Win2k3, Exchange 2k3, and BlueDragon (ColdFusion/JSP...none of that PHP for me....bleh)

Ever try tripwire?

I know that as soon as I hit the post button, the security log on my server will fill up

Edit: is your network at your house (ie: 100% yours), or is it shared with other people? perhaps someone poisoned your ARP table and performed a man-in-the-middle attack? Just a thought

 

[Cool Dood]

This one time, I was playing around with telnetd....

 

[laurence]

If it makes you feel any better, something similar happened to me recently (with my home network). And I'm supposed to be the computer security guy! What's that saying about cobbler's children feet go bare?

 

[BigMoneyJim]

if your mail server only communicates with clients on the local subnet, then why would spoofing a DNS entry matter? Maybe Laurence can fill me in

I knew I had phrased it confusingly. My orignal thought was: my server has an imap server. My mail client on my workstation is running. Local lan only, so no encryption. However, if the DNS suddenly pointed my mail client to a new server it sends the next "check mail" transaction complete with unencrypted password to the spoofed server. If they were smart enough to capture it and try it to ssh back into my server it would've worked (on my home server; on my web server none of the passwords are the same for accounts or mail access.) After poking around, I don't think that's what happened. In fact I don't think I was hacked...

So far my server ('kitchen'... what are you cooking?) hasn't been hax0red yet.... it's running Win2k3, Exchange 2k3, and BlueDragon (ColdFusion/JSP...none of that PHP for me....bleh)

I don't fully understand how they got my hosted web server. It's linux, and somehow (xmlrpc remote code execution vulnerability I presume) they changed the password for the daemon user and logged in interactively as daemon then su'ed to root. (I don't know how they did that; I must've had a privilege escalation vulnerability I haven't yet identified.) Luckily something got goofed up and my server dropped off the network. Actually I wonder if my host blocked my system off for spamming, but if so they didn't tell me they did. But the home server...

Ever try tripwire?

Not yet, but I'm going to real soon. The web server hack woke me up. If it hadn't have gone down I wouldn't have noticed the problem for a long time. My IP was already being bulkmail filtered by Yahoo when I got it; good luck on my ever getting off their list now.

Edit: is your network at your house (ie: 100% yours), or is it shared with other people? perhaps someone poisoned your ARP table and performed a man-in-the-middle attack? Just a thought

It's all mine. But it was my goof that compromised my home server. I have static IPs but added a private NAT'ted logical subnet. I accidentally enabled NAT both ways which effectively bypassed my main firewall rules *and* made all inbound packets appear to originate on my LAN relaxing my secondary firewall rules and leading the server to believe all inbound traffic was local. I wasn't actually hacked; it's just that my server effectively became an open mail relay through my own misconfiguration.

Even though I now think my logins were not compromised I'm going to rebuild and change passwords, anyway. And I'll have authenticated Submission protocol for mail submission and absolutely no SMTP relaying--even for the local LAN or even the local host. And IMAP will be encrypted, and passwords will not match between users & services.

The DNS thing still puzzles me. What alerted me to the problem was that my homepage quit working, and when I pinged the hostname it was pointing to a foreign server. It wasn't an ARP spoof, it was a DNS spoof, and they didn't seem to get into any local machine so I don't know how they managed that. It may be a side result of all inbound traffic appearing local. At the moment the DNS spoof appears to be coincidental, but it's a heck of a coincidence. Oh, this particular DNS is a DynDns entry, so they might have coughed up a wrong IP, too.

(edited for spelling...really need to get an in-browser spell checker for all the fat fingering I do)

 

[grumpy]

My son's blog describes a similar situation he dealt with:

http://zoomedin.blogspot.com/2006/01/spam.html

Grumpy

 

[laurence]

if your mail server only communicates with clients on the local subnet, then why would spoofing a DNS entry matter? Maybe Laurence can fill me in

So far my server ('kitchen'... what are you cooking?) hasn't been hax0red yet.... it's running Win2k3, Exchange 2k3, and BlueDragon (ColdFusion/JSP...none of that PHP for me....bleh)

Ever try tripwire?

I know that as soon as I hit the post button, the security log on my server will fill up

Edit: is your network at your house (ie: 100% yours), or is it shared with other people? perhaps someone poisoned your ARP table and performed a man-in-the-middle attack? Just a thought

Didn't see your post Marshac, geez, reading it I feel like I'm studying for my test again!

I highly recommend tripwire as well.

BMJ, your response was well written, that's a story I'll have to share at work!

You could get Draconian and use SUDO for everything...

Out of ignorance, since I'm more of a high level guy, how many password characters will your OS support? Some flavors will only recognize/encrypt 8 or even 5 characters and use weak algorithms, so while you may have a 14 character password with letters, numbers and special characters, the bad guys only have to solve the first part.

So you going to set up a DMZ?

 

[cube_rat]

You could get Draconian and use SUDO for everything...

SUDO is the only way to go. yeah, I'm draconian at work but not at home. I don't do IT crap at home very well.

 

[BigMoneyJim]

I highly recommend tripwire as well.

Any coments on the Open Source Tripwire compared to the commercial version and other alternatives? I figured on using the free open source one but have been searching for discussion on it versus alternatives.

BMJ, your response was well written, that's a story I'll have to share at work!

You could get Draconian and use SUDO for everything...

Thanks. And "sudo su -", now I'm root...heheh, maybe I'm not quite grasping the concept.

Out of ignorance, since I'm more of a high level guy, how many password characters will your OS support? Some flavors will only recognize/encrypt 8 or even 5 characters and use weak algorithms, so while you may have a 14 character password with letters, numbers and special characters, the bad guys only have to solve the first part.

I have md5 shadow passwords; they'll go over 8 characters...I'm not sure how high they'll go offhand, but my home passwords while not weak were not particularly strong.

So you going to set up a DMZ?

Ironically that's sort of what I was doing when I opened up the lan. Even though I have 5-8 static IP's (5 "usable", but part of my changes now let me use all 8 including my linux router/firewall) I decided to put machines behind NAT unless they need a publicly routable IP. That effectively makes my static IP range the DMZ.

(off to read grumpy's son's blog)

 

[laurence]

We are required to use commercial version, so I can't compare for you.

Well, you can limit what tasks a user account can execute with sudo, just add the accounts to the sudoers file and then they don't have to "know" the root password. Sudo is pretty granular, it doesn't have to be all or nothing on the priveleges (for example, my account at work will let me sudo, but I can't VI the password file, or even ls the shadow file). So the root password can me 28 characters and use all those hard password rules, then you can have a "emailadmn" account that can accomplish only certian tasks...well, I'm sure you know better than I.

 

[wabmester]

BMJ, I don't think anybody bothers sniffing passwords these days.   The script kiddies just get root through whatever exploit their scripts find.   Do you monitor CERT and the security mailing lists?   If you have a server on the public net, you just need to watch for the exploit of the day and dilligently patch them up.

Are you able to detect a port scan?    That's the first sign that you're about to be root-kit'd.

 

[laurence]

BMJ, I don't think anybody bothers sniffing passwords these days. The script kiddies just get root through whatever exploit their scripts find. Do you monitor CERT and the security mailing lists? If you have a server on the public net, you just need to watch for the exploit of the day and dilligently patch them up.

Are you able to detect a port scan? That's the first sign that you're about to be root-kit'd.

What I should have said.

 

[Marshac]

Anyone know of a cheap way to implement smartcard based authentication? On my workstation I have a keyboard with a nifty finger scanner, but if you want it to work with an AD domain- better cough up some dough. No thanks. It's just my house

Laurence- ever play with CAIN? You could conduct a "security audit" (yeah, that's the ticket) on Apr 1, and try an ARP poisoning attack... once you get in-between, you can rewrite DNS requests, and other fun things. Cain has a scary feature that makes it easy to grab login information for even encrypted websites.

 

[laurence]

I have been given it by my lead, but haven't messed with it yet. About once a month I set off our IDS at work and the network admins come yell at me. I just had my latest incident thursday with NMAP (port scan one lousy web server and everybody gets in a twist! ).

Only read about ARP poisoning attack, know how it works theoretically, it is very scary. I will have to try CAIN out in 29 days when I'm back off probation.

...or maybe I can notify my buddies first.

 

[BigMoneyJim]

I use nmap a fair bit at work. Very handy. I've caught and fixed a few vulnerabilities and troubleshot many problems. Corporate network control blocked my IP once, but have about 240 others to use (4 /26's).

 

[laurence]

You guys are the big dogs, I just live in your world. As a "systems engineer" I get a request from the customer like, "we want a high speed connection between site a and site b" and then I go distill requirements and reply, "so you want ATM or Gig-E? 1 megabit or 100 megabit?" etc. Then I go to hardware, software, network etc. and break off their respective chunks of requirements, they come up with how much time and material, and spit back at me, and I put together a nice little connection diagram, info flow diagram, hardware list, software list, CONOPS, etc. to throw at the customer, who changes it all around, and I start the process again. Bottom line, I "know of" and play with a whole mess of stuff, but I can't run with the big boys/specialists on anything.

That's why I keep quiet on the all the technical questions posted here. There is always some one who knows more about each specific aspect than I do, and better to keep silent and be thought a fool than open my mouth and confirm it!

 

[BigMoneyJim]

You guys are the big dogs, I just live in your world.

I'm not a big dog in pay, position or respect. Perhaps I need to promote myself better.

 

[Marshac]

I'm not a big dog in pay, position or respect. Perhaps I need to promote myself better.

It's not the size of the pay, position, or respect- it's the size of your IP block Mine is only a /29... you rule man. What do you use them all for?

I'm not a big dog either... i'm just one of those small shivering ones that bark a lot.... I wish my owners fed me better.

 

[BigMoneyJim]

It's not the size of the pay, position, or respect- it's the size of your IP block Mine is only a /29... you rule man. What do you use them all for?

Sorry, I'm being confusing again. The 4 /26's are at work and part of a couple of class B's and several class C's we have. (Big company.) And I'm running low on IP's and am short on switch ports. I'm over a local network of 100+ PCs, 20 or so printers and a bunch of other network devices of different sorts. Well, that's just my building; my workgroup whom I help out periodically covers several more buildings and an even more impressive array of subnets, VLANs and such. The big building has two routing networks, one of which runs production machinery and has two fully operational redundant networks. Everything on that network has at least one redundant link to the last switch...and perhaps to the network devices...I'll have to go double-check that as I don't usually touch that network. Even the computer systems that run production are fully redundant: if there's a critical error with today's operation they flip a switch and run on the other set of computers under yesterday's operating parameters and manually deal with any inconsistencies between the two days' operations. My bad restaurant service story was from when we were swapping out the core switches and patching in new gigabit fiber runs. 8) (My building still has a lot of 10Mbit and even has a dumb hub serving a few devices, but it's better than when I got here.)

The corporate network is fully contained on company leased lines. The only internet route is through corporate HQ. At work I don't have to worry about direct internet contact; just the occasional worm that gets lose in the network and users who try to bring in their own virus-and-spyware-infested laptops and PCs. Most of our network is publicly routable but well firewalled; we've outgrown our allotment so we're using private IP ranges in places. That's fun when you find a router that's still refusing to route private IPs.

I have a /29 at home but realized I can use all 8 publicly addressible IPs by calling it a /26 locally...I just lose access to 7 other /29's which are just other DSL home users and not likely ever to be used by me.

 

[Marshac]

I have a /29 at home but realized I can use all 8 publicly addressible IPs by calling it a /26 locally...I just lose access to 7 other /29's which are just other DSL home users and not likely ever to be used by me.

"ORLY?" that's a good idea.. never thought about doing that! Right now I only use two of the 8 IPs, but who knows...

I would have inserted the ORLY owl picture here, but he died

SanDiego, California (AP) — Jonathan, the Snowy Owl made famous as the “ORLY” owl, has died in captivity in the San Diego Zoo.

The owl, which was 17 years old, died Tuesday afternoon after the sudden onset of pneumonia. He was young for an owl in captivity, however wild owlsonly live an average of 15 years.

 

[BigMoneyJim]

Okay, the O RLY owl is new on me, but I caught up with Wikipedia.

Yeah, with the NAT I don't really need more than two at the moment, but just because I can....

Side note: All right, my $0.36 laptop power jacks came in today! Off to play with soldering irons...

 

[Nords]

Anyone know of a cheap way to implement smartcard based authentication?

I don't know if it's cheap, but the Navy Reserve has made their home users install ActivCard Gold smart-card readers on their PCs. It works with the chip in their CAC IDs. You can imagine who on our home tech support staff was placed in charge of the project.

Lemme rephrase that-- I know it's cheap, but I'm not sure how expensive it is...