"How to Handle and Identify Network Probes" - "Do you know what to do when suspicious network probes are detected on your network? It's surprising, but many people do not follow common sense and simple logic when analyzing malicious network activity. Even worse, when contacting other organizations to complain, security incidents can be misrepresented because all of the facts are not in order, incorrect or even erroneous theories. This paper details a variety of steps that you can take to get the most effectiveness and accuracy from your intrusion detection system. It also concentrates on determining the who, what, why, where, when and how of any network security event so that you can accurately relay this information to others."
8056f5f6db5b5bbbe1618e8241afeacea7523abaf7f66cd4434327c687d8bff4<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=windows-1252">
<META NAME="Generator" CONTENT="Microsoft Word 97">
<TITLE>How to Handle Network Probes</TITLE>
</HEAD>
<BODY>
<B><FONT SIZE=6><P ALIGN="CENTER"> </P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER">How to Handle and Identify </P>
<P ALIGN="CENTER">Network Probes</P>
</B></FONT><I><FONT SIZE=2><P ALIGN="CENTER"> </P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER">"What to do when your DNS server gets FIN-SYN </P>
<P ALIGN="CENTER">scanned from Russia at 4:00 in the morning."</P>
</I><P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P ALIGN="CENTER">By Ron Gula</P>
<P ALIGN="CENTER"> </P>
<P ALIGN="CENTER">April 1999</P>
<P ALIGN="CENTER"> </P>
<I><P ALIGN="CENTER">rgula@network-defense.com</P>
</I><P ALIGN="CENTER">http://www.network-defense.com</P>
<P ALIGN="CENTER">Copyright 1999 Network Defense Consulting</P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<P> </P><DIR>
<DIR>
</FONT><B><U><FONT SIZE=5><P>Abstract</P>
</B></U></FONT><P> </P>
<P>Do you know what to do when suspicious network probes are detected on your network? It's surprising, but many people do not follow common sense and simple logic when analyzing malicious network activity. Even worse, when contacting other organizations to complain, security incidents can be misrepresented because all of the facts are not in order, incorrect or even erroneous theories. This paper details a variety of steps that you can take to get the most effectiveness and accuracy from your intrusion detection system. It also concentrates on determining the who, what, why, where, when and how of any network security event so that you can accurately relay this information to others. </P>
<P> </P>
<B><I><FONT FACE="Arial"><P> </P></DIR>
</DIR>
<P>Introduction</P>
</B></I></FONT><FONT SIZE=2><P>This paper is loosely organized into a list of rules that can be applied to your network operation in the event of an external network scan. Each rule has several examples of what can go wrong and what can go right for a given situation. The rules are also in the order they should be applied in a network security situation. The last section discusses how to handle internal security events at a high level. Please use this paper as a guide. Think about it and what it means for your particular network. It may make the difference between deterring a network attack and having to respond to a network compromise.</P>
</FONT><B><I><FONT FACE="Arial"><P>Rule #1: Don't Panic</P>
</B></FONT><FONT SIZE=2><P> </P>
<P>It's 2:00 AM. You are in a deep sleep when suddenly the phone comes to life. Speaking to you is a late shift network operator who is frantically describing a list of ISS RealSecure events. The operator also describes that the firewalls are also going crazy and two NT machines just mysteriously rebooted. The VP of Operations has been notified and she is on her way in. What do you do?</P>
</I><P> </P>
<P>Even though network security events are reported in the media and they are a very serious threat, they are not likely to occur to you on a daily basis. (If they are occurring to you on a daily basis you must be pretty good at dealing with them by now and probably don't need this paper.) Most network organizations that suffer multiple attacks and probes experience them in groups. They are not sporadic events. </P>
<P> </P>
<P>With this in mind we can roughly classify network probes into two categories. First, the security event could actually be the result of a non-security event. This is known as a 'false positive'. In this case there is nothing to worry about. Second, the security event could have been a probe that tested your site for something. Tests could include determining the type of operating system of a server or even sweeping the network for open ports. If the probe turns up negative, there is a good chance that 'they' won't be coming back. With this situation, there is also little to worry about. At your leisure, you can pursue those responsible for the probe. If the probe seems to have found something that is vulnerable, then you may have returning visitors. Regardless of the outcome of the probe, it requires careful analysis and some judgement calls to determine its nature. That's what the rest of this paper is about. </P>
<P> </P>
<P>When presented with a security event, all you really know is that further investigation is required. However, knowing that these things don't happen that often shouldn't cause you to put the analysis of the event off to long. Timely analysis of any security event is the key to quickly finding the real ones from the false positives.</P>
<P> </P>
<P>Panic or at least an adrenaline rush is experienced by many network administrators when security incidents occur. Here are some rules of thumb to keep in mind when handling the situation. </P>
<P> </P>
<I><P>Initially, only tell people about the security event on a need to know basis</I> </P>
<P> </P>
<P>Telling one person who tells another can cause any office or operations center to quickly fill with people who are not the right ones to deal with the problem. They may also get in the way. Discretion is highly recommended.</P>
<P> </P>
<I><P>Watch out for overworked people</P>
</I><P> </P>
<P>When any network event occurs, there is a tendency for normal people to rise to the challenge and work long hours to see the event through. Security events are no different. If an event occurs at the end of a normal duty day or a shift, people working extra hours may suffer from fatigue, irritability and hunger. All of these can impair the judgement of any person. It may also endanger them for their ride home. Later on we discuss the importance of documentation. Documentation and tracking of a security event can make change over between employees much easier.</P>
<P> </P>
<I><P>Don't let people jump to conclusions</I> </P>
<P> </P>
<P>During high pressure situations, some people may feel the need to blame others in an attempt to find answers. It's important to downplay any of these comments until all of the facts are considered.</P>
<P> </P>
<I><P>Get second opinions on 'rash' decisions</I> </P>
<P> </P>
<P>When conducting a security investigation, it is very important to get input from peers and even management about your current theories and plans. For example, it may seem like a good idea to take the corporate web server offline for analysis, but a second opinion might ask you to stand up a replacement. If probes are occurring in real time, it is also temping to take certain courses of action such as 'hack backs', setting up of honey-pots or even trying to slow the scanning down. All of these actions may have unintended repercussions on your company or network.</P>
<P> </P>
<I><P>Focus on any obvious explanations</I> </P>
<P> </P>
<P>It may not seem obvious, but suspicious activity can be explained away most of the time with normal network events. Consider recent network changes or scheduled network testing when analyzing a security event.</P>
<P> </P>
<P>Like it or not, as the network security guru, you are also in a position of leadership during security events. If you are not sure of yourself, panic stricken or exhibiting a high level of stress, these traits can easily spread to other people. In ideal situations, the network security staff (if there are more than one of you) is regularly involved in network operations. Knowing your co-workers and making sure that they know you will reduce stress and panic during security events.</P>
<P> </P>
<I><P>"Don't worry", you say to the late shift operator, "I will dial in and check the logs. Tell Beth I'll give her an initial status report in about fifteen minutes. If it looks bad, I'll be coming in." </P>
</FONT><B><FONT FACE="Arial"><P>Rule #2: Document Activity</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<I><P>It's Monday morning and you receive a call from the Vice President of Information Security. He wants to know how many network probes we have received over the last three months and if any of them came from our competitors. What would you tell him?</P>
<P> </P>
</I><P>When any network security event occurs, you should document it. It doesn't matter how you record the information as long as it is secure, accurate and can be stored for some time. I like to keep a log book, but log books can be lost. Some network shops record suspicious logs directly to CD-ROM. More and more network shops are using ticketing systems to track security events. These have the advantage of existing in a database which can easily be searched for event correlation. You need a solution which is right for you. </P>
<P> </P>
<P>Why document? There are several reasons: </P>
<P> </P>
<I><P>People forget, even security gurus</I> </P>
<P> </P>
<P>Having a physical record of security events from days gone past is an immense help when analyzing security events of today and tomorrow. Being able to answer questions like, "Has this IP address ever scanned us before?", or "How many other IMAP probes have we had this year?" can only be answered by reviewing historical logs. </P>
<P> </P>
<I><P>Security systems may not keep logs forever</I> </P>
<P> </P>
<P>Some security programs do not keep logs forever. They delete old logs to make room for new ones. If you have a system such as this, then those security events from last month might not be in your security system any more. Keeping logs separate from the production systems ensures a greater level of data protection also. What happens if you have a hard drive crash? Many security systems do not use back-ups for data integrity. </P>
<P> </P>
<I><P>It might be evidence in court</I> </P>
<P> </P>
<P>If you keep good logs (and good is subject to lawyer's interpretation), then those logs may be used as evidence in a court case. It is very important to keep a 'chain of evidence' with any log system. This means you need to have proper access control on the log information. If the security or integrity of the log can be questioned, then the log may not be admissible in court. Paper print outs and CD-ROMs tend to be more believable than electronic media. Even logging of DNS names instead of IP addresses may be an issue, since DNS name resolution can be corrupted. </P>
<P> </P>
<I><P>It helps you sell security</I> </P>
<P> </P>
<P>If you are like most companies, network security is viewed as an important but expensive thing. Firewalls, intrusion detection systems and conferences to Black-Hat are all expensive. Keeping logs can help show management that there is indeed a threat and they are getting their money's worth from you and the fancy security equipment.</P>
<P> </P>
<P>Final thoughts:</P>
<P> </P>
<P>During the heat of the moment is when it is most important to write down or record any information about a security event. Don't forget to record the people involved, their phone numbers and what they have said. Recording all of the information allows for more efficient processing of the data once it is assembled together. </P>
<P> </P>
<P>Later on, when things are less hectic, it is good practice to write up the security event in a one or two page paper. Sharing this paper with any lessons learned can have a very positive effect on your entire network staff. Keeping records of these reports can also help you and possibly your successor. </P>
<P> </P>
<I><P>"I can have a report for you this afternoon," you tell the VP of Information Security. After hanging up, you leave your Quake 2 session and start to work on the report. You utilize the last three monthly security incident reports and look at some of the more interesting recorded logs from those events. You conclude that your network was probed four times, all from Asian IP domains, but never from your competitor's address space. You deliver the report to the VP and emphasize where the information came from and how the security staff is responsible for maintaining it. </P>
</FONT><B><FONT FACE="Arial"><P>Rule #3: Identify Activity</P>
</B></FONT><FONT SIZE=2><P> </P>
<P>While looking at the intrusion detection logs, you observe a variety of TCP packets going only to your DNS servers. The TCP FIN and SYN flags are set in each packet. The destination ports for the packets are ports 0, 21, 143 and 2049. None of those ports are active. What is going on?</P>
<P> </P>
</I><P>Depending on the situation and the available information, it can be very difficult to get a clear picture of all aspects of a security event or network probe. Distinct events may not seem related until another piece of the puzzle is added for clarity. Attempting to answer the basic questions of who, what, why, when and where is a good place to start and can provide you with a framework to paint a picture of what has transpired.</P>
<P> </P>
<B><U><P>WHO?</P>
</B></U><P> </P>
<P>If this is a network security event, then you are probably obtaining network data from an intrusion detection system, a firewall or some other network element. In most cases you know the source and destination IP addresses involved. This is the 'who' and there are a wide variety of tools that can be used to glean information about the owners of the addresses.</P>
<P> </P>
<I><P>nslookup </P>
<P> </P>
</I><P>This tool is available on NT and UNIX. The command can convert IP addresses to DNS names and vice versa. It is usually a good place to start to get some quick information about a suspicious IP address. Some care should be taken though. DNS information can be spoofed. One of the neatest hacks is to modify DNS answers to throw network security investigators off of your trail. DNS queries may also be observed by the network attacker. This may alert them that their scanning has been discovered. Many ISPs have taken to naming their dial-up PPP or SLIP addresses with the word 'dial', 'ppp', 'modem' or 'slip'. If you see these, there is a good chance that the source of the scan is a modem user. Similar assumptions can be made for 'home.com' names in which those IP addresses are assigned to cable modems. Consider obtaining a shell account well away from your corporate network to conduct DNS lookups. This may not alert the scanner that you have detected their activity.</P>
<P> </P>
<I><P>whois </P>
</I><P> </P>
<P>Once you have a DNS name, you can query the 'whois' database to find out contact information for that domain. When domains are registered, they usually require some sort of contact information to include a phone number, an email address and a name. Don't underestimate that the name in the 'whois' database could be the fellow doing the scanning. It's unlikely, but it has been known to happen. There are a variety of web interfaces to the 'whois' database and UNIX clients have a built in 'whois' command. Here is example 'whois' output for network-defense.com which has been slightly modified:</P>
<P> </P><DIR>
<DIR>
</FONT><B><FONT FACE="Courier New" SIZE=2><P>[root@gigan /root]# whois network-defense.com</P>
<P>[rs.internic.net]</P>
<P> </P>
<P>Registrant:</P>
<P>Company Name (NETWORK-DEFENSE-DOM)</P>
<P> 9305 Sun Down Pl</P>
<P> Nowhere, MD 21047</P>
<P> US</P>
</B></FONT><FONT SIZE=2><P> </P>
</FONT><B><FONT FACE="Courier New" SIZE=2><P> Domain Name: NETWORK-DEFENSE.COM</P>
<P> </P>
<P> Administrative Contact, Technical Contact, Zone Contact:</P>
<P> Gula, Ron (RG15449) rjgula@HOME.COM</P>
<P> 410-212-9898</P>
<P> Billing Contact:</P>
<P> Gula, Ron (RG15449) rjgula@HOME.COM</P>
<P> 410-212-9898</P>
<P> </P>
<P> Record last updated on 24-Nov-98.</P>
<P> Record created on 24-Nov-98.</P>
<P> Database last updated on 7-Apr-99 12:28:52 EDT.</P>
<P> </P>
<P> Domain servers in listed order:</P>
<P> </P>
<P> NS.AUTONO.NET 209.48.2.11</P>
<P> NS10.AUTONO.NET 206.86.247.30</P>
</B></FONT><FONT SIZE=2><P> </P></DIR>
</DIR>
<P>It can be seen here that there is a lot of information that can be used for documentation and contact purposes. There is a home address in case you want to launch Tomahawk missiles and a telephone number for voice contact. There is also an email address and a listing of which root level DNS servers 'take care of' the queried domain. </P>
<P> </P>
<I><P>arin </P>
</I><P> </P>
<P>The ARIN database is publicly available at http://www.arin.net/whois/arinwhois.html and allows one to find out the owners of a particular IP address. When DNS has not offered an answer to what a search, doing a reverse IP lookup at ARIN can still provide useful information. Here is an example ARIN search output for 24.3.17.92 which is a home.com IP address:</P>
<P> </P><DIR>
<DIR>
</FONT><B><FONT FACE="Courier New" SIZE=2><P>@Home Network (NETBLK-ATHOME) </P><DIR>
<DIR>
<P>ATHOME 24.0.0.0 - 24.7.255.255</P>
<P> </P></DIR>
</DIR>
<P>@Home Network (NETBLK-MD-COMCAST-HWRD-1) </P><DIR>
<DIR>
<P>MD-COMCAST-HWRD-1 24.3.16.0 - 24.3.23.255</P>
</B></FONT><FONT SIZE=2><P> </P></DIR>
</DIR>
</DIR>
</DIR>
<P>This query provides us with some interesting information. First, we know that the IP address is part of the home.com (@Home) ISP. A lot of hackers get cable modems. A lot of hackers hack people with cable modems. The second pieces of information tells us that @Home is using the Comcast Cable company for local access. One could even venture a bet that the 'MD' referred to Maryland and the 'HWRD' referred to Howard county. It's dangerous to assume things like this, but in some cases it makes sense. For instance, not every net block that has a 'TX' in it is from Texas. Use good judgement.</P>
<P> </P>
<I><P>The Web</I> </P>
<P> </P>
<P>If you know the IP address of a network probe source, you may want to try to look for web servers on that network. An easy way to do this is through guess work. Let's say a target's DNS name resolves to <I>name1.place.com</I> for an example. Attempting to go to the web server at <I>www.place.com</I> may provide you with the network's public web server. In some cases, using a tool such as nmap to scan a class C network for <I>any</I> web server can be fruitful. This should only be done as a last resort. In ISP networks, scanning a class C network may not bring you any closer to information about the scanner as you connect with one unrelated user's web page after another. The goal here of course is to discover some sort of contact for you to voice your distress over the network scanning.</P>
<P> </P>
<P>It is also very important to understand exactly 'who' locally is involved in the scanning. Recording IP addresses is not enough. What is the second level relationship between them. Is this a sequential scan? Are these systems mail systems? Do they run IRC daemons? Are they all DNS servers? You get the picture. Figuring this out may allow you to understand what drew the network scanners to your network in the first place.</P>
<P> </P>
<B><U><P>WHAT?</P>
</B></U><P> </P>
<P>Determining what a network scanner is probing for can be a very difficult activity. I offer some general guidelines to analyze suspicious activity, but no one can be certain exactly what the intention of a particular scan is without asking the person doing the scanning. More importantly, we discuss a variety of scanning techniques and the type of information that the scan returns. Using this to analyze activity on your network can be interesting. We do not consider techniques for direct vulnerability probing because this is a very cumbersome topic and a lot has been written about it already.</P>
<P> </P>
<I><P>Topology Mapping</P>
</I><P> </P>
<P>Much suspicious network activity concerns discovery of target network and systems by malicious individuals. Sophisticated attempts exist that can map out a network. Attackers equipped with knowledge of a network's hierarchy can plan effective attacks. Here are some common and not so common topology scanning techniques that you may observe on your networks.</P>
<P> </P><DIR>
<DIR>
<I><P>ICMP Sweeps</P>
</I><P> </P>
<P>ICMP packets may be used to determine if a target IP address is alive or not. Typically, a scanning program may send an ICMP ECHO REQUEST packet and anticipate an ICMP ECHO RESPONSE. When multiple hosts are queried this way, it is better know as a 'ping sweep'. If the host isn't there, there is no response. Firewalls and routers can be used to block this sort of traffic. This type of scan can be directed at one host, or many. It can also be spread out over time such that a target network may not become alarmed that it is being scanned.</P>
<P> </P>
<P>More advanced ICMP scanning techniques make use of non-ECHO ICMP protocols. There are a wide variety of ICMP protocols besides ECHO. These include support to request timestamp and netmask information. Many firewall and packet filter designers forget to block all ICMP traffic and only filter ECHO traffic. In this case, making non-ECHO requests is still a valid form of host identification.</P>
<P> </P>
<P>The ICMP protocol can also be used with broadcast addresses. Typically associated with ICMP denial of service attacks, ICMP broadcast packets may be able to map out large portions of a network with one packet. </P>
<P> </P>
<I><P>TCP/UDP Sweeps</P>
</I><P> </P>
<P>Most people associate TCP and UDP scanning with determining which ports are available on a particular network server. The reality is that responses from any port may indicate that an IP address is active. Sending a UDP or TCP packet to a high port and receiving a response is a good indication that something is there. Exactly what comes back depends on the target operating system, the packet sent and any firewalls or packet filters.</P>
<P> </P>
<P>TCP packets have flags that indicate which part of a TCP conversation they are in. Typically, TCP sessions start with a SYN packet from a client to a server. This is followed by a SYN-ACK packet from the server to the client if the target service (or port) is active. If it isn't, then the server responds with a RESET packet. Both the SYN-ACK and the RESET packets can be used to identify active IP addresses by network scanners. It should be noted that some firewalls can spoof a RESET packet from an IP address, so that technique may not be that reliable.</P>
<P> </P>
<P>UDP scanning is tougher to do than TCP for a variety of reasons. First, UDP packets can be dropped by routers as they cross the Internet. This is true! Try it for yourself! (Use a program to send UDP packets across the net and see how many actually get there). Second, many UDP services <I>don't</I> respond when correctly probed. It's the response of an ICMP PORT UNREACHABLE message that identifies a UDP port that is not active. Considering that UDP packets may be dropped, and firewalls may also be configured to drop them, UDP scanning may seem very unreliable. UDP scanning does have an advantage of being able to make use of IP broadcast addresses. A network that allows UDP packets could be mapped by sending a packet to the broadcast address on a high port. If the port were not filtered, then the scanning node would receive ICMP PORT UNREACHABLE messages from many target network systems.</P>
<P> </P>
<I><P>SNMP Scanning</P>
</I><P> </P>
<P>The Simple Network Management Protocol (SNMP) is used and supported by a wide variety of network elements and network management software. Modern hubs, switches and routers all support SNMP. Many servers such as Solaris and Windows NT also have SNMP functionality. SNMP has several versions, the most common of which is version 1. Security in version 1 is handled by a clear text community string that functions as a password. Any SNMP packet must have the proper community string. Without it, the packet is rejected. </P>
<P> </P>
<P>The problem with SNMP is that all network vendors ship their products with default community strings of 'public'. Any scanner that has access to SNMP enabled network nodes simply uses the 'public' community string and starts to send queries. Data obtained over SNMP can completely diagram a network and include other information such as CPU type, firewall rule configurations and even web server performance. Tools exist that help attackers brute force community strings. </P>
<P> </P>
<I><P>Obtaining Router Configurations</P>
</I><P> </P>
<P>Another way to map a network is to get a copy of some router configuration files. With enough different router configurations, a network scanner can map your network without sending any packet probes. I've even been on some penetration tests where a network organization has published there router configurations publicly on a web server! This information should be protected. </P>
<P> </P>
<P>Many routers and network elements such as hubs and switches also have vendor passwords that are used for diagnostics and maintenance. These passwords are well known to network crackers and can easily be used to obtain configurations, which in turn can be used to map a network.</P>
<P> </P>
<P>Some routing protocols allow for polling of route information. RIP is a classic example of this. With RIP, any client can query another network device to obtain the routing table. </P>
<P> </P>
<I><P>Time To Live</P>
</I><P> </P>
<P>Almost every one has used the 'traceroute' program. This program discovers the number of hops between IP addresses. It does this though the use of the IP time-to-live value. Every time a packet is routed, this value decrements by one. When the value is zero, the packet is discarded and an ICMP error message is returned to the sending IP address. The TTL prevents packets that are misrouted from floating around on the Internet forever. By purposely sending out packets with low TTL values and watching for the ICMP error messages, automated programs can be used to discover how many hops (routers) there are between them and a particular IP address.</P>
<P> </P>
<P>When attempting to map the topology of a remote network, combinations of 'traceroute' attempts can provide a good picture of how the network is put together. Attackers can use this information to find sub-networks that may be weakly defended and possibly exploit trust relationships. </P>
<P> </P>
<I><P>Scanning with non-standard IP Protocols</P>
</I><P> </P>
<P>Another technique to map out a network and identify live hosts is to send IP packets to target networks that are non-standard protocols. Lets assume that the 'standard' IP protocols are ICMP(1), TCP(6) and UDP(17). Most firewalls and packet filters designers tend to focus on these protocols when designing firewall rules. They may inadvertently allow traffic from other protocols. There are 128 possible IP protocols. Experimenting with different operating systems may show a way to remotely identify certain types. If so, then a network scanner may be able to illicit a response from a target IP address by sending in this sort of 'non-standard' traffic. The response will most likely be an ICMP PROTOCOL UNREACHABLE message. Some of these protocols may also be sent to the broadcast address.</P>
<P> </P>
<I><P>Multicast Packets</P>
</I><P> </P>
<P>The last example of scanning to discover a target network's topology is by exploiting multicast packets. Multicast IP addresses have been set aside by the Internet community and are handled different by routers and servers. With multicast packets, one IP address can theoretically send the same packet to many other IP addresses. If a network scanner is 'upstream' from you, they may be able to send multicast packets into your network for mapping purposes. The scanner needs to be upstream so that the multicast packets go only to the target and not to the rest of the Internet. Being 'upstream' is required so the scanner can correctly spoof multicast packets only to your network. Some packet filters and servers ship with multicast functionality enabled. This can be exploited by remote network scanners to discover live hosts. </P>
<P> </P></DIR>
</DIR>
<I><P>Remote Operating System Identification</P>
</I><P> </P>
<P>Another piece of information that network probes attempt to discern is the type of operating system of a given IP address. There are a variety of methods that exist to do this and we discuss them here. These scanning techniques are very popular in the hacker community.</P>
<P> </P><DIR>
<DIR>
<I><P>Banner Grabbing</P>
</I><P> </P>
<P>If a system is not secured behind a firewall or through disabling of network resources, then most services can be used to identify an operating system. The TELNET banner is a classic example of this. Almost all TELNET banners have a very distinct look about them and actually say what the operating system is. Other services such as mail transfer agents can identify the operating system too. </P>
<P> </P>
<I><P>DNS names</P>
</I><P> </P>
<P>If you observe many DNS queries, a remote network scanner may gain knowledge of your operating systems if you've named them descriptively. Many DNS schemes include the operating system. Examples such as 'node123-w95.example.com' and 'nt4-102.example.com' could name Windows 95 and Windows NT systems. </P>
<P> </P>
<I><P>TCP Trickery</P>
</I><P> </P>
<P>A technique that uses distinct variations in TCP stack implementations to determine the type of remote operating system is known as TCP fingerprinting. The basic concept of this probe is to send specific TCP packets to an IP address and observe the response. The TCP traffic sent is a rather odd combination of destination ports and flag combinations. TCP responses are also considered in the identification process. These responses include sequence number randomness and initial window sizes. There are probably other techniques. Several tools exist such as Queso and Nmap that have a large database of responses to these odd TCP probes and can be used to reliably identify servers and network elements.</P>
<P> </P>
<P>Some of the more common techniques you may see are TCP probes that have the FIN and SYN bits set. This combination does not occur naturally. (At least I have not observed it). Other flag combinations used are nulls which have no flags set and SYN-RESET. Both of these scan types occur naturally in a variety of network traffic such as normal web browsing. This is important to consider because every TCP packet that has no flags set is not part of an operating system probe. </P>
<P>Consider the source. Strange TCP packets to high volume DNS(53) and WEB(80) servers may be explained. But a few stray packets to low volume ports such as IMAP(143) and NFSD(2049) should be viewed as suspicious.</P>
<P> </P>
<P>One of the more advanced scanning techniques I've come across is the use of a bogus 'error' packet. The packet is TCP with a normal IP header. All of the TCP data is identical except for the TCP flags. For example, the TCP packet could entirely consist of bytes with the value of '0x4e'. This would correspond to a source and destination port of 20046. But for the TCP flags, the correct bit values for a FIN-SYN or a SYN-RESET would be used instead of '0x4e'. If the packet is recorded, it may look like an error packet and be ignored by a network analyst. But it could be performing remote operating system identification.</P>
<P> </P>
<I><P>Standard services</P>
</I><P> </P>
<P>When trying to identify a particular remote operating system, another technique used is to probe for specific combinations of ports. These port combinations can reliably identify the target platform. Testing for DNS or SMTP services are not distinct enough for scanners to test for because they are very common. However testing for servers that have IMAP(143) and NFSD(2049) active may indicate the Linux operating system. Solaris servers have a variety of RPC services that are enabled by default. The same goes for HP-UX and SGI. Network scanners who know these combinations can identify target systems this way.</P>
<P> </P>
<I><P>Peculiar Behavior</P>
</I><P> </P>
<P>Some network nodes may exhibit very odd or strange protocol behaviors. This can best be illustrated with an example. Cisco routers communicate with each other on port 1999. During the three way TCP handshake, the Cisco router will identify itself by placing the word 'cisco' in the data portion of the SYN-ACK packet. This is a very trivial way to identify Cisco routers. Knowledge of this type of behavior can help discreetly identify remote systems.</P>
<P> </P></DIR>
</DIR>
<I><P>Port Scanning Techniques</P>
</I><P> </P>
<P>Many network probes are attempts to discover active services on a network or on a particular server. Typically, these scans are automated and connect with ranges of ports for a given IP address. They then report the ports that were open or active. Network attackers can then select exploits based on the active ports found. Here are some different port scanning techniques that you may encounter when analyzing network probes.</P>
<P> </P><DIR>
<DIR>
<I><P>Slow Scanning</P>
</I><P> </P>
<P>Since typical port scans can show up in system logs (Sendmail, TCP wrappers, Klaxon, etc. ) , network scanners can simply spread out the scan over a long period of time. Determining if a single packet is part of a larger port scan is very difficult if the other packets aren't there. Depending on the level of network activity, it may be possible to avoid detection simply by delaying scan packets for one minute. </P>
<P> </P>
<I><P>Random Scanning</P>
</I><P> </P>
<P>Again, another way to avoid port scan detection is to randomize the order in which the ports are tested for. Many commercial IDS products and firewalls watch for sequential connection attempts. They may have a threshold of common sequential destination ports and when this threshold is crossed, a port scan is reported. Randomizing the port testing avoids exceeding the sequential destination port threshold. </P>
<P> </P>
<I><P>SYN Scans</P>
</I><P> </P>
<P>A SYN scan is yet another attempt to bypass system level port scan detection. On most systems, a network connection is only recorded if the TCP three way handshake is completed. SYN scans send a single SYN packet to the destination port and then wait to see the response. If the response is a RESET packet, then the port is dead and no further action is taken. If the response is a SYN-ACK packet, then a RESET packet is sent back to the target that disengages the three way handshake. Sometimes this RESET packet is generated by the SYN scan program and other times it is generated directly by a response from the scanning operating system's IP stack.</P>
<P> </P>
<I><P>Spoofed SYN scan</P>
</I><P> </P>
<P>A trivial modification to the SYN scanning technique is to completely spoof the SYN packet from another IP address on the scanner's network. The spoofed IP address has to be one in which the return traffic from the server will flow past the scanner. It sniffs the network traffic to discover which ports are active. If a scanner is 'upstream' from a spoofed IP address (for instance in a DMZ in front of 10,000 computers) then it can be very hard to trace. The spoofed IP address can be from a machine that is alive or dead. </P>
<P> </P>
<P>This technique can also be used to generate many other simultaneous scans from other IP addresses. A defending network would perceive that it is being scanned by several different networks. The extra data can be missed by IDS nodes and can also be very hard to analyze. Determining the real IP address responsible for the scan is possible in some cases, but usually not. One way to tell if you are the victim of spoofed scans is to check for similar time-to-live (TTL) values in each of the scanned packets. If all of the incoming packets have the exact same TTL values, then this is suspicious. Conducting a 'traceroute' to each of the IP addresses may allow you to eliminate some of the spoofed sources. Some network scanners such as Nmap randomize their initial TTL settings with a value between 51 and 65. </P>
<P> </P>
<I><P>Fragmented Scan</P>
</I><P> </P>
<P>In an attempt to hide their probes, network scanners may also fragment their packets. All IP packets that carry data can be fragmented. For TCP packets, fragmentation can occur in which the destination port is in one packet and the flags are in another. Network IDS nodes may incorrectly reassemble or completely miss portions of the scan. Fragmentation that places ports in one packet and flags in another is something that does not occur naturally on IP stacks. </P>
<P> </P>
<I><P>Proxy Scanning</P>
</I><P> </P>
<P>If a protocol or service can be exploited by a network scanner such that the service can make arbitrary network connections, then the protocol can be used for port scanning. Some proxy servers and most FTP daemons can be used to conduct port scans. The classic example is the FTP Port Scanner in which a surrogate FTP server is used to make many network connections to a target system. The FTP protocol allows for the client to specify which IP address and port the server should send data to. Information returned by the FTP server can be used to identify open or closed ports on the target system. </P>
<P> </P></DIR>
</DIR>
<I><P>Finding Targets of Opportunity</P>
</I><P> </P>
<P>Some scanning is only focused on one thing, finding vulnerable systems. This type of scanning is subjective, but basically involves a lot of automation. There are different strategies used by network scanners to find vulnerable systems. Here are a few of them:</P>
<P> </P><DIR>
<DIR>
<I><P>Wide Scanning</P>
</I><P> </P>
<P>Very simply, a network scanner will scan large sets of IP addresses for a particular service or set of services. Scanning usually encompasses 'Class B' ranges of IP addresses. This type of scanning can be identified by two factors. One, the scanning is very sequential. It is so sequential that computers not part of the range scanned don't see any traffic from the scanning host. Second, follow on activity from the scanning host usually doesn't happen for some finite amount of time. This could be a day or a few hours. It reflects the notion that a scanning program takes a long time to complete. Once it is complete, the person running the scanner usually starts to test out the data. </P>
<P> </P>
<I><P>Finding vulnerable servers using that service</P>
</I><P> </P>
<P>When looking for vulnerable servers, many exploitable services have their own system of organization. DNS is a classic example. If a hacker wants to find a list of DNS servers, there are a variety of tools and databases that can be utilized. The same goes for IRC, Usenet News and web crawlers. Probes that occur on your network may be the result of chance or they could be happening simply because you have that service! Your service is actually part of a larger network of services on the Internet. Later on we consider what draws hackers to one network over another.</P>
<P> </P></DIR>
</DIR>
<I><P>Access Control List Mapping</P>
</I><P> </P><DIR>
<DIR>
<I><P>Fire-walking</P>
</I><P> </P>
<P>Very recently, a paper was released that detailed a technique dubbed 'fire-walking'. This technique combines port scanning and 'traceroute' tools. This tool analyzes the aggregate protocol map allowed to a particular host. In other words, this allows remote users to map out a particular set of firewall rules or access control lists. Knowing what sort of packet filter or firewall rules are present, can help an attacker plan their exploits.</P>
<P> </P>
<I><P>SNMP</P>
</I><P> </P>
<P>SNMP queries may be inadvertently allowed directly to firewalls and packet filters. If this condition is true, then remote network scanners could be able to obtain the exact filter rules for your network. </P>
<P> </P></DIR>
</DIR>
<I><P>Direct RPC Scanning</P>
</I><P> </P>
<P>Normal RPC scanning (rpcinfo -p) sends a query to the rpcbind program which is more commonly known as the portmapper. This query can ask for a list of all other RPC programs on the server. RPC programs historically have always existed around port 1024, or usually below that. Several years ago, Solaris and many other flavors of UNIX started to run RPC programs around port 32771. Many packet filter and firewall designers were unaware of this situation and deployed access control lists that did not prevent connections to these ports. </P>
<P> </P>
<P>In the case of 'normal' RPC behavior, it is possible for an RPC program to be assigned a port slightly above port 1024. If the firewall rules do not prevent this sort of connection, then the RPC service is directly accessible from external IP addresses. The same goes for RPC programs that are assigned ports above port 32771. These are a problem because they may be directly connected to.</P>
<P> </P>
<P>Network scanners may search for these 'high' RPC ports by using port scanning to identify ports and then conduct RPC queries to any ports that are open. If an RPC service is identified, it may have a vulnerability that can be exploited.</P>
<P> </P>
<B><U><P>WHY?</P>
</B></U><P> </P>
<P>Figuring out 'why' a particular suspicious network event occurred can be a very challenging and daunting task. The important thing to realize (and sometimes this only comes through experience) is that some things can't be explained. When an explanation seems to elude you or your staff, don't let it consume more and more resources. Prioritize your investigation and don't let it be hindered by not understanding exactly why something happened.</P>
<P> </P>
<P>For example, a server may have been rebooting sporadically and your staff suspects that a new denial of service attack is being used. Sniffing, intrusion detection and system security analysis indicate otherwise. Finally, a maintenance person discovers a faulty power supply. This example may seem trivial, but occurs many times on modern networks. </P>
<P> </P>
<P>Another example of a 'why' explanation is to consider the source of the security information. Many network management intrusion detection systems are complex and have many threshold levels for causing alarms to trip. If these threshold levels are drastically changed, then all of a sudden there may be many system alerts and possible intrusions. Try to identify any recent system changes that could attribute such activity. </P>
<P> </P>
<P>Unfortunately, all suspicious network activity can not be ruled out. Network probes should be considered hostile until you know exactly what you are dealing with. Why would someone want to break into your network? You tell me! There are many reasons for breaking into networks. Are they looking to deface a web page? Do you have political enemies that are network savvy? The good news is that you may have detected their early network probing efforts and now you can act accordingly. </P>
<P> </P>
<B><U><P>WHEN?</P>
</B></U><P> </P>
<P>Knowing when something happened allows you to construct a timeline or order of events. The order of events may be very crucial for determining the exact steps taken by hackers using network probes. Did they attempt a DNS zone transfer before we deployed the new DNS server? Did the scan on the internal web server occur after the firewall changes? These are example questions that depend on accurate time reporting.</P>
<P> </P>
<P>One key to determining the order of events is to use a common time source. The Network Time Protocol (NTP) is very reliable, accurate and resistant to a variety of attacks. NTP can be used to synchronize all network and server nodes with a very accurate and uniform time source. With all of them synchronized, log analysis becomes a lot easier. I also recommend trying to keep all of your systems on the same time zone clock. If you are unlucky enough to have servers in multiple time zones all keeping local time, then log analyses can become very cumbersome. With one unified time clock, it may be easier to detect network wide probes of your systems.</P>
<P> </P>
<P>Having a good and reliable time system is also crucial if you want to enter any of the logs into a court case as evidence. </P>
<P> </P>
<B><U><P>WHERE?</P>
</B></U><P> </P>
<P>When analyzing network probes, the question of 'where' is often overlooked. We are referring to the physical and electronic location of all of the computers involved, including the security systems. Identification of system locations is important to help understand and analyze recorded data. It may also explain why some information is missing or inaccurate. </P>
<P> </P>
<P>Confirmation of physical and logical location is often necessary when conducting an investigation. Your network maps may not be up to date. There have been cases when a simple network connection mistake has placed a vulnerable server outside of a protecting firewall. It may be helpful (and surprising) to obtain a remote account and conduct network mapping probes to see how things are connected.</P>
<P> </P>
<P>The use of Ethernet addresses can also be of great use when identifying the sources of spoofed IP packets. Although Ethernet packets can be trivially spoofed locally, they can't be spoofed across the Internet. This can help you determine which router or switch interface spoofed packets may have originated from.</P>
<P> </P>
<I><P>Analysis of the IP packets indicate an automated probe of only the DNS servers. Other ISP security contacts report their DNS servers have been scanned for similar ports. You theorize that the port 0 is being used to remotely identify LINUX servers. This theory is further corroborated when you also realize that ports 21, 143 and 2049 have all had recent LINUX remote buffer overflow attacks published. </P>
</FONT><B><FONT FACE="Arial"><P>Rule #4: Determine if you are vulnerable</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<I><P>Continuing with this scenario, you do a quick inventory check of the DNS servers and discover that one of them is a brand new LINUX install. The server is on the other side of the country and they won't be up for another four hours. What do you do?</P>
</I><P> </P>
<P>Every network security program should conduct regular vulnerability testing using commercial and free network security tools. It is one of the best ways to determine if you are at risk to common network attacks. But what if someone is probing you on a port that you have never heard of? What if they are probing you on a port that you thought had no security problems? There are several things you can do. </P>
<P> </P>
<P>The first thing to do is to identify the port if it is not well known to you. Set up a network analyzer and collect some traffic on that port. Analysis of the traffic may help identify it. There are a variety of proprietary network protocols such as PC Anywhere. These may seem very strange and unfamiliar to you. Another technique you could try is to take an inventory of all of your network software. This is unrealistic in a short time frame, but usually identifies a variety of programs (and protocols) that could cause the traffic you are looking for. If all else fails, consult the Internet news groups. There is a lot of open discussion about network protocols and you may be lucky enough to find one about yours. </P>
<P> </P>
<P>For example, when I first got my cable modem, I saw a variety of traffic to my computer that was UDP and port 22. All of the packets contained two bytes of ASCII data that were 'NQ'. I had no luck finding out what the protocol was until I stumbled onto a firewalls discussion list that was talking about PC Anywhere. </P>
<P> </P>
<P>Once you know the target, try to determine the threat. Again, this is very subjective. I try to look for recent vulnerabilities described in the various network security groups on the Internet. Recently, I've begun to observe scanning for port 21 on a variety of firewalls and intrusion detection platforms I have access to. Prior to this about three weeks ago, there were several posts that could remotely exploit the Washington University FTP daemon. It makes sense that people are looking for FTP servers to attack because this is a new vulnerability.</P>
<P> </P>
<P>Of course, the only way to know that you are vulnerable is to actually test the problem. You can be safe and disable the service if you don't need it, but not everyone has that luxury. Getting your hands on the latest exploits is usually not a problem. There are a wide variety of full disclosure security mailing lists and web sites. There are also a wide variety of consultants available to help you with this sort of thing. Testing your network lets you know what hackers and network scanners may see or already have seen. </P>
<P> </P>
<P>Don't believe that you may not be vulnerable to an attack just because an exploit has not be created for your particular operating systems and platforms. This is a trap that many system administrators fall into. For example, there are all sorts of remote buffer overflows written for the LINUX platform. Just because you are running a SPARC station, does not mean you are safe and out of harm's way. Ports of exploits to new systems can appear at any moment and any hacker worth his or her salt should be able to covert exploits between systems.</P>
<P> </P>
<P>If you are vulnerable, then you may have a problem. First of all, you've seen scanning and you think you are vulnerable. It would be wise to approach the box with caution as it may have been compromised. Second, you need to figure out what sort of impact that box has on the network so you can decide what actions you want to take to secure it. Can you down the box to make some patches? Is the box a single point of failure for the network? Can you protect the box by making a firewall rule change someplace else? The important thing here is to mitigate any negative impact on your network.</P>
<P> </P>
<I><P>Using a port scanner on the LINUX server finds five open ports including FTP(21), IMAP(143) and NFSD(2049). Your staff also confirms that the server is used for testing. Because of the malicious scanning, the possible vulnerabilities and its lack of impact on the network, you decide to disable the server. You connect to the west coast SSH gateway and disable the Ethernet port on the network switch, effectively isolating the server in case it was compromised. </P>
</FONT><B><FONT FACE="Arial"><P>Rule #5: Tell Someone!</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<I><P>Continuing with this scenario, you send some encrypted email to your counterparts on the west coast. You also leave some voice mail explaining the situation. Their security staff will conduct a forensic analyses of the server. Trying to keep everyone in the loop, you document the situation and email your management and selected operations people. You also contact the corporate security staff. You also consider putting the incident in your monthly security newsletter, but decide that the information is still need-to-know.</P>
</I><P> </P>
<P>It may seem surprising, but many security problems and events do not get reported for a variety of reasons. These reasons are a combination of technical and political factors that prevent a clear reporting system. We will discuss some of the specific reasons that security incidents don’t get reported.</P>
<P> </P>
<I><P>Blame </P>
</I><P> </P>
<P>Many system and network administrators do not report security events because they believe it will reflect poorly on them. An administrator may have previously boasted that "no one" can break into their network. Managers need to realize that these claims are ludicrous and should expect to see monthly security reports detailing suspicious network activity. </P>
<P> </P>
<I><P>Chain of Command</I> </P>
<P> </P>
<P>Who should a security event get reported to? If an organization has not stated how to handle security events, then when one happens it will get handled ad hoc. Most people correctly assume they should tell their immediate supervisor. This is true, but if a security department exists that has been trained to handle security situations, then they may be kept out of the loop. It may also be unclear what your supervisor may do with the information. (They should inform their supervisor and so on.)</P>
<P> </P>
<I><P>Management Indifference</I> </P>
<P> </P>
<P>It may be difficult to explain exactly why a specific type of network activity is 'bad' to inexperience management. Technology marches on and it is difficult to keep up with it for some people. However, this should not prevent an employee from reporting a possible security situation. Don't be afraid to use analogies to get your points across about network probes. Be prepared to demonstrate how an attacker could be probing your network to gain proprietary information. </P>
<P> </P>
<I><P>Management Overreaction</I> </P>
<P> </P>
<P>The opposite of the above example is true. If a network manager is not accustomed to experiencing network probes and scans, then the first time they occur may be traumatic. The best way to handle this is to be up front about the threat to your networks when you talk with your manager or supervisor. Be wary of any drastic measures taken by your management such as calling the FBI or the newspapers. That may not be the best course of action for the given situation. </P>
<P> </P>
<I><P>The corporate security staff contacts you immediately. They have hired computer security consultants to conduct a penetration test of the corporate networks. So far you have been the only person to detect and report the scanning.</P>
</FONT><B><FONT FACE="Arial"><P>Rule #6: Continue to Monitor</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<I><P>On a different day, one of your TCPDUMP sensors starts to collect a variety of suspicious traffic. Someone had caused a bunch of RealSecure alarms a few days ago. You responded with placing some TCPDUMP sensors on your network that collected anything from the suspicious IP addresses. It is strange that none of this traffic has caused a RealSecure alarm. What could it be?</P>
</I><P> </P>
<P>Once security probes have been noticed, the most important thing that a network administrator can do to their network is to make sure it is monitored. Depending on you network, you may be able to leverage your operations center. You may also have to place extra intrusion detection sensors to monitor uncovered network sections. Some people may even wish to deploy packet analyzers. Regardless of your technique, it is important to keep a watchful eye for any suspicious activity. The heightened state of alert exists because your network has been probed and you have made a determination that a network attack may be imminent.</P>
<P> </P>
<P>When leveraging any operations center, its important that you give them as much information on how to contact you and to accurately describe the security situation. In security situations, most operations personnel will contact you when anything suspicious occurs. Unfortunately, this may include normal network performance problems such as routers failing and Windows NT machines blue screening for no reason. Giving an operations center access to the intrusion detection systems is also a good thing. Hopefully, this has already occurred on your network. Leveraging quality 24x7 people can only be effective if they are properly trained and have well planned security response procedures.</P>
<P> </P>
<P>If you choose to deploy extra intrusion detection infrastructure, then you really have two choices. You can change the current rules used by the IDS to be more sensitive or you can deploy completely new systems. All IDS products are 'tuned' to a particular network. Thresholds need to be set and when they are exceeded, alerts and events are recorded. In a hostile environment where an attack may be imminent, lowering these thresholds may record extra probing from an attacker. It will also record more false positives. Deploying more IDS platforms may also be an option. If you have portions of your network that are not covered by the current set of IDS platforms, then deploying additional sensors may expose further attacks or network probe activity. </P>
<P> </P>
<P>Some security gurus may also wish to deploy a set of packet analyzer filters. The most common of these is TCPDUMP. These packet analyzers are deployed with similar strategies to packet based IDS platforms. You want to expose them to as much traffic as possible. In a switched environment, there is a possibility that you may want to run TCPDUMP on a web server or some other isolated production server. If you do this, keep in mind that you should try to obscure the sniffing program. Most network attackers are very familiar with network packet monitors. If they compromise a server that has a network analyzer running, they may find these logs and delete or modify them. </P>
<P> </P>
<P>When constructing packet filters to monitor suspicious traffic there are several strategies. You can log by destination port, destination address, source address or for a specific packet signature. Watching for specific destination ports can be very effective if you are in an environment where those ports are not active. Consider monitoring the network for all IMAP traffic which may be a service that a network scanner is targeting. If you do not have any IMAP servers, then scan attempts for this service will stand out. Sniffing all HTTP traffic in a web environment would not be a good idea unless you had some sophisticated analysis tools to deal with the barrage of recorded data. The same rules apply to destination addresses. If you have busy server, then logging all traffic to it may not be a good idea. But if the server is not heavily loaded, then recording all traffic may be an option. Sniffing based on the source IP address or source IP network may also find interesting activity. And finally, if you know or suspect the scanning techniques in use by a hacker, consider writing specific packet capture rules. A very easy example of this type would be to record any FIN-SYN packets. Hopefully your IDS is doing that already. </P>
<P> </P>
<I><P>Analysis of the traffic shows that the attacker is directly probing for RPC ports in the 32700-32800 range only on your Solaris servers. They must have performed some operating system fingerprinting last time they scanned your network. Looking at the traffic, there are an equal number of packet to each Solaris server except for two. Both of those have a third party backup program on them which uses an open RPC port. Sure enough, the scanning has focused in on these two servers. Your interest becomes very intense when you realize that there was an attempt to spawn an XTERM session from one machine to the attacker's IP address.</P>
</FONT><B><FONT FACE="Arial"><P>Rule #7: Contact the Source</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<I><P>Continuing with this scenario, after consulting with your security staff, you decide to contact the ISP where these RPC scanning events are originating from. You get all of the facts together and find the 'abuse' point of contact on their web page. Luckily, the ISP is in the same time zone so you can call during normal business hours. The ISP security manger tells you they had one of their LINUX DNS servers compromised and it was used to scan other networks. They are sorry for the inconvenience and assure you it won't happen again.</P>
</I><P> </P>
<P>When you choose to contact another network organization that is part of the Internet, you must keep a lot of things in mind. The most important thing you can be is organized and present your information clearly. Separate your conclusions from your solid data. Allow the person you are contacting to think about the situation for a moment. Don't demand immediate action. Here are some other guidelines to use when making contact:</P>
<P> </P>
<I><P>Don't expect to talk to someone right away </P>
</I><P> </P>
<P>Other security people are just like you. They get sick, go to lunch and take vacations. It is not unreasonable to contact an organization and find your security point of contact unavailable. In this case, you may have to deal with someone who is less skilled in security terminology or network administration. When this happens I like to ask for anyone who operates the servers or routers. Usually these people are very capable and can help analyze security events. </P>
<P> </P>
<I><P>Language </P>
</I><P> </P>
<P>If you attempt to contact foreign countries to chase down suspicious events, you may encounter someone who does not speak your language. English is very common worldwide, but there is no guarantee that you can use it to tell someone about a security incident. Some cultures can read English better than listening to it so consider sending emails or even fax communications. If you have any non-English speaking assets in your organization, it may be a good opportunity to tap them for translator duty.</P>
<P> </P>
<I><P>Time Zone </P>
</I><P> </P>
<P>Most commercial networks have some sort of 24x7 operations, but security gurus still seem to keep normal daylight hours. The person you are calling may be asleep. Realizing this, you may be lucky enough to get an organization that realizes the importance of your call and follows a procedure to wake the key individuals up for some analysis and hopefully some answers. On the other hand, you may also call and get an answering machine. In this situation you should leave concise information and multiple points of contact on your end. Remember, they may call back when you are asleep.</P>
<P> </P>
<I><P>You may be talking to the hacker </P>
</I><P> </P>
<P>In some cases, the person listed as the security point of contact may actually be responsible for the scanning of your network. Many hackers get jobs at ISP and other commercial network organizations. Many of them even get jobs in network security capacities. It is not unthinkable that these individuals may abuse their powers. Some telltale signs of this may include apathy to the situation, denial of the events and even open hostility. If nobody asks you for your name, telephone number or email address then something may be amiss. If the person has any knowledge of the scanning that you did not volunteer, then this may also be an indicator they are hiding something. </P>
<P> </P>
<I><P>They might not do anything </P>
</I><P> </P>
<P>Some network organizations are very busy. In rare cases, the people you contact won't do anything. Some ISP networks have an attitude that they provide unrestricted connectivity for their users and aren't responsible for their actions. Other organizations will help you, but are so busy with other security events that it may take some time before they can handle your complaint. Providing clear and concise facts about the incident can make their job easier and get quicker results for yourself.</P>
<P> </P>
<I><P>They might do to much </P>
</I><P> </P>
<P>Depending on what the situation is, your information could get some people fired, kicked out from the ISP and even sent to jail. I've had at least one ISP tell me they have "black listed" an individual for hacking. Some security events are honest mistakes, but it is possible for your point of contact to overreact and take some action that you are not comfortable with. For instance, an ISP may place a rule in their outbound packet filters that prevents any traffic to your network from theirs. This is very secure, but now nobody from that network can get to your web servers, send email or play Quake. </P>
<P> </P>
<I><P>They may have had a break in </P>
</I><P> </P>
<P>During many security situations, the people you are contacting may have suffered a security compromise. If this is the case then either they know it or they don't. If they know about the break-in, then they may be forth coming with all sorts of information. They may also be deceptive and try to hide the incident. If they do not indicate that they have been compromised, but you think they have, it's very important to try and get this information to the correct people. Telling a receptionist isn't going to help the problem get fixed unless you need his or her help to find the correct people. Taking a chance and calling a webmaster, the sales line or even customer support will usually get you within one or two phone calls of the correct person to talk with. They may ask you to work with them in analyzing and tracking down hackers. Be carefully not to discus any proprietary network information or security invents that do not directly involve your point of contact. This will shield you from inadvertently compromising someone's right to privacy.</P>
<P> </P>
<I><P>Everything you say may be used against you </P>
</I><P> </P>
<P>It's been said before, but I'll say it again. All of the conversations, intrusion detection events, logs, packet dumps and reports that you or your staff are involved in may be admissible in a court case. Be careful who you give logs to. You may or may not want your logs used in a court case or even have you or members of your security staff called as witnesses in a trial. I'm not going to preach moral values of network security, but it might not be in your best interests to assist some other network organization half way across the country to prosecute a hacker. Of course, the corollary to this is true also. Wouldn't you like it if another security expert was willing to testify that they detected probes from the individual(s) on trial for breaking into your network? Another aspect to keep in mind is to only give out log information to people who truly need access to it. The data can contain a variety of privacy information such as passwords and network usage. Only give the data to people you feel have a need to know and can effect responsible changes to prevent further network abuse.</P>
<P> </P>
<I><P>Email may have been compromised </P>
</I><P> </P>
<P>One last comment is to think twice before sending security event information over email. Email can be easily compromised. However, monitoring a large volume of email traffic may be outside the scope of a hacker. I like to email points of contact that have multiple accounts and choose one that is not on the possibly compromised network. Also consider the use of encryption.</P>
<P> </P>
<P>Final thoughts:</P>
<P> </P>
<P>Be professional when you handle yourself with other people outside of your organization. If you are in the business of selling Internet access, professionalism and common courtesy can take you a long way. The security community is very small and you may be surprised how often you'll run into other security people from different organizations.</P>
<P> </P>
<I><P>You are satisfied with your analysis and resulting conversation about the RPC scanning with the ISP. However, when writing the summary report you realize that the scanning did not originate from the ISP's DNS server, but one of their file servers. There is a good chance that they have been further compromised and do not know it. You review your logs once more to confirm your suspicions and then contact the ISP once again.</P>
</FONT><B><FONT FACE="Arial"><P>Rule #8: Consider Telling Outsiders</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<P>When security events occur, you may want to consider getting the word out to other Internet groups. There are a variety of outlets for this sort of information. What is right for you depends on the situation. Consider telling other organizations through the following means:</P>
<P> </P>
<I><P>FBI or Authorities </P>
</I><P> </P>
<P>The FBI and other authorities are continually getting better at tracking down hackers. The United States military also has an excellent program to track down suspicious network events. Regardless, the security events that you wish to bring to the attention of these organizations should be something new or serious activity. What is this sort of activity? I would say any activity that spans multiple networks or organizations. An example of this could be organized targeting of web servers that accepted credit card data at multiple locations at multiple companies. Another reason to contact the authorities is if you have information that could be useful to an ongoing investigation. Many security events are covered in the media and your information could be related and prove useful.</P>
<P> </P>
<I><P>CERT Organizations </P>
</I><P> </P>
<P>Computer Emergency Responses Teams track network security events and in some cases, will directly respond to them. Most people have heard of CERT advisories. Some of these advisories do not have vulnerability information, but warn of hacker activity. Information that you report to CERT can help produce better and more timely advisories. If you think that network scanners are looking for a new vulnerability, then CERT can use this information when issuing new vulnerability advisories. There are many different CERT organizations. If you are in the United States military, you should report security incidents to your branch's CERT agency. There are many CERT organizations that span international and corporate entities. Choosing which CERT agency to report an incident is sometimes a difficult task, but can get you more focused support.</P>
<P> </P>
<I><P>Mailing Lists</P>
</I><P> </P>
<P>In some cases, reporting scanning activity directly to a security mailing list can be beneficial for a variety of reasons. First, the information is very timely. Readers of the mailing lists are there to exchange information about new security vulnerabilities and your report of network scanning may spawn discussion as to what it could mean. Second, other people that have had similar network security events may come forward with other information. They may post to the mailing list or contact you directly. One word of caution though, hackers and malicious individuals also have access to these mailing lists. Don't disclose any information about vulnerabilities in your network or unique information about your network such as IP addresses or domain names. You may even want to get a second shell account just to send email to these mailing lists. Free web based email services such as Hotmail are very useful for this sort of activity. </P>
<U><P> </P>
</U><I><P>Security Web Sites</P>
</I><P> </P>
<P>There are also a wide variety of security web servers on the Internet today. These sites track vulnerabilities and hacker activity. Writing a story or submitting some content about recent scanning activity may help find a solution to the problems. Follow the same sanitation rules with your content to prevent drawing undue attention to your network.</P>
<U><P> </P>
</U><I><P>Company Outsiders </P>
</I><P> </P>
<P>In some cases, it may be appropriate to raise the level of concern at your company. If more money or resources should be involved with network security, then telling Vice-Presidents and other upper corporate management may get you some results and raise awareness. In some cases, CEOs and CIOs may have misconceptions about the current level of network security. There is nothing like a security event to bring a little reality to the situation.</P>
<P> </P>
<I><P>The Media </P>
</I><P> </P>
<P>Bringing the media into a security situation is usually a recipe for disaster. Most security experts agree that the last thing you should do is to tell other people how secure you are or tell them that you are under network attack. This is a magnet for hackers. You might as well hang a sign on your web page that says, "Hack Me!". Realistically though, the media can be used to tell your customers about your network security if it is done right. If you have firewalls and intrusion detection systems, then there is really nothing wrong with telling people that you have firewalls and intrusion detection systems. Just don't make the leap and state that you have "impenetrable" security. With security incidents, care must be taken to present a positive image about your company or network without reflecting poorly on yourself. Press releases are an excellent tool for this purpose. </P>
</FONT><B><I><FONT FACE="Arial"><P>Rule #9: Determine What Attracted Attention</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<P>Any investigation of suspicious activity should attempt to conclude what attracted the network scanning in the first place. By attempting to discover this information, you may prevent some scanning in the future. Hackers tend to be opportunistic. Unless they have a bone to grind with you, most malicious hackers are simply looking for vulnerable servers to compromise. Here are some things that opportunistic hackers look for:</P>
<P> </P>
<I><P>Default Web Server Splash Pages</P>
</I><P> </P>
<P>One of the easiest ways to quickly assess the security of a network is to visit each of its web servers and see how many of them display the default Internet Information Server or Apache splash pages. These can usually indicate recently deployed systems, development networks or systems that are not maintained that well. All of these may have security problems. The combination of the security possibilities and the lack of maintenance can be a green light to most hackers. </P>
<P> </P>
<I><P>DNS Entries</P>
</I><P> </P>
<P>If your DNS server has names for network nodes such as 'test-linux-1', 'guest' or even 'web-development', then these names could attract a hacker's attention. Naming things by their function ('router1', 'firewall-3', etc.) tells your staff and the entire world exactly what the operation of a particular network node is. If an attacker is looking for a particular exploit to try, they may consult your DNS server to find a target. In some cases, poorly maintained DNS entries can also indicate a poorly administered network. Many hackers (correctly) assume this also means poor security. A good example of a DNS problem is the advertisement of RFC 1918 addresses through normal DNS queries. </P>
<P> </P>
<I><P>Default Exploitable Services</P>
</I><P> </P>
<P>Many hackers will conduct a quick limited probe to find out how secure or unsecured a given network is. What they may be looking for is a group of computers not protected by a firewall and offering a variety of services such as DNS, TELNET, FTP, SMTP, HTTP and RPC. For example, if a hacker does conduct a quick port scan of a web server which results in only port 80 being active, then the hacker may conclude that the site is reasonably secured. Compare this to a port scan of the web server that discovers twelve active ports. Each situation represents a different level of perceived security posture.</P>
<P> </P>
<I><P>Press Releases</P>
</I><P> </P>
<P>Nothing draws attention to your site like the media. If your company or clients to your company have media exposure, then it follows that some hackers will get the idea to try and scan or hack you. At least one ISP I am aware of was dismayed when one of its customers actually challenged hackers to break into their web server. The ISP suffered fratricide when attacks on the web server spilled over to the operations center.</P>
<P> </P>
<I><P>Internet Relay Chat</P>
</I><P> </P>
<P>Running IRC servers is an easy way to attract hackers. Many hackers stereotypically spend hours on end chatting about a variety of topics. When hackers disagree with each other, they will use a variety of denial of service attacks to disable the other person's network access. In some cases direct attacks on the other person's network will also ensue. Having network users that spend time in IRC chat discussions is also an advertisement to hackers. The debate to allow or not to allow IRC access from your network may be abated by providing free access to a shell accounts outside of your network, or by placing an IRC server outside of your firewall.</P>
<P> </P>
<I><P>Hacker Web Pages</P>
</I><P> </P>
<P>Nothing attracts hordes of hackers like a hacker web page. Many of these pages suffer network attacks. Many of these pages are also seen as 'trophy' pages that various hacker groups would like to claim credit for hacking. If you run a network that has this sort of information, you may be experiencing a certain level of security events simply because people are poking around your network looking for ways to break into the hacker web page. </P>
<P> </P>
<I><P>Do you have hackers working for you?</P>
</I><P> </P>
<P>If you have a hacker working for you or at your company, then the suspicious probes you may see could be coming from the hacker during off hours or from the hacker's friends (or enemies). </P>
<P> </P>
<I><P>Could it be corporate espionage?</P>
</I><P> </P>
<P>Not all hacking events are random acts of opportunity on a hacker's part. The suspicious events could indicate a coordinated effort to compromise the security of your network to obtain some level of information. This information could include email, trade secrets, plans, spread sheets or any number of other proprietary pieces of information. </P>
</FONT><B><I><FONT FACE="Arial"><P>Internal Security Events</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<P>If you suspect that an internal person in your company is conducting network probes or doing something nefarious, then you should follow some loose guidelines when analyzing and presenting evidence.</P>
<P> </P>
<I><P>Limit your opinions</P>
</I><P> </P>
<P>Do not spread rumors or false accusations about any individual suspected of abusing their network access. This could be viewed as slander in a court case. When presenting suspicious internal security information to other individuals, do not associate any suspected individuals with the data. Try to sanitize the data before presenting it to any group of people. Once someone is branded a hacker, they have a tough time loosing that image even if they are shown to be innocent. Also keep in mind that other people may not have the same opinions about security as you do. Depending upon their position and yours, you may have to work out an agreement or get an interpretation of your network security policy. </P>
<P> </P>
<I><P>Document, Record & Save</P>
</I><P> </P>
<P>Try to save as much information about the suspicious events as possible. In most cases, these events are more powerful and damaging (and damning) than your word alone. Make sure that the records are in a secured place where they cannot be tampered with. Physical media such as CD-ROMs are ideal for this sort of application. These records will become very crucial in any decisions to terminate an individual. They may also play a role in any police or FBI investigations.</P>
<P> </P>
<I><P>Involve Human Resources and Corporate Security</P>
</I><P> </P>
<P>Most companies have an HR department that deals with personnel issues. Some companies also have a corporate security group that handles internal security issues. Involving both of these organizations can help protect your interests as well as the company's . They may also have other information that you don't have. Consider the example where an employee has submitted their two week notice and has started to hack the network form the inside. This is a very serious situation which you may have only perceived as a minor network nuisance. If you are the technical subject matter expert on security, you may be asked by these groups to assist in an investigation.</P>
<P> </P>
<I><P>Avoid Target Fixation</P>
</I><P> </P>
<P>We are all human. Chances are you would not like your actions to be monitored on a regular basis. Small acts such as taking office supplies for home use seem much more serious when an investigation is underway. The same is true for network security investigations. For example, just because a person is visiting hacker web sites does not mean they are a hacker. Focusing to much on a person's network activities can provide a myopic view of the big picture. </P>
<P> </P>
<I><P>Challenge The Individual</P>
</I><P> </P>
<P>Once the proper authorities are involved and there is agreement that the individual is doing something strange on the network, the individual should be challenged. If this is done correctly, then the individual will be surprised and may give themselves away through statements they make. That is of course if they are hiding something. You should carefully use these meetings to find out what the individual was doing. Ask direct questions about network usage and try to find anything that can explain the suspicious network traffic. If you hit a brick wall, you may want to tell the individual what sort of traffic was observed. Stereotypically, this is when the person confides that they let Bruce from the accounting department use the computer during lunch time. </P>
<P> </P>
<P>You should also have a physical inspection of the computer performed while the meeting is taking place. These may seem like very extreme measures, but if an individual is to be terminated immediately, the chance to discover Trojan horse programs or logic bombs needs to be addressed. Also a physical inspection of their computer may provide contradicting evidence to their testimony or explanation of their activity. </P>
<P> </P>
<P>Final thoughts:</P>
<P> </P>
<P>Computer analysis should best be left to computer experts. However, since security events involve people, the best chance of catching an internal hooligan may come from an ex-police officer or an ex-investigator. People with these backgrounds and some good computer knowledge are becoming very popular to handle computer crime investigation. If your company has these assets, then I strongly urge you to involve them in computer security incidents.</P>
</FONT><B><I><FONT FACE="Arial"><P>Final - 'Final Thoughts'</P>
</B></I></FONT><FONT SIZE=2><P> </P>
<P>Following the steps outlined here may save you a lot of time and headaches when dealing with network probes and security events. I urge you to discuss these concepts with your staff and management. You should also discuss this information with any other people in your organization associated with network operations. </P>
<P> </P>
<P>The one thread that holds all of these rules together is 'common sense'. Analysis and termination of network probes and security events occurs in the human world. It does not occur in the computer world. Dealing with other people can be tricky. Be nice. Be courteous. There are some people at your company and at other networks that won't be easy to work with, but this is a fact of life that usually shows itself during a security event. Act professional and stick to the facts. For a variety of reasons, many people tend to forget these when placed under pressure and in high tension situations. Good luck!</P></FONT></BODY>
</HTML>
© 2022 Packet Storm. All rights reserved.
%7Cutmcsr%3D(direct)%7Cutmcmd%3D(none)){kind=small}
Follow us on Twitter
Follow us on Facebook
Subscribe to an RSS Feed