Detecting malicious hosts with Darknets Security Camp Boston University 10 March 2006 Christopher Misra Network Systems and Services UMass statistics ƒ 25,000 active hosts ƒ 142 buildings ƒ All 42 Residential buildings networked ƒ 12000+ Residence hall connections (port-per-pillow) ƒ 8000+ Academic building connections ƒ 1200+ Cisco 24 port Switches ƒ 8 Cisco core routers ƒ 600 Off-campus dial-in modem lines ƒ 500Mb/s - Commodity Internet connections ƒ 155Mb/s - Internet2 connection • Each over leased GigE private fiber Network Systems and Services 2 Our Environment ƒ 2 Enterprises in one: Academic & Residential ƒ Residential network • 12,000 Residential hosts • We operate functionally as an ISP • However, they are still our students, we need to teach them ƒ Academic network • Traditionally open network • Academic freedom • Minimal content inspection Network Systems and Services 3 Higher education challenges ƒ Students exploring the boundaries of policy ƒ Many systems within our enterprise boundary, but not centrally managed. ƒ Sense of academic freedom occasionally leads students to make poor choices • Copyright • Hacking/Cracking • Wireless everywhere Network Systems and Services 4 Boundary Conditions ƒ Require robust security with limited staffing and budgets • More to do than we could hope to accomplish ƒ Parts of the enterprise are highly de- centralized. • Self-supporting • Many unmanaged systems Network Systems and Services 5 Incident Response ƒ Given the nature of our network and user base, we have a high proportion of incidents • DMCA, Spam, peer-to-peer ƒ Open network, complex challenges • allow most, deny bad • botnet hosts and C&C • limited edge filtering • Large pipes Network Systems and Services 6 Incident Response ƒ Since we don’t manage many of the hosts on our campus, we need to find solutions that meet this need. ƒ Robust help desk to meet the needs of campus users ƒ How can we automate detection of activity? • And leverage help desk to improve security Network Systems and Services 7 Detection ƒ We don’t use any agent based detection on endpoint systems ƒ Heterogeneous environment ƒ Majority of these devices are not centrally managed Network Systems and Services 8 Netflow ƒ Originally developed by Cisco ƒ Versions • 5: most frequently in use (what we use) • 7,8: Cisco specific • 9 : Basis for IETF IPFIX (IP Flow Information Export ) ƒ IPFIX • An effort to standardize flow export • http://ipfix.doit.wisc.edu/ Network Systems and Services 9 IPFIX ƒ Architecture for IP Flow Information Export • Architecture for the selective monitoring of IP flows, and for the export of measured IP flow information from an IPFIX device to a collector http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architecture-09.txt ƒ Information Model for IP Flow Information Export • Protocol for transmitting information related to measured IP traffic over the Internet http://www.ietf.org/internet-drafts/draft-ietf-ipfix-info-11.txt Network Systems and Services 10 IPFIX: Security Analysis and Intrusion Detection ƒ IPFIX provides information about the traffic in a network. Therefore, it is very well suited to take a key role in the detection of network threats • intrusions • propagation of viruses and worms • port scanning ƒ Detection • visually monitor events (with a console) • collect data from sensors (with one or more event collectors) • store data from sensors (in a database) http://www.ietf.org/internet-drafts/draft-ietf-ipfix-as-06.txt Network Systems and Services 11 Detection: Netflow ƒ Visual analysis is often the most efficient detector for us ƒ Great tool for detecting Denial of Service attacks • Prone to data loss under abnormal load ƒ Useful tool for post-incident analysis • Provided the data has not been cycled off the system ƒ As links become faster, many flow exports are sampled ƒ Useful for Capacity planning and DoS detection, but of limited use for forensic purposes Network Systems and Services 12 Detection: Netflow Network Systems and Services 13 Detection: Netflow data ƒ srcIP dstIP prot srcPort dstPort octets packets ƒ 80.116.163.85 xxx.yyy.131.204 17 3111 1434 404 1 ƒ 81.3.162.10 xxx.yyy.131.182 17 1514 1434 404 1 ƒ 200.74.27.228 xxx.yyy.131.246 6 447 8080 40 1 ƒ 200.74.27.228 xxx.yyy.131.246 6 64068 80 40 1 ƒ 200.74.27.228 xxx.yyy.131.246 6 50265 3128 40 1 ƒ 142.179.169.213 xxx.yyy.131.178 17 1126 1434 404 1 ƒ 213.60.21.96 xxx.yyy.131.171 17 1923 1434 404 1 ƒ 212.180.2.68 xxx.yyy.131.114 6 63559 41544 40 1 ƒ 200.29.164.162 xxx.yyy.131.233 17 1051 1434 404 1 ƒ 202.103.13.62 xxx.yyy.131.35 6 9001 30185 40 1 ƒ 213.119.233.63 xxx.yyy.131.7 17 1246 1434 404 1 ƒ 216.51.150.219 xxx.yyy.131.7 17 1157 1434 404 1 ƒ 24.112.24.160 xxx.yyy.131.122 17 1129 1434 404 1 Network Systems and Services 14 Detection: Darknets ƒ We use NetFlow heavily for diagnostics and capacity planning ƒ Large campus network requires campus IGP ƒ Mix together and what do you get? • Pointing unused (dark) network space at a flow sensor provide good data with low overhead • Non-intrusive sensor Network Systems and Services 15 Darknets ƒ A darknet collector listens to one or more blocks of routed, allocated, but unused IP address space. ƒ Because the IP space is unused (hence "dark") there should be very little if any legitimate traffic entering the darknet ƒ Team Cymru Darknet Project • http://www.cymru.com/Darknet/index.html Network Systems and Services 16 Darknets: Campus IGP ƒ Large, complex campus network necessitates an IGP • Dynamic network provisioning • Network retirement • Many hands ƒ We use hold-down (nailed-up) routes anyways • Static route at the border to minimize route flapping • Pointing our address space to Null0 with a high metric • Fail safe Network Systems and Services 17 Darknets: Campus IGP ƒ Originally we static routed unused address space to our darknet • Not scalable with many hands in the network • Relatively narrow aperture sensor ƒ Why not inject hold-down routes for unused space to a stub router? • And generate these netflow records in one place • Doesn’t need a lot of horsepower • Unused space dynamically falls in to the Darknet Network Systems and Services 18 Darknets: Bogons, Martians, and friends ƒ In addition to aggressive ingress and egress filtering we filter bogons, martians, etc. ƒ A bogon prefix is a route that should never appear in the Internet routing table. • No packet routed over the public Internet should never have a bogon source address • Commonly found as the source addresses of DDoS attacks. http://www.cymru.com/Bogons/index.html Network Systems and Services 19 Darknets: Bogons, Martians, and friends ƒ A martian prefix is a reserved and special use IPv4 prefix ƒ RFC 1918, RFC 3330 ƒ Filtered by campus ingress/egress filters • Also interesting to catch in the darknet. • Often result of misconfigurations. Network Systems and Services 20 Darknets: Bogons, Martians, and friends ƒ Other fun ones include: • Unused 128.119.0/24 from our /16 • Dumb viruses love scanning this space first • Unused 128.119.255/24 ƒ Some people are fans of allocated but non- routed space filtering • Some DoD allocated space, etc. ƒ These are far more subject to local politics/peculiarities. ƒ Goal is to increase aperture of sensor without compromising network functionality. Network Systems and Services 21 Darknets: Bogons, Martians, and friends ƒ Static routes on Darknet router(s) • Scalability, etc ƒ Dynamic routes • Bogon Route Server Project • Peering is conducted over a multihop eBGP peering session. http://www.cymru.com/BGP/bogon-rs.html Network Systems and Services 22 Darknets ƒ We still filter at our edge appropriately. • Best practices ƒ We also filter inappropriate sources at each subnet router interface • Unicast reverse-path forward filtering (Cisco) ƒ Makes our Darknet far more predictable • Signal-to-noise ratio much higher. Network Systems and Services 23 Detection: Darknets ƒ Non-content logging data source • Privacy retains high community value ƒ Non-signature based detection ƒ Catches malicious activity as well as anomalous traffic • Often due to misconfigured/faulty hosts Network Systems and Services 24 Detection: So what do we use it for ƒ Compromised host detection • This is our primary use • Many viruses still just scan campus subnets • Though this seems to be changing ƒ Network Mapping detection • Reconnaissance, etc ƒ Trends • There is a lot of garbage out there Network Systems and Services 25 Example Darknet Data ƒ 9 March 2006 20:00-20:59 EST ƒ 33,472 flow from Non-local sources to Darknet ƒ Top sources IPaddr flows octets packets 208.38.28.100 4539 375600 7825 202.97.170.135 3513 276908 6286 221.214.141.34 591 32652 800 202.105.233.25 581 45556 1085 221.1.220.30 508 24384 508 Network Systems and Services 26 Darknet data: Flows by IP protocol ƒ 9 March 2006 20:00-20:59 EST protocol flows octets packets 6 3461 402916 8613 17 1676 354254 2167 1 301 42302 594 2 1 28 1 Network Systems and Services 27 Darknet data: Flows by destination port ƒ 9 March 2006 20:00-20:59 EST port flows octets packets 80 7714 706378 14602 6346 3813 306939 5664 6348 3751 363570 6378 1026 1129 236651 1243 30592 723 43605 765 6349 623 51545 962 Network Systems and Services 28 Detection ƒ User is identified based on initial registration information ƒ Username<-> MAC Address mapping ƒ MAC address to IP address mapping maintained in a database • With some historical information ƒ All data is considered confidential • Restricted access • Retained pursuant to policy Network Systems and Services 29 Device location ƒ To perform most isolation, detected device must be located ƒ Given an IP address we must be able to determine: • MAC Address at time of infraction • Associated username • Switch and port device is currently connected to Network Systems and Services 30 Isolation ƒ Isolation is enforced by changing network devices to limit the access of a non- compliant host. ƒ This protects the detected host from: • additional compromise • protects other hosts from it • provide a conduit for notifying the responsible individual(s) Network Systems and Services 31 Isolation ƒ Capable of isolating hosts at layer 2 and layer 3 • Dependent on violation and severity of activity. ƒ Capable of disconnecting host from network completely • Shutting down interface on switch ƒ Isolation enforced at building edge and perimeter routers • Cisco ACLs Network Systems and Services 32 Notification ƒ In most cases, the user is notified by email • Even when isolated, users can access campus email services ƒ Help Desk is notified • Within trouble ticketing system ƒ Status is updated in real time ƒ In cases of managed computers, isolation may not be performed Network Systems and Services 33 Darknet Conclusions ƒ This data is useful • To the point of reconfiguring our stub router (located as a development box in our NOC) to have better availability • You don’t realize how much you miss it until it is gone. ƒ Effective sensor with low overhead • We leveraged existing infrastructure ƒ Empirically as good a sensor as some commercial IDS sensors • Thought the vendors still dispute this Network Systems and Services 34 Automated Darknet Processing ƒ We currently have some automated processing tools to analyze the data • Hits per unit time • Number of targets ƒ Not yet brave enough to have these tools push the button • Still a person in the middle to approve isolation action ƒ Likely we will increase automation • Increased trust in validity of sensor Network Systems and Services 35 Sharing Darknets ƒ While we have drastically improved the sensor aperture, it is still scoped to campus address space • Difficult to see some trends • Early warning only to traffic that hits us ƒ There’s no reason we couldn’t look to coordinate our data with others • Policy issue not withstanding ƒ We have engaged in these partnership within closed, vetted, trusted communities of interest Network Systems and Services 36 Futures ƒ As we address these policy issues, can we organize this on a broader scale? ƒ Addressing these concerns in working group format ƒ Partnerships with groups like REN-ISAC • And with that… ƒ Questions: After Doug Pearson’s presentation… Network Systems and Services 37