INSIGHTS | June 11, 2013

Tools of the Trade – Incident Response, Part 1: Log Analysis

By Wim Remes

There was a time when I imagined I was James Bond zip lining into a compromised environment, equipped with all kinds of top-secret tools. I would wave my hands over the boxes needing investigation, use my forensics glasses to extract all malware samples, and beam them over to Miss Moneypenny (or “Q” for APT concerns) for analysis. I would produce the report from my top-notch armpit laser printer in minutes. I was a hero.

As wonderful as it sounds, this doesn’t ever happen in real life. Instead of sporting a classy tuxedo, we are usually knee deep in data… often without boots! I have recently given a few presentations⁽¹⁾ on Incident Response (IR). The question I am most often asked concerns the tool chain that would enable an individual or a team to perform the basic actions one would expect from an Incident Responder.

Let me be clear. There is an immense difference between walking on to a crime scene in which your organization’s intent is to bring a perpetrator to court and approximately 90% of the actual IR work you will perform. The biggest difference has to do with how you collect and handle evidence, and you can rest assured that any improperly handled evidence will be relentlessly attacked by the defense.

However, I am writing this blog post from the point of view of a blue team member. I’ll focus on the tools that are available to determine what has happened, when it happened, and (most importantly) how to make sure it does not happen again.

TL;DR : Many, if not all, of the tools you will need are available in the SANS Investigate Forensics Toolkit (SIFT) VMWare image⁽²⁾ . You can skip the rest of this post and start playing with these tools right away if you want to, but I hope I can at least give you some pointers on where to look first.

As I illustrated in the introduction, IR work is incredibly unsexy. Almost without exception, you will run into a special case that requires you to work on the command line for hours trying to figure out what has occurred. The next-next finish of many a security product installation and use process is something you will nostalgically reflect on over a burning log (pun intended).

You will typically have in your possession three important pieces of evidence:

Log files
Memory images
Disk images

I will address these in three separate blog posts and try to give you some ideas about the tools you can use during your investigation.

In this blog post, we will start off by looking at log files.

Every device on the network creates a gigantic amount of data. Some of it is used for monitoring purposes, but most of it is (hopefully) stored and never to be looked at again. Until a system is compromised, this data is generally useless with the exception of a few hidden golden threads. These threads will give you a good starting point for figuring out what has happened. They might also be useful in tying the whole story together. Essentially, you must figure out what the log files want to tell you. (Cue the X-Files soundtrack.)

Sifting through log files is like looking for gold. Your initial sift will be very coarse grained. As you get rid of more and more of the dirt, the mesh of your sift will get finer and finer.

In my humble opinion, there is nothing better than having a thorough understanding of the Unix command-line tools ‘grep’, ‘sed’, and ‘awk’. The good thing about these tools is that you will find them on almost any Unix or Linux box you can get your hands on. The bad thing is that you will either need to know or need to find out what you are looking for.

In the absence of a full-fledged SIEM infrastructure, two tools exist that I recommend you become familiar with:

OSSEC⁽³⁾
log2timeline⁽⁶⁾

First up, OSSEC. Not only is this a good (and open-source) host-based intrusion detection system, but it also allows you to feed it collected logs through its command-line tools! The events will trigger OSSEC out-of-the-box alerts and give you a good overview of the most interesting events that have taken place. You can also easily develop custom rule sets based on (as an example) publicly available data from other compromises⁽⁴⁾

As you become more experienced with OSSEC, you can build your own set of scripts, drawing inspiration from a variety of places. In cases where ye olde *nix is not available, make sure you are also familiar with a Windows shell tool. (Powershell is one heck of an alternative!) Also, keep your Command Line Kung Fu⁽⁵⁾ handy!

You will not be able to perform IR without the powerful yet extremely elegant log2timeline⁽⁶⁾ tool. This blog post does not offer enough space to provide a decent overview of log2timeline. In essence, it groks data from different sources and creates a ‘Super Timeline’ that helps you to narrow in on those timeframes that are relevant to your investigation. It is one thing to dig through information on a command line, but it is a completely different ball game when you can visualize events as they have occurred.

In upcoming blog posts, I will dig into the juicy secrets that can be found in memory and disk images. A major lesson to be learned as you climb on the ladder of IR is that many of the tools you will use are produced by your peers. Sure, IR software is available and works well. However, the bulk of tools available to you have been developed by your fellow incident responders to address a particular problem. Many responders generously release their tools to the public domain for others to use. We learn from others probably more than we teach.

(1) BYO-IR: Build Your Own Incident Response: http://www.slideshare.net/wremes/isc2-byo-ir
(2) http://computer-forensics.sans.org/community/downloads
(3) http://www.ossec.net
(4) e.g. APT1 Indicators of Compromise @ http://intelreport.mandiant.com/
(5) http://blog.commandlinekungfu.com/
(6) http://log2timeline.net/

INSIGHTS | June 4, 2013

Industrial Device Firmware Can Reveal FTP Treasures!

By Sofiane Talmat

Security professionals are becoming more aware of backdoors, security bugs, certificates, and similar bugs within ICS device firmware. I want to highlight another bug that is common in the firmware for critical industrial devices: the remote access provided by some vendors between their devices and ftp servers for troubleshooting or testing. In many cases this remote access could allow an attacker to compromise the device itself, the company the device belongs to, or even the entire vendor organization.

I discovered this vulnerability while tracking connectivity test functions within the firmware for an industrial device gateway. During my research I landed on a script with ftp server information that is transmitted in the clear:

The script tests connectivity and includes two subtests. The first is a simple ping to the host “8.8.8.8”. The second test uses an internal ftp server to download a test file from the vendor’s ftp server and to upload the results back to the vendor’s server. The key instructions use the wget and wput commands shown in the screen shot.

In this script, the wget function downloads a test file from the vendor’s ftp server. When the test is complete, the wput function inserts the serial number of the device into the file name before the file is uploaded back to the vendor.

This is probably done to ensure that file names identify unique devices and for traceability.

This practice actually involves two vulnerabilities:

Using the device serial number as part of a filename on a relatively accessible ftp server.
Hardcoding the ftp server credentials within the firmware.

This combination could enable an attacker to connect to the ftp server and obtain the devices serial numbers.

The risk increases for the devices I am investigating. These device serial numbers are also used by the vendor to generate default admin passwords. This knowledge and strategy could allow an attacker to build a database of admin passwords for all of this vendor’s devices.

Another security issue comes from a different script that points to the same ftp server, this time involving anonymous access.

This vendor uses anonymous access to upload statistics gathered from each device, probably for debugging purposes. These instructions are part of the function illustrated below.

As in the first script, the name of the zipped file that is sent back to the ftp server includes the serial number.

The script even prompts the user to add the company name to the file name.

An attacker with this information can easily build a database of admin passwords linked to the company that owns the device.

The third script, which also involves anonymous open access to the ftp server, gathers the device configuration and uploads it to the server. The only difference between this and the previous script is the naming function, which uses the configs- prefix instead of the stats- prefix.

The anonymous account can only write files to the ftp server. This prevents anonymous users from downloading configuration files. However, the server is running an old version of the ftp service, which is vulnerable to public exploits that could allow full access to the server.

A quick review shows that many common information security best practice rules are being broken:

Naming conventions disclose device serial numbers and company names. In addition, these serial numbers are used to generate unique admin passwords for each device.
Credentials for the vendor’s ftp server are hard coded within device firmware. This would allow anyone who can reverse engineer the firmware to access sensitive customer information such as device serial numbers.
Anonymous write access to the vendor’s ftp server is enabled. This server contains sensitive customer information, which can expose device configuration data to an attacker from the public Internet. The ftp server also contains sensitive information about the vendor.
Sensitive and critical data such as industrial device configuration files are transferred in clear text.
A server containing sensitive customer data and running an older version of ftp that is vulnerable to a number of known exploits is accessible from the Internet.
Using Clear text protocols to transfer sensitive information over internet

Based on this review we recommend some best practices to remediate the vulnerabilities described in this article:

Use secure naming conventions that do not involve potentially sensitive information.
Do not hard-code credentials into firmware (read previous blog post by Ruben Santamarta).
Do not transfer sensitive data using clear text protocols. Use encrypted protocols to protect data transfers.
Do not transfer sensitive data unless it is encrypted. Use high-level encryption to encrypt sensitive data before it is transferred.
Do not expose sensitive customer information on public ftp servers that allow anonymous access.
Enforce a strong patch policy. Servers and services must be patched and updated as needed to protect against known vulnerabilities.

INSIGHTS | May 29, 2013

Security 101: Machine Learning and Big Data

By Gunter Ollmann

The other week I was invited to keynote at the ISSA CISO Forum on Incident Response in Dallas and in the weeks prior to it I was struggling to decide upon what angle I should take. Should I be funny, irreverent, diplomatic, or analytical? Should I plaster slides with the last quarter’s worth of threat statistics, breach metrics, and headline news? Should I quip some anecdote and hope the attending CISO’s would have an epiphany that’ll fundamentally change the way they secure their organizations?

In the end I did none of that… instead I decided to pull apart the latest batch of security buzzwords – “Big Data” and “Machine Learning”.

If you attended RSA USA (or any major security vendor/booth conference) this year you can’t have missed the fact that everything from Antivirus through to USB memory sticks now come with a dab of big data, a sprinkling of machine learning, and a dollop of cloud for good measure. Thankfully I’m a cynic; or else I’d have been thrashing around on the ground in an epileptic fit from all the flashy marketing claims and trademarked nonsense phrases.

I guess I’m lucky to be in the position of having had several years of exposure to some of the greatest minds at Georgia Tech as they drummed in to me on a daily basis the “what and how” of machine learning in the context of solving many of today’s toughest security problems.

So, it was with that in mind that I thought “If I’m a CISO and everything I know about machine learning and big data came from carefully rehearsed vendor sound bites and glossy pamphlets, would I be able to tell the difference between Chanel #5 and cow manure?” The obvious answer would result in some very happy farmers.

What was the net result of this self-inflection and impending deadline? I crafted a short presentation for CISO’s… a 101 course on machine learning and big data… and it included ducks.

If you’re in the upper tiers of your organization and you’ve had sales folks pimping you their latest cloud-infused, big data-munching, machine learning, and world-hunger-solving security solution, please carry on reading as I attempt to explain the basics of the latest and greatest in buzzwords…

First of all – some context! In the world of breach detection and incident response there’s a common idiom: “If it walks like a duck, flies like a duck, and quacks like a duck… it must be a duck.”

Now I could easily spend another 5,000 words explaining why such an idiom doesn’t apply to modern security threats, targeted attacks and advanced persistent threats, but you’ll have to wait for a different blog post. Rather, for this 101 lesson, it’s important to understand the concept of “Feature Selection” – which in the case of this idiom includes: walking, flying and quacking.

If you’ve been tasked with dealing with a duck problem, ideally you’d be an aficionado on the feet, wings and sounds of ducks. You’d be able to apply this knowledge to each bird you have the time to focus your attention on and make a determination: Duck, or Not a Duck. As a security professional, you’d be versed in the various attributes of certain threats – and able to derive a conclusion as to the nature of the security problem.

The problem though is that at scale things break down.

What do you do when there’s too many to analyze, when time is too short, and when you can’t make out all the duck features from afar? This is typical of the big data problem (and your everyday network traffic). Storing the data is the easy part. Retrieving the data is mechanically complicated, but solvable.

Meanwhile, making decisions and taking actions upon the data is typically the most difficult part. With every doubling of data, your problem grows exponentially.

The traditional method of dealing with the situation has been to employ signature matching systems. In essence, we build rules based upon the features we’ve previously identified as significant and capable of bounding the problem (or duck selection). We then compare these rules against the sample animal and receive a binary answer – Duck, or Not a Duck.

Signature systems can be very efficient at classification. Just look at your average Intrusion Prevention System (IPS). A problem though lies in the scope of the features that had been defined.

If those features (or parameters) used for classification are too narrow (or too broad) then evasion is not only probable, but guaranteed. In essence, for a threat (or duck) to be classified, it must have been observed in the past or carefully predicted (although rare).

From an attacker’s perspective, knowledge of those features and triggering parameters makes it a trivial task to evade or to conduct false flag operations. Just think – hunters do this all the time with their floating duck decoys. Even fellow duck hunters have been known to mistakenly take pot-shots at them too.

Switching pace a little, let’s look at the network a little.

The big green blob is all the network traffic for an organization for a week. The red blog right-of-center is traffic associated with an active breach, and the lighter red blob with the dotted lines are just general malicious traffic observed within the network. In this two-dimensional view (if I hadn’t color-coded it previously) you’d have a near impossible task differentiating between them. As it is, the malicious traffic is mixed with both the “safe” and “breach” traffic.

The trick in differentiating between the network traffic types lies in increasing the dimensionality of the problem. What was a two-dimensional blob suddenly becomes much clearer when an appropriate view or perspective to the data is added. In the context of the above diagram, the addition of a z-axis and an extension in to the third-dimension allows the observer (i.e. analyst) to easily differentiate between the traffic types – for example, the axis could represent “country code of destination IP address”. In this context, the appropriate feature selection can greatly simplify the detection problem. Choosing appropriate features is important – nay, it’s critical!

This is where advances in machine learning over the last half-decade have really come to the fore in computer science and more recently in information security.

Without getting in to any of the math behind the various machine learning algorithms or techniques, the key concept you need to understand is “training”. It can mean many things to many a mathematician, but since we’re likely not one of those, what training means in our context is that we already have samples of what we’re going to be looking for, and samples of things we know we’re definitely not interested in. The better we define and populate these training sets, the more precise the machine learning system we’re employing will be in differentiating between them – and potentially classifying other contenders.

So, in this example we’ve taken a bunch of ducks and grouped them together. They become our “+ve class” – which basically means these are the things we’re interested in. But, equally important, is our “-ve class” – our collection of things we know not to be ducks. In many cases our -ve class may be more important than our +ve class because it contains all those false positives and “nearly” ducks – the things that may have caught us out once before.

One function of a good machine learning system is to automatically determine which attributes make the most sense in differentiating between your +ve and -ve classes.

While our poor old hunter (or analyst) was working with three features – walks, flies, and talks – the computer-based system may have reviewed all the attributes that were available and determined which ones are the most useful in differentiating between “ducks” and “not ducks”. In many cases the system will have weighted the various features (or attributes) to indicate which features are more deterministic of the classes.

For example, texture may be a good indicator of “not a duck” – since none of the +ve class were made from plastic or wood. Meanwhile features such as “wing length” may not be such a good criteria and will be weighted in a way to not have an influence on determining whether a duck is a duck or not – or may be dropped by the system entirely.

The number of features reviewed, assessed and weighted by the machine learning system will eventually be determined by the type of data being used and how “rich” it is. For example, in the network security realm we may be feeding the system with collated samples of firewall logs, DNS traffic samples, IP blacklists/whitelists, IPS alerts, etc. It’s important to note though that the “richer” the data set (i.e. the more possible features there could be), the more complex the problem is for the computer to solve and the longer it’ll take to train the system.

Now let’s switch back to that “big data” we originally talked about. In the duck realm we’re talking about all the birds within a national park (say). Meanwhile, in the network security realm, we may be talking about all the traffic observed in real-time across a corporate network and all the alerting instrumentation (e.g. firewalls, IPS, etc.)

I’m going to do some hand-waving here because it can get rather complex and there’s a lot of proprietary tweaks that can be undertaken here… but in one representation we can get our trained system to automatically group and cluster events on our network.

Using our original training data, or some other previously labeled datasets, it’s possible to automatically label the clusters output by a machine learning system.

For example, in the graphic above we see a number of unique clusters (or blobs if you insist). Through automatic labeling we know that the green blobs are types of ducks, the red blobs are various groupings of not ducks, and the gray blobs are clusters of previously unknown or unlabeled clusters – each one mathematically distinct from the other – based upon the features the system chose.

What the system can also do is assign a probability that the unknown clusters are associated with our +ve or -ve training sets. For example, in this particular graphical representation the proximity of the unlabeled clusters to labeled (and classified) clusters allows the system to assign a probability of whether the cluster is a duck or not a duck – even though the system had never seen these things before (i.e. “birds” the system hasn’t encountered before).

The next (and near final) stage is to manually label these new clusters. For example, we ask an ornithologist to look at each cluster of “ducks” and “not ducks” in turn and to label them… “rubber duckies”, “robot duckies”, and “Madagascar mallard ducks”.

Then, to improve our machine learning system further, we add these newly labeled clusters to our +ve and -ve training sets… and the system continues to learn and become more precise over time.

In addition, since we’ve now labeled these clusters, in the future we’re able to automatically flag new additions to these clusters and correctly label the duck (or threat).

And, if we’re a really smart CISO, we can use this clustering system (and labeled clusters) to automatically alert us to new threats or to initiate automatic network security actions – e.g. enable blocking of a new malicious URL, stop blocking a new cloud service delivering official updates to applications, etc.

The application of machine learning techniques to the toughest security problems facing business today has come along in leaps and bounds recently. However as with any buzz word that falls in to the hands of marketers and gets manipulated until it squeaks and glitters, or oozes onto every product in this year’s price list, senior technical staff need to take added care not to be bamboozled by well-practiced but meaningless word salad.

A little understanding of the concepts behind big data and machine learning can not only cut through the latest batch of sales promises, but can also form the basis of constructing a new generation of meaningful breach detection and incident response solutions.

INSIGHTS | May 23, 2013

Identify Backdoors in Firmware By Using Automatic String Analysis

By Ruben Santamarta

The Industrial Control Systems Cyber Emergency Response Team (ICS-CERT) this Friday published an advisory about some backdoors I found in two programmable gateways from TURCK, a leading German manufacturer of industrial automation products.

http://ics-cert.us-cert.gov/advisories/ICSA-13-136-01

Using hard-coded account credentials in industrial devices is a bad idea. I can understand the temptation among manufacturers to include a backdoor “support” mechanism in the firmware for a product such as this. This backdoor allows them to troubleshoot problems remotely with minimal inconvenience to the customer.

On the other hand, it is only a matter of time before somebody discovers these ‘secret’ backdoors. In many cases, it takes very little time.

The TURCK backdoor is similar to other backdoors that I discussed at Black Hat and in previous blog posts. The common denominator is this: you do not need physical access to an actual device to find its backdoor. All you have to do is download device firmware from the vendor’s website. Once you download the firmware, you can reverse engineer it and learn some interesting secrets.

For example, consider the TURCK firmware. The binary file contains a small header, which simplifies the reverse engineering process:

The offset 0x1c identifies the base address. Directly after the header are the ARM opcodes. This is all the information you need to load the firmware into an IDA disassembler and debugger.

A brief review revealed that the device has its own FTP server, among other services. I also discovered several hard-coded credentials that are added when the system starts. Together, the FTP server and hard-coded credentials are a dangerous combination. An attacker with those credentials can completely compromise this device.

Find Hidden Credentials Fast

You can find hidden credentials such as this using manual analysis. But I have a method to discover hidden credentials more quickly. It all started during a discussion with friends at the LaCon conference regarding these kinds of flaws. One friend (hello @p0f ! ) made an obvious but interesting comment: “You could just find these backdoors by running the ‘strings’ command on the firmware file, couldn’t you?”

This simple observation is correct. All of the backdoors that are already public (such as Schneider, Siemens, and TURCK) could have been identified by looking at the strings … if you know common IT/embedded syntax. You still have to verify that potential backdoors are valid. But you can definitely identify suspicious elements in the strings.

There is a drawback to this basic approach. Firmware usually contains thousands of strings and it can take a lot of time to sift through them. It can be much more time consuming than simply loading the firmware in IDA, reconstructing symbols, and finding the ‘interesting’ functions.

So how can one collect intelligence from strings in less time? The answer is an analysis tool we created and use at IOActive called Stringfighter.

How Does Stringfighter Work?

Imagine dumping strings from a random piece of firmware. You end up with the following list of strings:

[Creating socket]

ERROR: %n

Initializing

Decompressing kernel

GetSrvAddressById

NO_ADDRESS_FOUND

Xi$11ksMu8!Lma

From your security experience you notice that the last string seems different from the others—it looks like a password rather than a valid English word or a symbol name. I wanted to automate this kind of reasoning.

This is not code analysis. We only assume certain common patterns (such as sequential strings and symbol tables) usually found in this kind of binary. We also assume that compilers under certain circumstances put strings close to their references.

As in every decision process, the first step is to filter input based on selected features. In this case we want to discover ‘isolated’ strings. By ‘isolated’ I’m referring to ‘out-of-context’ elements that do not seem related to other elements.

An effective algorithm known as the ‘Levenshtein distance’ is frequently used to measure string similarity:

http://en.wikipedia.org/wiki/Levenshtein_distance

To apply this algorithm we create a window of n bytes that scans for ‘isolated’ strings. Inside that window, each string is checked against its ‘neighbors’ based on several metrics including, but not limited to, the Levenshtein distance.

However, this approach poses some problems. After removing blocks of ‘related strings,’ some potentially isolated strings are false positive results. For example, an ‘isolated’ string may be related to a distant ‘neighbor.’

We therefore need to identify additional blocks of ‘related strings’ (especially those belonging to symbol tables) formed between distant blocks that later become adjacent.

To address this issue we build a ‘first-letter’ map and run this until we can no longer remove additional strings.

At this point we should have a number of strings that are not related. However, how can we decide whether the string is a password? I assume developers use secure passwords, so we have to find strings that do not look like natural words.

We can consider each string as a first-order Markov source. To do this, use the following formula to calculate its entropy rate:

http://en.wikipedia.org/wiki/Entropy_(information_theory)#Data_as_a_Markov_process

We need a large dictionary of English (or any other language) words to build a transition matrix. We also need a black list to eliminate common patterns.

We also want to avoid the false positives produced by symbols. To do this we detect symbols heuristically and ‘normalize’ them by summing the entropy for each of its tokens rather than the symbol as a whole.

These features help us distinguish strings that look like natural language words or something more interesting … such as undocumented passwords.

The following image shows a Stringfighter analysis of the TURCK BL20 firmware. The first string in red is highlighted because the tool detected that it is interesting.

In this case it was an undocumented password, which we eventually confirmed by analyzing the firmware.

The following image shows a Stringfighter analysis of the Schneider NOE 771 firmware. It revealed several backdoor passwords (VxWorks hashes). Some of these backdoor passwords appear in red.

Stringfighter is still a proof-of-concept prototype. We are still testing the ideas behind this tool. But it has exceeded our initial expectations and has detected backdoors in several devices. The TURCK backdoor identified in the CERT advisory will not be the last one identified by IOActive Labs. See you in the next advisory!

INSIGHTS | May 7, 2013

Bypassing Geo-locked BYOD Applications

By Gunter Ollmann

In the wake of increasingly lenient BYOD policies within large corporations, there’s been a growing emphasis upon restricting access to business applications (and data) to specific geographic locations. Over the last 18 months more than a dozen start-ups in North America alone have sprung up seeking to offer novel security solutions in this space – essentially looking to provide mechanisms for locking application usage to a specific location or distance from an office, and ensuring that key data or functionality becomes inaccessible outside these prescribed zones.

These “Geo-locking” technologies are in hot demand as organizations try desperately to regain control of their networks, applications and data.

Over the past 9 months I’ve been asked by clients and potential investors alike for advice on the various technologies and the companies behind them. There’s quite a spectrum of available options in the geo-locking space; each start-up has a different take on the situation and has proposed (or developed) a unique way in tackling the problem. Unfortunately, in the race to secure a position in this evolving security market, much of the literature being thrust at potential customers is heavy in FUD and light in technical detail.

It may be because marketing departments are riding roughshod over the technical folks in order to establish these new companies, but in several of the solutions being proposed I’ve had concerns over the scope of the security element being offered. It’s not because the approaches being marketed aren’t useful or won’t work, it’s more because they’ve defined the problem they’re aiming to solve so narrowly that they’ve developed what I could only describe as tunnel-vision to the spectrum of threat organizations are likely to face in the BYOD realm.

In the meantime I wanted to offer this quick primer on the evolving security space that has become BYOD geo-locking.

Geo-locking BYOD

The general premise behind the current generation of geo-locking technologies is that each BYOD gadget will connect wirelessly to the corporate network and interface with critical applications. When the device is moved away from the location, those applications and data should no longer be accessible.

There are a number of approaches, but the most popular strategies can be categorized as follows:

Thick-client – A full-featured application is downloaded to the BYOD gadget and typically monitors physical location elements using telemetry from GPS or the wireless carrier directly. If the location isn’t “approved” the application prevents access to any data stored locally on the device.
Thin-client – a small application or driver is installed on the BYOD gadget to interface with the operating system and retrieve location information (e.g. GPS position, wireless carrier information, IP address, etc.). This application then incorporates this location information in to requests to access applications or data stored on remote systems – either through another on-device application or over a Web interface.
Share-my-location – Many mobile operating systems include opt-in functionality to “share my location” via their built-in web browser. Embedded within the page request is a short geo-location description.
Signal proximity – The downloaded application or driver will only interface with remote systems and data if the wireless channel being connected to by the device is approved. This is typically tied to WiFi and nanocell routers with unique identifiers and has a maximum range limited to the power of the transmitter (e.g. 50-100 meters).

The critical problem with the first three geo-locking techniques can be summed up simply as “any device can be made to lie about its location”.

The majority of start-ups have simply assumed that the geo-location information coming from the device is correct – and have not included any means of securing the integrity of that device’s location information. A few have even tried to tell customers (and investors) that it’s impossible for a device to lie about its GPS location or a location calculated off cell-tower triangulation. I suppose it should not be a surprise though – we’ve spent two decades trying to educate Web application developers to not trust client-side input validation and yet they still fall for web browser manipulations.

A quick search for “fake location” on the Apple and Android stores will reveal the prevalence and accessibility of GPS fakery. Any other data being reported from the gadget – IP address, network MAC address, cell-tower connectivity, etc. – can similarly be manipulated. In addition to manipulation of the BYOD gadget directly, alternative vectors that make use of private VPNs and local network jump points may be sufficient to bypass thin-client and “share-my-location” geo-locking application approaches.

That doesn’t mean that these geo-locking technologies should be considered unicorn pelts, but it does mean that organization’s seeking to deploy these technologies need to invest some time in determining the category of threat (and opponent) they’re prepared to combat.

If the worst case scenario is of a nurse losing a hospital iPad and that an inept thief may try to access patient records from another part of the city, then many of the geo-locking approaches will work quite well. However, if the scenario is that of a tech-savvy reporter paying the nurse to access the hospital iPad and is prepared in install a few small applications that manipulate the geo-location information in order to remotely access celebrity patient records… well, then you’ll need a different class of defense.

Given the rapid evolution of BYOD geo-locking applications and the number of new businesses offering security solutions in this space, my advice is two-fold – determine the worst case scenarios you’re trying to protect against, and thoroughly assess the technology prior to investment. Don’t be surprised if the marketing claims being made by many of these start-ups are a generation or two ahead of what the product is capable of performing today.

Having already assessed or reviewed the approaches of several start-ups in this particular BYOD security realm, I believe some degree of skepticism and caution is warranted.

— Gunter Ollmann, CTO IOActive

INSIGHTS | April 30, 2013

Fact or Fiction: Is Huawei a Risk to Critical Infrastructure?

By Gunter Ollmann

How much of a risk does a company like Huawei or ZTE pose to U.S. national security? It’s a question that’s been on many peoples lips for a good year now. Last year the U.S. House of Representatives Permanent Select Committee on Intelligence warned American companies to “use another vendor”, and earlier in that year the French senator and former defense secretary Jean-Marie Bockel recommended a “total prohibition in Europe of core routers and other sensitive IT equipment coming from China.” In parallel discussions, the United Kingdom, Australia and New Zealand (to name a few) have restricted how Huawei operates within their borders.

Last week Eric Xu – executive vice-president and one of the triumvirate jointly running Huawei – stunned analysts when he told them that Huawei was “not interested in the U.S. market any more.”

Much of the analysis has previously focused upon Huawei’s sizable and influential position as the world’s second largest manufacturer of network routers and switching technology – a critical ingredient for making the Internet and modern telecommunications work – and the fact that it is unclear as to how much influence (or penetration) the Chinese government has in the company and its products. The fear is that at any point now or in the future, Chinese military leaders could intercept or disrupt critical telecommunications infrastructure – either as a means of aggressive statecraft or as a component of cyber warfare.

As someone who’s spent many years working with the majority of the world’s largest telecommunication companies, ISP’s, and cable providers, I’ve been able to observe firsthand the pressure being placed upon these critical infrastructure organizations to seek alternative vendors and/or replace any existing Huawei equipment they may already have deployed. In response, many of the senior technical management and engineers at these organizations have reached out to me and to IOActive to find out how true these rumors are. I use the term “rumor” because, while reports have been published by various government agencies, they’re woefully lacking in technical details. You may have also seen François Quentin (chairman of the board of Huawei France) claiming that the company is a victim of “rumors”.

In the public technical arena, there are relatively few bugs and vulnerabilities being disclosed in Huawei equipment. For example, if you search for CVE indexed vulnerabilities you’ll uncover very few. Compared to the likes of Cisco, Juniper, Nokia, and most of the other major players in routing and switching technology, the number of public disclosures is miniscule. But this is likely due to a few of the following reasons:

The important Huawei equipment isn’t generally the kind of stuff that security researchers can purchase off Ebay and poke around with at home for a few hours in a quest to uncover new bugs. They’re generally big-ticket items. (This is the same reason why you’ll see very few bugs publicly disclosed in Cisco’s or Nokia’s big ISP-level routers and switches).
Up until recently, despite Huawei being such a big player internationally, they haven’t been perceived as such to the English-speaking security researcher community – so have traditionally garnered little interest from bug hunters.
Most of the time when bugs are found and exploitable vulnerabilities are discovered, they occur during a paid-for penetration test or security assessment, and therefore those findings belong to the organization that commissioned the consulting work – and are unlikely to be publicly disclosed.
Remotely exploitable vulnerabilities that are found in Huawei equipment by independent security researchers are extremely valuable to various (international) government agencies. Any vulnerability that could allow someone to penetrate or eavesdrop at an international telecommunications carrier-level is worth big bucks and will be quickly gobbled up. And of course any vulnerability sold to such a government agency most certainly isn’t going to be disclosed to the vulnerable vendor – whether that be Huawei, Cisco, Juniper, Nokia, or whatever.

What does IOActive know of bugs and exploitable vulnerabilities within Huawei’s range of equipment? Quite a bit obviously – since we’ve been working to secure many of the telecommunications companies around the world that have Huawei’s top-end equipment deployed. It’s obviously not for me to disclose vulnerabilities that were uncovered on the dime of an IOActive client, however many of the vulnerabilities we’ve uncovered during tests have given great pause to our clients as remedies are sought.

Interesting enough, the majority of those vulnerabilities were encountered using standard network discovery techniques – which to my mind is just scratching the surface of things. However, based upon what’s been disclosed in these afore mentioned government reports over the last year, that was probably their level of scrutinization too. Digging deeper in to the systems reveals more interesting security woes.

Given IOActive’s expertise history and proven capability of hardware hacking, I’m certain that we’d be able to uncover a whole host of different and more significant security weaknesses in these critical infrastructure components for clients that needed that level of work done. To date IOActive the focus has be on in-situ analysis – typically assessing the security and integrity of core infrastructure components within live telco environments.

I’ve heard several senior folks talk of their fears that even with full access to the source code that that wouldn’t be enough to verify the integrity of Chinese network infrastructure devices. For a skillful opponent, that is probably so, because they could simply hide the backdoors and secret keys in the microcode of the devices semiconductor chips.

Unfortunately for organizations that think they can hide such critical flaws or backdoors at the silicon layer, I’ve got a surprise for you. IOActive already has the capability strip away the layers of logic within even the most advanced and secure microprocessor technologies out there and recover the code and secrets that have been embedded within the silicon itself.

So, I’d offer a challenge out there to the various critical infrastructure providers, government agencies, and to manufacturers such as Huawei themselves – let IOActive sort out the facts from the multitude of rumors. Everything you’ve probably been reading is hearsay.

Who else but IOActive can assess the security and integrity of a technology down through the layers – from the application, to the drivers, to the OS, to the firmware, to the hardware and finally down to the silicon of the microprocessors themselves? Exciting times!

— Gunter Ollmann, CTO IOActive Inc.

INSIGHTS | April 18, 2013

InfoSec Europe 2013 – Security on Tap

By Gunter Ollmann

It’s that time of the year again as Europe’s largest and most prestigious information security conference “Infosecurity Europe” gets ready to kick off next week at Earls Court, London, UK.

This year’s 18th annual security gathering features over 350 exhibitors, but you won’t find IOActive on the floor of the conference center. Oh no, we’re pulling out all the stops and have picked a quieter and more exclusive location to conduct our business just around the corner. After all, why would you want to discuss confidential security issues on a floor with 12,500 other folks?

We all know what these conferences are like. We psych ourselves up for a couple of days for shuffling from one booth to the next, avoiding eye contact with the glammed-up booth-babes working their magic on blah-blah’s vendor booth – who’s only mission in life is to scan your badge so that a far-off marketing team can spam you for the next 6 months with updates about a product you had no interest in – who you unfortunately allowed to scan your badge because they were giving away an interesting foam boomerang (that probably cost 20 pence) which you thought one of your kids might like as recompense for the guilt you’re feeling at having to be away from home one evening so you could see all of what the conference had to offer.

Well fret no more, IOActive have come to save the day!

After you’ve grown tired and wary of the endless shuffling, avoided eye-contact for as long as possible, grabbed enough swag to keep the neighbors grandchildren happy for a decade’s worth of birthdays, and the imminent prospect of standing in queues for over priced tasteless coffee and tea has made your eyes roll further in to the back of your skull one last time, come visit IOActive down the street at the pub we’ve taken over! Yes, that’s right, IOActive crew have taken hostage a pub and we’re inviting you and a select bunch of our VIP’s to come join us in a more relaxed and conducive business environment.

I hear tell that the “Security on Tap” will include a range of fine ales, food and other refreshments, and that the decibel level should be a good 50dB lower than Earls Court Conference Center. A little birdy also mentioned that there may be a whisky tasting going on at some point too. Oh, and there’ll be a bunch of us IOActive folks there too. Chris Valasek and I, along with some of our top UK-based consultants will be there talk about the latest security threats and evil hackers. I think there’ll be some sales folks there too – but don’t worry, we’ll make sure that their badge readers don’t work.

If you’d like to join us for drinks, refreshments and intelligent conversation in a venue that’s comfortable and won’t have you going hoarse (apparently “horse” is bad these days) – you’re cordially invited to join us at the Courtfield Pub (187 Earls Court Road, Earls Court, London, SW5 9AN). We’ll be there Tuesday and Wednesday (April 23rd & 24th) between 10:00 and 17:00.

— Gunter Ollmann, CTO IOActive

INSIGHTS | April 16, 2013

Can GDB’s List Source Code Be Used for Evil Purposes?

By Alejandro Hernandez

One day while debugging an ELF executable with the GNU Debugger (GDB), I asked myself, “How does GDB know which file to read when you use the list command?” (For the uninformed, the list command prints a specified number of lines from a source code file -— ten lines is the default.)

Source code filenames are contained in the metadata of an ELF executable (in the .debug_line section, to be exact). When you use the list command, GDB will open(), read(), and display the file contents if and only if GDB has the permissions needed to read the source file.

The following is a simple trick where you can use GDB as a trampoline to read a file which originally you don’t have enough permission to read. This trick could also be helpful in a binary capture-the-flag (CTF) or reverse engineering challenge.

Here are the steps:

1. Compile ‘foo.c‘ with the GNU Compiler (GCC) using the -ggdb flag.

2. Open the resulting ELF executable with GDB and the list command to read its source code as shown in the following screen shot:

3. Make a copy of ‘foo.c’ and call it ‘_etc_shadow.c’, so that this name is hardcoded within the internal metadata structures of the compiled ELF executable as in the following screen shot.

4. Open the executable with your preferred hex editor (I used HT Editor because it supports the ELF file format) and replace ‘_etc_shadow.c’ with ‘/etc/shadow’ (don’t forget the NULL character at the end of the string) the first two times it appears.

5. Evidently, it won’t work unless you have sufficient user privileges, otherwise GDB won’t be able to read /etc/shadow.

6. If you trace the open() syscall calls executed by GBD:

($strace -e open gdb ./_etc_shadow)

you can see that it returns -1 (EACCES) because of insufficient permissions.

7. Now imagine that for some reason GDB is a privileged command (the SUID (Set User ID) bit in the permissions is enabled). Opening our modified ELF file with GDB, it would be possible to read the contents of ‘/etc/shadow’ because the gdb command would be executed with root privileges.

8. Imagine another hypothetical scenario: a hardened development (or CTF) server that has been configured with granular privileges using a tool such as Sudo to allow certain commands to be executed. (To be honest I have never seen a scenario like this before, but it’s an example worth considering to illustrate how this attack might evolve).

9. You cannot display the contents of‘/etc/shadow’ by using the cat command because /bin/cat is an unauthorized command in our configuration. However, the gdb command has been authorized and therefore has the rights needed to display the source file (/etc/shadow):

Voilà!

Taking advantage of this GDB feature and mixing it with other techniques could make a more sophisticated attack possible. Use your imagination.

Do you have other ideas how this could be used as an attack vector, either by itself or if combined with other techniques? Let me know.

INSIGHTS | April 10, 2013

What Would MacGyver Do?

By Sofiane Talmat

“The great thing about a map: it gets you in and out of places in a lot different ways.” – MacGyver

When I was young I was a big fan of the American TV show, MacGyver. Every week I tuned in to see how MacGyver would build some truly incredible things with very basic and unexpected materials — even if some of his solutions were hard to believe. For example, in one episode MacGyver built a futuristic motorized heat-seeking gun using only a set of batteries, an electric mixer, a rubber band, a serving cart, and half a suit of armor.

From that time I always kept the “What would MacGyver do?” spirit in my thinking. On the other hand I think I was “destined” to be an IT guy, and particularly in the security field, where we don’t have quite the same variety of materials to craft our solutions.

But the “What would MacGyver do?” frame of mind helped me figure out a simple way to completely “own” a network environment in a MacGyver sort of way using just a small set of steps, including:

Exploiting a bad use of tools.
A small piece of social engineering.
Some creativity.
A small number of manual configuration changes..

I’ll relate how I lucked into this opportunity, how easy it can be to exploit certain circumstances, and especially how easy it would be to use a similar technique to gain domain administrator access for a company network.

The whole situation was due to the way helpdesk support was provided at the company I was working for. For security reasons non-administrative domain users were prevented from installing software on their desktops. So when I tried to install a small software application I received a very respectful “access denied” message. I felt a bit annoyed by this but still wanted my application, so I called the helpdesk and asked them to finish the installation remotely for me.

The helpdesk person was amenable, so I gave him my machine name and soon saw a pop-up window indicating that someone had connected to my machine and was interacting with my desktop.

My first impression was “Oh cool, this helpdesk is responsive and soon my software will be installed and I can finally start my project.”

But when I thought about this a bit more I started to wonder how the helpdesk person could install my software since he was trying to do so with my user privileges and desktop session rather than logging me out and connecting as an administrator.

And here we arrive at our first Act.

Act #1: Bad use of tools

Everything became clear when the helpdesk guy emulated a Ctrl+Alt+Delete combination that brings up the Windows menu and its awesome Switch User option.

The helpdesk guy clicked the Switch User option and here I saw some magic — namely the support guy logging in right before my eyes with the local Administrator account.

Picture this: the support guy was typing in the password directly in front of my eyes. Even though I am an IT guy this was the first time I ever saw a support person interacting live with the Windows login screen. I wished I could see or intercept the password, but unfortunately I only saw ugly black dots in the password dialog box.

At that moment I felt frustrated, because I realized how close I was to having the local administrator password. But how could I get it?

The magic became clearer when the support guy logged in as an administrator on the machine and I was able to see him interacting with the desktop. That really made my day.

And then something even more magnificent happened while I was watching: for some undefined reason the support guy encountered a Windows session error. He had no choice but to log out, which he did, and then he logged in again with the Domain Administrator account … again right before my eyes!

(I don’t have a domain lab set up right now, so I can’t duplicate the screen image, but I am sure you can imagine the login window, which would look just like the one above except that it would include the domain name.)

When he logged in as the domain administrator I saw another nice desktop and the helpdesk guy interacting with my machine right in front of my eyes as the domain admin.

This is when I had a devious idea: I moved my mouse one millimeter and it moved while this guy was installing my software. At this point we arrive at the second Act.

Act #2: Some MacGyver magic

I asked myself, what if I did the following:

Unplug the network cable (I could have taken control of the mouse instead, but that would have aroused suspicion).
Stop the DameWare service that is providing access to the support guy.
Reconnect the network cable.
Create a new domain admin account (since the domain administrator is the operative account on my computer).
Restart the DameWare service.
Log out of the machine.

By completing these six steps that wouldn’t take more than two minutes I could have assumed domain administrator privileges for the entire company.
Let’s recap the formula for this awesome sauce:

1. Bad use of tools: It was totally wrong for the help desk person to open a domain admin session directly under the user’s eyes and giving him the opportunity to take control of the session.

2. A small piece of social engineering: Just call the support desk and ask them to install some software for you.

3. A small amount of finagling on your part: Use the following steps when the help desk person logs in to push him to log in as Domain Admin:

• Unplug the network cable (1 second).
• Change the local administrator password (7 seconds).
• Log out (2 seconds).
• Plug the network cable back in (1 second).

4. Another small piece of social engineering: Call the support person back and blame Microsoft Windows for a crash. Cross your fingers that after he is unable to login as local admin (because you changed the password) he will instead login as a domain administrator.

5. Some more finagling on your part: Do the same steps defined in step 3 to create a new domain admin account.

6. Success: Enjoy being a domain administrator for the company.

Final Act: Conclusion

At the beginning of my IT security career I was a bit skeptical of the magic of social engineering. But through the years I have learned that social engineering still works and will always work. Even if the social engineering involves the tiniest and most basic request, if you combine social engineering with some imagination you can own a huge company. And if you are conducting a pentest you don’t always have to rely exclusively on your technical expertise in pentesting. You can draw on your imagination and creativity to build a powerful weapon using small and basic tools …. just like MacGyver.

INSIGHTS | April 2, 2013

Spotting Fake Chips in the Supply Chain

In the information security world we tend to focus upon vulnerabilities that affect the application and network architecture layers of the enterprise and, every so often, some notable physical devices. Through various interrogatory methods we can typically uncover any vulnerabilities that may be present and, through discussion with the affected business units, derive a relative statement of risk to the business as a whole.

An area of business rarely dissected from an information security perspective however is the supply chain. For manufacturing companies and industrial suppliers, nothing is more critical to their continued business success than maintaining the integrity and reliability of their supply chain. For some industries – such as computer assembly or truck fabrication facilities – even the smallest hiccup in their just-in-time ordering system can result in entire assembly lines being gummed up and product not being rolled out the front door.

The traditional approach to identifying vulnerabilities within the supply chain is largely through a paper-based audit process – sometimes top-and-tailed with a security assessment of PC-based assessments. Rarely (if ever) are the manufacturing machines and associated industrial control systems included in physical assessments or penetration tests for fear of disrupting the just-in-time manufacturing line.

Outside the scope of information security assessment, and often beyond the capabilities of automated quality assurance practices within an organizations assembly line, lies the frailty of being victim to failure of a third-party supplier’s tainted supply chain.

For example, let’s look at a common microprocessor ordered through a tainted supply chain.

Dissecting a ST19XT34 Microprocessor

In early 2012 samples of the ST ST19XT34 were ordered from https://us.hkinventory.com/. The ST19XT34 is a secure microprocessor designed for very large volume and cost-effective secure portable applications (such as smartcards used within Chip&PIN technologies). The ST19X platform includes an internal Modular Arithmetic Processor (MAP) and DES accelerator – designed to speed up cryptographic calculations using Public Key Algorithms and Secret Key Algorithms.

The ST19XT34 chips that IOActive were charged to investigate were encapsulated within a standard SOIC package and were supposed to have 34kb of EEPROM.

Upon visual analysis the devices appeared to be correct. However, after decapsulation, it was clear that the parts provided were not what had been ordered.

In the above image we have a ‘fake’ ST19XT34 on the left with a sample of the genuine chip on the right. It is almost impossible to tell the left device was altered unless you have a known original part.

After decapsulation of the various parts it was easy to immediately recognize the difference between the two SOIC part. The left ‘fake’ device was actually an ST ST19AF08 with the right being the genuine ST19XT34.

The ST19AF08 is a 600 nanometer 3 metal device (on left). It contains an 8 KB EEPROM.

The ST19XT34 is a 350 nanometer 3 metal device (on right). It contains a 34 KB EEPROM making the die much larger than the older and smaller sized device.

Microprocessor Supply Chain Frailty

As the example above clearly shows, it is often very difficult to identify a tainted supply chain. While an x-ray in the above case could also have verified the integrity of the supplier if it had been part of the quality assurance process, it would not have detected more subtle modifications to the supplied microprocessors.

If it is so easy to taint the supply chain and introduce fraudulently marked microprocessors, how hard is it to insert less obvious – more insidious – changes to the chips? For example, what if a batch of ST19XT34 chips had been modified to weaken the DES/triple-DES capabilities of the chip, or perhaps the random number generator was rigged with a more predictable pseudo random algorithm – such that an organized crime unit or government entity could trivially decode communications or replay transactions?

The frailty of today’s supply chain is a genuine concern for many. The capability of organized crime and foreign government entities to include backdoors, add malicious code, or subvert “secure” routines within fake or counterfeit microprocessors is not a science fiction story, but something that can occur today. The ability to inject these modified chips in to the supply chain of any global manufacturer of goods is, quite frankly, trivial.

The cost of entry for organized criminals and government entities to undertake this kind of manipulation of the supply chain is high, but well within their financial capabilities – and, more importantly, they could reap great rewards from their investment.

Identifying a tainted supply chain is not a simple task. It requires specialized equipment capable of dissecting microprocessors at the nanometer scale, fiddly extraction of microcode, and skilled security analysts to sift through the code looking for backdoors and deliberate weaknesses in the algorithms.

It’s an expensive and time consuming proposition – but fast becoming a critical component when it comes to assuring that today’s smartphones, Chip&PIN technologies and critical infrastructure control systems aren’t subject to organized subversion.