Author Archives: Soroush Dalili

About Soroush Dalili

Web application security researcher and penetration tester.

Additional notes on “A Forgotten HTTP Invisibility Cloak” talk!

As I could not share the video for my BSides Manchester 2017 talk due to naming a few vendors, I thought the best plan was to explain a few things in a casual blog post. Most of the techniques explained in the presentation were not new although I did not know that initially! In some cases, I was 15 years late to the bypass partei; therefore, I have included links in the references for those who want to read more about them. I should mention that the request encoding technique is particularly the one I am proud of, as I still have not seen it anywhere else.

I think all we need now is the presentation file which has been shared already but here is the PDF version of the original PowerPoint file (to see the slides without SlideShare changes!): https://soroush.secproject.com/downloadable/A_Forgotten_HTTP_Invisibility_Cloak_v1.1.pdf

The http.ninja website is currently linked to a Github repository that contains the Python code I had used to perform the initial tests on my targets. My plan is to update this website with a cheat-sheet of different setups and different test cases. If you have an open source template in mind that can help me with this, please let me know via my Twitter.

So let us begin with the notes about some of these slides (the rests are hopefully self-explanatory and/or mentioned in the references):

Slide #13 (Strange Mutations):

These are not the only methods; some other documented behaviours can be seen in https://drive.google.com/file/d/0B5Tqp73kQStQU1diV1Y0dzd1QU0/view. That said, there are probably loads of other test cases that I have not tried yet and will be a great area for research.

Some of the methods explained in this slide could help us bypass protections that were based on the application path or the parameters.

HTTP v0.9 with full header supports on Apache Tomcat is a lovely feature/bug that can confuse loads of WAFs.

Slide #16 (HTTP Verb Replacement):

This method was only useful for those of us who were testing IIS. It was not really necessary to do so in order to bypass the WAFs as shown on the next slides.

Slide #23 (Removing Unnecessary Parts):

The last boundary “- -1- -” could be fully removed if it was a PHP application on Apache or IIS or on Nginx-uWSGI-Django-Python2/3 setup.

Slide #28 (Adding Useless Parts!):

There is just some space characters after the “filename” and before the “=” sign. This is to make it an invalid file upload request and therefore it would remain a normal POST request parameter as before. This mutation on its own can bypass some famous cloud-based WAFs.

Slide #29 – #31 (Changing Request Encoding!):

If charset was used before the boundary, a comma character should have been used as the delimiter between them. Otherwise, a semicolon character should have separated them.

Slideshare has changed the encoded values incorrectly. To see the actual values, you can view them in the PDF file of the presentation.

In order to find more information about this technique, please see the following: https://soroush.secproject.com/downloadable/request-encoding-to-bypass-web-application-firewalls.pdf

Slide #42 (HTTP Pipelining Example):

If you are using Burp Suite or other web proxies, ensure that the auto update Content-Length feature is disabled.

I, however recommend that to use direct socketing to see the full response as sometimes web proxies do not show it, although you can see it using Wireshark. I have created a simple Python code to do so that you can get it from https://github.com/irsdl/httpninja/blob/master/Generic%20Codes/web_request_socket.py

Fun fact: HTTP Pipelining can be used to exploit DoS issues, smuggle HTTP requests, or to exploit race condition flaws. However, that is not all it can do! Although I am not giving anybody any ideas, HTTP Pipelining can help you to book your precious online ticket faster than others. This is when you need to send more than one request to book your ticket but the following requests do not use a parameter taken from the previous responses.

References:

I had used the following references when working on my original research (I have added them to the slides as well now):

http://www.cgisecurity.com/lib/HTTP-Request-Smuggling.pdf

http://www.ussrback.com/docs/papers/IDS/whiskerids.html

https://media.defcon.org/DEF%20CON%2024/DEF%20CON%2024%20presentations/DEFCON-24-Regilero-Hiding-Wookiees-In-Http.pdf

https://securityvulns.ru/advisories/content.asp

https://dl.packetstormsecurity.net/papers/general/whitepaper_httpresponse.pdf

https://cdivilly.wordpress.com/2011/04/22/java-servlets-uri-parameters/

https://msdn.microsoft.com/en-us/library/system.text.encodinginfo.getencoding.aspx

I also found this afterwards so while not using it as my reference, it is still related and is a good read:

https://blog.qualys.com/wp-content/uploads/2012/07/Protocol-Level%20Evasion%20of%20Web%20Application%20Firewalls%20v1.1%20(18%20July%202012).pdf

Final Notes:

This talk was a reserved talk and was not planned as a main one; I had been assured that the probability of this happening was about %0.00…01; but hey, even that can win you the lottery sometimes! As one of the first presenters at 9am did not show up on the day, I had to deliver my talk with about 10 minutes notice so I should say thanks to the audience who did not leave the room despite having a different show and had to listen to my random presentation – you are awesome! :)

And here is an interesting Tweet about this:

And yes, BeerSides Manchester was awesome too with some consequences:

Request encoding to bypass web application firewalls

I “think” I have discovered a “new” technique in bypassing external web application firewalls using request encoding. The idea is very simple but I had not seen this before to be used to bypass any protection mechanisms. Details of this technique has been published via NCC Group’s blog:

https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/august/request-encoding-to-bypass-web-application-firewalls/

This technique was presented as one the methods to mutate the HTTP requests in:

There are loads of other anomalies that can be used to bypass WAFs using webservers behaviour in accepting HTTP requests; my plan is to complete this research and put all the results via the https://http.ninja/ website. Please feel free to contact me via my Twitter (@irsdl) if you have some ideas regarding this.

The unofficial PDF version of this blog post can be downloaded from here:
https://soroush.secproject.com/downloadable/request-encoding-to-bypass-web-application-firewalls.pdf

Almost all the cloud-based WAFs that I had tested could be bypassed using this technique at the time of discovery.

When a web application SSRF causes the cloud to rain credentials & more

The following blog post was written by me and Daniele Costa:

https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/august/when-a-web-application-ssrf-causes-the-cloud-to-rain-credentials-and-more/

In this blog post we have demonstrated an SSRF exploitation to steal AWS credentials to access Amazon S3. What made this attack special was the fact that http://169.254.169.254/latest/meta-data/iam/ was not accessible to our users during the exploitation. Therefore, we had to use the ‘userData’ attribute in EC2 describe-instance-attribute operation to extract the sensitive data.

The unofficial PDF version of this blog post can be downloaded from here:

https://soroush.secproject.com/downloadable/when-a-web-application-ssrf-causes-the-cloud-to-rain-credentials-and-more.pdf

CVE-2017-8592 – XMLHttpRequest in IE followed 307 redirections with additional or customised headers

If you are doing web application security assessments, you probably have seen many APIs that do not have any cross-site request forgery (CSRF) protections other than checking the Content-Type header.

Although this seems like a bad way of doing this, browsers do not normally allow you to use unusual Content-Type headers for another website (without CORS) and therefore the exploitation is quite impossible. However, a vulnerability was found in Internet Explorer and Edge that allowed users to set these arbitrary headers (anything that JavaScript can set). This issue was patched in July 2017. More details can be seen here:

https://www.nccgroup.trust/uk/our-research/technical-advisory-cve-2017-8592-xmlhttprequest-in-ie-followed-307-redirections-with-additional-or-customised-headers/

As mentioned in the above URL, bypassing security features using redirection is a common attack vector and it was interesting that this issue had not been reported to Microsoft previously. A similar issue in Adobe Flash had been patched a long time ago (see https://www.whitehatsec.com/blog/flash-307-redirect-game-over/).

Using Firefox Profiles in Security Testing

I have started using browser profiles for normal web application assessment since working at NCC Group due to the number of projects I had since the 1st week! I thought it is a good idea to share my solution as I have seen other people facing the same issue when dealing with multiple web application testing projects.

Why should we use a new browser profile for testing?

There are a number of reasons to use separate browser profiles per web application assessments. I have listed some of these reasons from my point of view:

A clean browser environment can be useful to keep the history/data especially when re-testing the same target.

Browser that is used for testing is normally unsafe for day to day browsing. For instance, client-side protections such as XSS auditors or NoScript add-on in Firefox should be disabled while testing in order to detect client-side issues properly. Proxies’ or other self-signed root certificates could be imported in the browser during the test.

It is sometimes necessary to use the private browsing option within the same browser with all the add-ons in order to test session management or broken access control issues (or perhaps to test an issue very quickly using another session but with the same user-agent). It should be noted that private browsing/incognito is not recommended for the actual testing as cache control, sensitive data in URL, or autocomplete issues might be missed and there will be no web history in the browser (especially useful for those times that the proxy is off for some reason).

Why should we automate this?

Having a fresh browser per each test with predefined settings and pre-installed add-ons/extensions can be time consuming if not automated.

I use Mozilla Firefox to make everything easier if the target website is not browser specific or if different user-agents in Firefox do not break the website functionality.

An easy way to do this using Firefox

Step 1: Creating a new empty profile (template)

A Firefox shortcut can be created or edited with the “-p” argument to show the “Choose User Profile” panel. Alternatively, the “-profilemanager” argument can be used to open this panel:

After clicking the “Create Profile” button, choose an appropriate profile name such as “Clean For Security Testing” to create a new profile (you can save it in a preferred location). This profile will be added to the list:

Now click the “Start Firefox” button to customise it.

Step 2: Preparing the template – Firefox settings

As Firefox hides the top menu automatically, we need to use the “Alt” key to access it. It is therefore my preferred choice to make it permanently visible by selecting “Menu Bar” from Views>Toolbars after pressing the “Alt” key. Throughout this post, I have used the top menu to point at items’ locations but the “Open Menu” button in Firefox can be used as well.

In order to reduce the number of requests Firefox sends automatically during testing and to improve the privacy, I normally do this:

– Untick all the boxes in Tools>Options>Advanced>Data Choices

– Untick all the boxes in Tools>Options>Advanced>Update and select “Never check for updates” – It is important to note that check for updates for both Firefox and its add-ons must be performed manually when using profiles.

– From “about:config”, set the value of “browser.newtabpage.enabled” and “network.captive-portal-service.enabled” to “false”.

– From “about:config”, set an empty string for “browser.selfsupport.url”

Other automated requests can be disabled using the Mozilla guideline ifnecessary: https://support.mozilla.org/en-US/kb/how-stop-firefox-making-automatic-connections

Step 3: Preparing the template – Proxy root certificates

Proxies’ root certificates should also be imported into the browser’s root certificate at this point to make it easier every time we use a new profile.

Step 4: Preparing the template – Installing required extensions / plugins / themes

Add-ons that are installed at this stage are not compulsory and it is up to you to install your favourites. However, the “FEBE” add-on should be installed to automate the process of creating new profiles based on the template:

https://addons.mozilla.org/en-GB/firefox/addon/febe/

http://softwarebychuck.com/febe/tutorial/febeRestoreProfile.html

CLEO extension can also be useful to back up all the extensions in a single file when needed!

My favourite generic extensions for testing are as follows:

ColorfulTabs: this helps me to instantly realise when the testing domain changes – so I can detect out-scoped or different domains using the tab colours! I have chosen “Generate Colors By Domain Hostname” in the extension configuration. I have also set the “Tab Border Radius” to 1 to fix the UI issue it may cause.

Firebug: I still like this extension although Firefox has implemented some of it internally now!

Form History Control: useful to create a quick screenshot for the report when reporting autocomplete issues!

FoxyProxy: very useful to manage different proxies. For instance, I have a few ports configured in it for different purposes and different proxies. If you configure it in your template profile, it will be backed up into your testing profiles as well.

HackBar: limited but quite handy extension to send quick GET/POST requests or to decode/encode something. I use its other features too and sometimes even use it like a notepad!!! As I do not like its panel to be always visible on top, I have added its icon (“Toggle HackBar”) to the address bar menu.

ProfileSwitcher: to switch between profiles quickly when needed!

Random Agent Spoofer: to change the user agent when needed very quickly.

Regular Expressions Tester: to test a Regular Expression when I have no better tools.

Web Developer: An old habit just like Firebug. Makes it easier to manipulate items in a HTML page. I used to use it to see the HTML/JS errors (to detect a lame XSS perhaps!) but it seems Firefox is doing a good job on that itself too.

Perhaps an awesome dark Firefox theme for the security template profile also helps. Something that is appropriate for the report screenshots of course! It may also help you to know when a security profile is in use so you will not use it for normal browsing.

The template is now ready. Do not use it for testing before restoring it into a new separate profile though!

Step 5: FEBE Options

Using Tools>FEBE, select “FEBE Options” and ensure that “Full profile” has been chosen. Then on the “Where to backup” menu in “FEBE Options”, select the backup files location and press the Ok button.

Step 6: Updates

You can ignore this step if you have already updated Firefox and its add-ons.

Ensure that Firefox is updated by going to Help>About Firefox from the top menu. Then from the top menu, go to Tools>Add-ons, select “Extensions” from the left menu and click on the settings button and choose “Check for Updates”. Restart the browser if it is required after the update.

Step 7: Clean up

It is time to clean the history before backing up the template to have a fresh start every time! You can ignore this step if there is nothing in the browser history.

From History>Clear Recent History select “Everything” and tick all the boxes and press “Clear Now”. From Tools>Advanced>Network, clear any remaining cached web contents.

Step 8: Backing up the template

Using Tools>FEBE, select the “Perform backup now” option.  Run it again if it was not successful the first time!

Step 9: Creating new profiles based on the template

When you are using the “Clean For Security Testing” profile, using Tools>FEBE, select the “Restore Profile” option:

First click on the “Create new profile” button and enter a name such as “BugBounty_Facebook”. Close any FEBE messages to make the process smoother.

Now, click on the “Select local backup to restore” button and choose the template profile:

After this stage, click on “Start profile restore” (twice if it did not work).

Step 10: Using the new profile

Now close Firefox and all its messages and open it again. The new profile should appear in the list:

Select it and start Firefox.

Creating another profile

Follow steps 6 to 10 whenever you need to create a new profile for testing now.

Quick start sample

I have created a backup file using my Firefox that you can download from here (https://soroush.secproject.com/downloadable/images/firefox_profile/profileFx53.0(FEBE8.9.3.1){Clean For Security Testing}.zip).

You need to have the FEBE extension installed to use this and to restore the “Clean For Security Testing” profile. Follow steps 6 to 10 after restoring this profile.