Author Archives: Soroush Dalili

About Soroush Dalili

Web application security researcher and penetration tester.

Bug Bounty vs Penetration Testing (Simple Unbiased Comparison)

I have listed some notes, based on my personal experience so far, about working with some limited direct and indirect bug bounty programs and performing vulnerability assessment. I hope this is useful for the readers; please feel free to contact me on my Twitter (@irsdl) to discuss any of these if you wish.

In my humble opinion, direct bug bounty programs and vulnerability assessments are both necessary although may not match some business models where one can be ineffective in practice. For example, a bug bounty program may not work for sensitive platforms or targets that are only accessible from certain locations or IP addresses. On the other hand, a broad time limited vulnerability assessment may not work for large open source applications that have been around and tested for quite some time. As far as I am aware, bug bounty programs cannot offer any assurance to the customers based on the regulations at the moment, but this may change in future.

By the way, I am not discussing indirect bug bounty programs in which a third party vendor pays for vulnerabilities of other vendors’ products.

My notes about each of these solutions are as follows:

Direct Bug Bounty Programs

Pros:

  • More testers to cover more targets and to find more vulnerabilities. However, normally in practice only attractive programs receive high-quality vulnerabilities.
  • More time or unlimited time to cover more targets and to find more vulnerabilities. Again, this happens when a program is attractive to highly skilled testers and treats them well.
  • They can change the scope dynamically at any time. That said, this can cause problems between the program and reporters who have already started looking at targets that are in the scope.
  • Only pays for valid/in-scope findings (not saying it cannot be expensive though).
  • Can prevent breaches by buying bugs for a good price.
  • It can help companies with a huge attack surface to identify hidden issues that were not identified throughout previous assessments.
  • Works well for open source or non-profitable applications when sponsors such as Internet Bug Bounty pays for it.
  • Customers may feel safer when a company security model is mature and is handling reported security issues gracefully.
  • Encourage responsible disclosure culture.
  • Testers might care about the patching process when their bounties depend on it so they might be eager to help with the retest. However, this might make program less attractive to those who want to be paid quickly.
  • It helps security researchers to legally test the targets and makes the responsible disclosure process simpler for anyone who might want to report a security issue.

Cons:

  • It is a competitive market. If a few companies pay more for vulnerabilities, it will make other programs less attractive and will reduce their quality. Therefore, quality in the bug bounty world is relative and those programs that cannot afford paying as much as giant companies for their security issues will be on the losing end.
  • Unless a program is attractive enough and is restricted to top bug bounty reporters in the world, the majority of testers in bug bounty are not experienced penetration testers and have less skills and are only after the low hanging fruits. As a result, there are high amount of false positives, fake, or duplicates from less experienced testers or those who have not read the bug bounty policy properly. In practice, less than %5 of reported issues can be valid for a popular program that increases human mistakes in handling real issues and put a high pressure on the triage team (https://www.synack.com/2017/05/05/managed-bug-bounties-quality-is-in-the-secret-sauce/).
  • Highly skilled penetration testers who are new to the bug bounty platforms cannot join private programs that are normally more attractive as they do not have enough points on the system. As a result, when most attractive programs are private, they can never climb up the point-based ladder without spending their time on less attractive programs which is not ideal.
  • The main problem with the penetration testing assessments is their time constraints because of limited budget. A Bug bounty program with limited time and limited scope is not even close to a penetration testing assessment and might not be useful at all unless it can attract good and strong testers.
  • Free or low paid programs can give customers false sense of security as they are not attractive to strong testers.
  • It is impossible to stop all the testers at once when needed as the communication method is normally via email and can be slow.
  • Newly developed functionality or less visible ones normally remain hidden to the testers. Therefore, a new section may remain untested while other more visible sections are being tested over and over again by different testers.
  • Testers are not normally known and therefore it is more difficult to trace their activities if needed in compare with a penetration testing. Attackers may also claim that they were after responsible disclosure.
  • Hard to design the right policy with a correct scope (requires knowledge of the whole system). Badly designed scope can reduce the effectiveness of bug bounty as well as damaging the vendor’s reputation regarding its security.
  • It may give attackers the opportunity to legally assess an organisation while having no intention to report the most serious issues. However, it is possible in some platforms to limit a penetration testing to certain testers who are better known because of their longer activities (this may make a bug bounty program private).
  • It may cause denial of service if a system is not ready for a heavy load as testers may use automated scanners even if using it is forbidden. Although systems are constantly being scanned automatically on the Internet, defining a bug bounty program on them will certainly increase the amount of scanners being used on the targets.
  • A lot of dispute over prioritising and paying for a vulnerability. Reporters constantly compare their issues with each other.
  • Mistrust over duplicate issues due to the lack of transparency in most of programs.
  • Reputation damage over solving issues badly (e.g. by ignoring the reports, not paying the right amount, or keeping the reporter in the dark while patching an issue)
  • Sorting the tax money is harder outside of the US as they use different currencies. Options in the UK are currently limited to use accountants, commercial software, or paper-based forms by using the correct currency conversion chart (if you receive them in different currencies than British Pound).

Vulnerability Assessment (commonly known as penetration testing*)

Pros:

  • An official report or certificate of security assessment can be produced by the penetration testing company. This can be presented to partners or other organisations when a proof of security assessment is needed.
  • A higher quality report in compare with bug bounty reports in general.
  • Full attention on the specific target as days and scope are pre-defined and will not be deviated from.
  • A responsible company that can be contacted and is accountable for their assessment and their testers.
  • Level of support after the assessment in order to rectify the issues.
  • No false-positives due to the consultants’ experience and the QA process. However, human errors can happen but at least it will be investigated and the report will be updated when it is incorrect.
  • Better communication between the vendor and the testers in order to resolve any possible issues.
  • Regardless of the number of issues identified by the penetration testers, all will be presented to the client for the same fixed cost.
  • Testing and progress visibility
  • Good assurance for the customers regarding the assessment coverage, quality of testing, and an accountable testing company.
  • Accepted by different regulations such as by PCI.

Cons:

  • Limited time of testing may reduce the likelihood of discovering well-hidden vulnerabilities that require more research or different skillset.
  • Vendors should normally pay additional fees for a retest when required as the retest might not be part of the original assessment.
  • Assigning more time to an assessment can be very expensive
  • Human factor plays an important role as number of testers on each assessment are limited. For example, experience and knowledge of assigned testers on an assessment and how they feel during the test plays an important role in identifying security issues! However, in a bug bounty program anyone can look at the system at the same time or later in time.
  • Difficult to change the scope during an assessment or assign more testers upon a short notice as resources are normally limited and testers’ specialities and skills are different.

I do like Bug Bounties! For me, it is like catching a fish! 🎣

With all that, bug bounty is still young and has a great future ahead!

A number of the following suggestions have already been implemented in some of the well-known bug bounty platforms but not in all.

Recommendations to Improve Bug Bounty Platforms **:

  • Giving people access to previously reported issues when they have reported a duplicate issue to make it clearer (similar to what Mozilla does in their issue tracker). If the duplicate issues have been received from another source, perhaps an additional tracking number and name of the finder can help people understand that the vendor is not cheating on them by announcing an issue as a duplicate.
  • Registering testers’ mobile number to keep them up to date with the latest changes when they sign up for a certain bug bounty program. The scope can only be visible to those who have signed up and confirmed a text message. Testers can then opt out if they are not interested anymore in receiving text messages about scope changes or any issue with the testing.
  • A verification process (perhaps in different levels) in which the consultants can provide their identification documents, certificates, address, and CV to the bug bounty platform in order to make their account verified. This can be used if vendors only want to pass their bug bounty to trusted/known people based on their verification level or their certificates (not just their bug bounty reputation).
  • A VPN service for the verified testers so they can always use certain IP addresses while testing. This gives vendors the opportunity to design appropriate firewall rules and to make their testing system only accessible to certain IPs.
  • A survey for those who have reported vulnerabilities measuring their investment in time or other resources, their view on the specific bug bounty program’s performance alongside possible vendor’s view about the tester. This data can be publicly available upon approval from the tester and the vendor.
  • A transparent platform to show statistics of every bug bounty programs. This should include but not be limited to the following information selectable/searchable by date: number of reported/accepted/duplicate/invalid issues with their severity, number of top 100 testers (or if there is another measure to show strong testers) reported issues, average amount of days to fix the issues/pay the bounty, average amount of bounty based on issue severity, number of accepted but unfixed issues.
  • Defining a consultancy system for the vendors so they can refine their scope based on feedbacks from the testers.
  • Creating a centralised platform to include the bug bounty targets for research purposes. It is often the case the people go to a great length to obtain a list like this that can be used for research purposes. For instance, to check a specific vulnerability on targets that use a certain technology.
  • Working with the governments to make it easier for testers to pay their tax or to give the money to charities outside the US.

Improvements for Bug Bounty Program Owners **:

A bug bounty program will be successful when it can attract strong testers to spend some time assessing the targets. It is important to design and perform it in a way that it encourages testers to come back to spend more time and report more vulnerabilities.

The following list shows some of the features that a bug bounty program should have to be popular among testers:

  • Clear scope and exclusion list
  • Quick acknowledgement of receiving issues
  • Quick decision on issues validity for the bounty
  • Reasonably short patching time
  • Quick pay-out (e.g. on validity rather than resolution when patching will take time)
  • Generous rewards for the findings, good quality reports, and help in retesting
  • Surprise rewards on nice findings and reports even when they are out of the scope
  • Supporting multiple payment options
  • Polite and professional triage team that can handle difficult cases gracefully

 

 

* IMHO, they are different though but that discussion is not the topic!
** whoami to comment on this? Just an experienced security researcher like many others.

Rare ASP.NET request validation bypass using request encoding

I had blogged about this in NCC Group’s website. I thought it is the best to add a link to it here as well.

It is possible to bypass the ASP.NET request validation capability when errors are ignored using request encoding techniques. This can be abused to perform stored cross-site scripting (XSS) attacks.

The full article can be read here: https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/september/rare-aspnet-request-validation-bypass-using-request-encoding/

PDF version can be downloaded from: https://soroush.secproject.com/downloadable/Rare_ASP.NET_Request_Validation_Bypass_Using_Request_Encoding.pdf

Additional notes on “A Forgotten HTTP Invisibility Cloak” talk!

As I could not share the video for my BSides Manchester 2017 talk due to naming a few vendors, I thought the best plan was to explain a few things in a casual blog post. Most of the techniques explained in the presentation were not new although I did not know that initially! In some cases, I was 15 years late to the bypass partei; therefore, I have included links in the references for those who want to read more about them. I should mention that the request encoding technique is particularly the one I am proud of, as I still have not seen it anywhere else.

I think all we need now is the presentation file which has been shared already but here is the PDF version of the original PowerPoint file (to see the slides without SlideShare changes!): https://soroush.secproject.com/downloadable/A_Forgotten_HTTP_Invisibility_Cloak_v1.1.pdf

The http.ninja website is currently linked to a Github repository that contains the Python code I had used to perform the initial tests on my targets. My plan is to update this website with a cheat-sheet of different setups and different test cases. If you have an open source template in mind that can help me with this, please let me know via my Twitter.

So let us begin with the notes about some of these slides (the rests are hopefully self-explanatory and/or mentioned in the references):

Slide #13 (Strange Mutations):

These are not the only methods; some other documented behaviours can be seen in https://drive.google.com/file/d/0B5Tqp73kQStQU1diV1Y0dzd1QU0/view. That said, there are probably loads of other test cases that I have not tried yet and will be a great area for research.

Some of the methods explained in this slide could help us bypass protections that were based on the application path or the parameters.

HTTP v0.9 with full header supports on Apache Tomcat is a lovely feature/bug that can confuse loads of WAFs.

Slide #16 (HTTP Verb Replacement):

This method was only useful for those of us who were testing IIS. It was not really necessary to do so in order to bypass the WAFs as shown on the next slides.

Slide #23 (Removing Unnecessary Parts):

The last boundary “- -1- -” could be fully removed if it was a PHP application on Apache or IIS or on Nginx-uWSGI-Django-Python2/3 setup.

Slide #28 (Adding Useless Parts!):

There is just some space characters after the “filename” and before the “=” sign. This is to make it an invalid file upload request and therefore it would remain a normal POST request parameter as before. This mutation on its own can bypass some famous cloud-based WAFs.

Slide #29 – #31 (Changing Request Encoding!):

If charset was used before the boundary, a comma character should have been used as the delimiter between them. Otherwise, a semicolon character should have separated them.

Slideshare has changed the encoded values incorrectly. To see the actual values, you can view them in the PDF file of the presentation.

In order to find more information about this technique, please see the following: https://soroush.secproject.com/downloadable/request-encoding-to-bypass-web-application-firewalls.pdf

Slide #42 (HTTP Pipelining Example):

If you are using Burp Suite or other web proxies, ensure that the auto update Content-Length feature is disabled.

I, however recommend that to use direct socketing to see the full response as sometimes web proxies do not show it, although you can see it using Wireshark. I have created a simple Python code to do so that you can get it from https://github.com/irsdl/httpninja/blob/master/Generic%20Codes/web_request_socket.py

Fun fact: HTTP Pipelining can be used to exploit DoS issues, smuggle HTTP requests, or to exploit race condition flaws. However, that is not all it can do! Although I am not giving anybody any ideas, HTTP Pipelining can help you to book your precious online ticket faster than others. This is when you need to send more than one request to book your ticket but the following requests do not use a parameter taken from the previous responses.

References:

I had used the following references when working on my original research (I have added them to the slides as well now):

http://www.cgisecurity.com/lib/HTTP-Request-Smuggling.pdf

http://www.ussrback.com/docs/papers/IDS/whiskerids.html

https://media.defcon.org/DEF%20CON%2024/DEF%20CON%2024%20presentations/DEFCON-24-Regilero-Hiding-Wookiees-In-Http.pdf

https://securityvulns.ru/advisories/content.asp

https://dl.packetstormsecurity.net/papers/general/whitepaper_httpresponse.pdf

https://cdivilly.wordpress.com/2011/04/22/java-servlets-uri-parameters/

https://msdn.microsoft.com/en-us/library/system.text.encodinginfo.getencoding.aspx

I also found this afterwards so while not using it as my reference, it is still related and is a good read:

https://blog.qualys.com/wp-content/uploads/2012/07/Protocol-Level%20Evasion%20of%20Web%20Application%20Firewalls%20v1.1%20(18%20July%202012).pdf

Final Notes:

This talk was a reserved talk and was not planned as a main one; I had been assured that the probability of this happening was about %0.00…01; but hey, even that can win you the lottery sometimes! As one of the first presenters at 9am did not show up on the day, I had to deliver my talk with about 10 minutes notice so I should say thanks to the audience who did not leave the room despite having a different show and had to listen to my random presentation – you are awesome! :)

And here is an interesting Tweet about this:

And yes, BeerSides Manchester was awesome too with some consequences:

Request encoding to bypass web application firewalls

I “think” I have discovered a “new” technique in bypassing external web application firewalls using request encoding. The idea is very simple but I had not seen this before to be used to bypass any protection mechanisms. Details of this technique has been published via NCC Group’s blog:

https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/august/request-encoding-to-bypass-web-application-firewalls/

This technique was presented as one the methods to mutate the HTTP requests in:

There are loads of other anomalies that can be used to bypass WAFs using webservers behaviour in accepting HTTP requests; my plan is to complete this research and put all the results via the https://http.ninja/ website. Please feel free to contact me via my Twitter (@irsdl) if you have some ideas regarding this.

The unofficial PDF version of this blog post can be downloaded from here:
https://soroush.secproject.com/downloadable/request-encoding-to-bypass-web-application-firewalls.pdf

Almost all the cloud-based WAFs that I had tested could be bypassed using this technique at the time of discovery.

When a web application SSRF causes the cloud to rain credentials & more

The following blog post was written by me and Daniele Costa:

https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/august/when-a-web-application-ssrf-causes-the-cloud-to-rain-credentials-and-more/

In this blog post we have demonstrated an SSRF exploitation to steal AWS credentials to access Amazon S3. What made this attack special was the fact that http://169.254.169.254/latest/meta-data/iam/ was not accessible to our users during the exploitation. Therefore, we had to use the ‘userData’ attribute in EC2 describe-instance-attribute operation to extract the sensitive data.

The unofficial PDF version of this blog post can be downloaded from here:

https://soroush.secproject.com/downloadable/when-a-web-application-ssrf-causes-the-cloud-to-rain-credentials-and-more.pdf