Hello All, As a party who had numerous occasions to deal with Oracle in the past, I'd like to write a few words of comment to the company's CSO's blog post [1]. These are grouped under separate sections below. ["we find 87% of security vulnerabilities ourselves"] Oracle CSO's stated that the company finds 87% of security vulnerabilities itself, security researchers find about 3% and the rest are found by customers. We did some brief analysis with respect to the security issues reported to Oracle over the recent years and came up with different numbers than the one given by the company's CSO. The 20 CVE-IDs Oracle assigned to 43 (Oracle tends to cumulate multiple different security issues under one CVE identifier [2]) Java SE weaknesses we reported to the company in 2012 and 2013 were addressed by 7 different CPUs / Alerts [3]. These CPUs / Alerts announced fixes for 307 CVE-IDs. That gives 6.5% "contribution rate". However one still needs to keep in mind that each Oracle CPU credits numerous 3rd parties for reporting security vulnerabilities to the company. This means that the percentage of bugs reported to Oracle by security researchers could be much more higher (the number of credited parties varies from 4-35). For that reason we reviewed 23 Oracle CPUs and 12 Java SE CPUs released by the company since 2010. We made a conservative assumption that each 3rd party (a customer or a security researcher) credited in a given CPU is responsible for a discovery of one issue only. We also optimistically assumed that there hasn't been any collisions between discoverers regarding their findings. We obtained the following results: - for Oracle CPUs released from Jan 2010 to Jul 2015, 2210 security bugs were fixed and 485 parties credited, which yields 21.95% contribution rate, - for Java SE CPUs released from Mar 2010 to Jun 2013, 309 security bugs were fixed and 118 parties credited, which gives 38,19% contribution rate. Assuming that credits are given to both security researchers and customers, the 13% number given by Oracle CSO does not seem to be reflected by the CPU data. This is where things started to get intriguing to us. The possible cases to the Oracle CSO vulnerability math could be as following: 1) the numbers are real, Oracle CPUs issued so far reflect both security vulnerabilities found internally and those reported by 3rd parties (customers + security researchers), 2) the numbers are real, but Oracle CPUs reflect only 13% of the issues reported by 3rd parties, the remaining 87% of internally discovered issues have been silently fixed or has not been fixed yet, 3) the numbers provided by Oracle CSO are bogus and have no foundation in a real life data. For case 1, the numbers could be real if 3rd party contribution was smaller by approx. 45% (primarily due to collisions for credited issues / as a result of multiple names beng used in a single credit). This would bring down the overall contribution of 3rd parties to 13%. We don't think this is a viable option (independent bug discoveries do happen, but not at such a rate and frequency). On the other hand, there are several CPUs that seem to support case 1. Jan 2015 CPU contained 169 fixes and credited 24 3rd parties for them (14.2% contribution rate). It would be surprising to have these 24 parties to be each responsible for a discovery of 7 separate bugs. Case 2 looks least probable to us as it would mean that internal discoveries of Oracle are in the range of 17000 in the last 5.5 years only (6.6 times more than 'security weenies' [4]). This would also mean that Oracle is indeed a true leader when it comes to nixing security vulnerabilities in software and that it does not have a match in the whole industry (17000 vulnerabilities is twice as much as all vulnerability IDs in CVE database [5] corresponding to Microsoft, IBM, HP and Apple issues combined together). This leaves us with option 3. One possible way to verify all of that is to have a more clear information in Oracle Critical Patch Updates and Alerts. Three years ago (May 28, 2012) we asked Oracle whether "it would be possible for the company to change the form of the credit statement used in its security advisories, so that it is much more clear for the general public which party is actually responsible for reporting a given bug". On Jun 01, 2012 Oracle informed us that the company "is considering our suggestion regarding the format of the credit statement" and that it "will look at adding this to its document generating utilities but it will take some time". Three years has passed and Oracle CPUs has not changed a bit with respect to the clarity of the vulnerabilities origin (which party is responsible for a given issue, whether internal issues are taken into account, etc.). It's a perfect time for Oracle to finally do it as this would in particular: - prove that numbers provided by Oracle CSO are not bogus, - show actual contribution of 3rd parties / security research community, - provide more information about the effectiveness of Oracle's own bug hunting efforts. In the past, Oracle played with both vulnerability counting and their impact [2][6]. Oracle's words cannot be taken for granted. Thus, we call the company with respect to the numbers provided by its CSO and expect that it will show a clear evidence to the public regarding their credibility. In any other case, we believe that it is reasonable to assume that: - the numbers provided by Oracle CSO are bogus and have no foundation in a real life data, - their use was solely for the purpose of both downplaying the contribution of 3rd parties and to strengthen the message passed by the CSO. ["the usual security hygiene"] In her message, Oracle CSO also pointed out the lack of "the usual security hygiene" to its customers. It's probably a good moment to remind the top-notch security hygiene at Oracle and the following in particular: - the use of 2 years old Java SE software in a production cloud environment [7] (Oracle US1 and EMEA1 Commercial data centers), - the use of the same administrative password for all customers' Weblogic server instances and a cloud management system supervising them (EMEA1), storage of that password in a customer environment (US1 and EMEA1), - storage of various credentials / passwords in plaintext form including the administrative ones (EMEA1). - the use of "strong" administrative passwords (data from Sep 2013): - "tasWelcome1" - Weblogic server admin (OCLOUD9_WLS_APPID) pass (US1), - "fusionapps1" - Oracle LDAP admin (cn=orcladmin) pass (US1). - sending out vulnerability status reports to reporting parties in plaintext and risking the leak of a sensitive vulnerability information prior to the availability of the fixes: - Apr 25, 2012 (status report for SE-2012-01 project) * Oracle was notified of a reception of the unencrypted status report - Jun 21, 2012 (status report for SE-2012-01 project) * Oracle was notified of a reception of the unencrypted status report - Feb 12, 2014 (status report for SE-2013-01 project) * Oracle was notified of a reception of the unencrypted status report - Aug 22, 2014 (status report for SE-2014-01 project) - Sep 24, 2014 (status report for SE-2014-01 project) ["everybody will get the fix at the same time"] Oracle CSO also stated "if there is an actual security vulnerability, we will fix it...to protect all our customers, meaning everybody will get the fix at the same time". Unfortunately, this is not true. Oracle does not always release fixes for security vulnerabilities at the same time and the company officially admitted this to us. According to the status report received from Oracle on Oct 24, 2014, Oracle CPU from Oct 14, 2014 CPU addressed all 22 security vulnerabilities reported in Oracle Database Java VM component [8]. Not all fixes for them have been available to the public on the CPU date though. Oracle Support Document 1912224.1 indicated that patches for many platforms were made available 1-3 weeks later. As a response to our inquiry regarding the reasons of issuing incomplete CPU fixes, Oracle claimed that "it occasionally allowed the patches to be released the end of the month when the CPU was issued [9]. As a result some of these patches have been delayed". Thank you. Best Regards, Adam Gowdiak --------------------------------------------- Security Explorations http://www.security-explorations.com "We bring security research to the new level" --------------------------------------------- References: [1] No, You Really Can’t, Mary Ann Davidson Blog https://web.archive.org/web/20150811090106/https://blogs.oracle.com/maryanndavidson/entry/no_you_really_can_t [2] SE-2012-01-CVE_Map http://www.security-explorations.com/materials/SE-2012-01-CVE_Map.pdf [3] Java SE CPUs from Jun 2012, Oct 2012, Feb 2013, Apr 2013, Jun 2013, Oracle CPU from Oct 2013 and Oracle Security Alert for CVE-2012-4681 http://www.oracle.com/technetwork/topics/security/alerts-086861.html [4] Is Your Shellshocked Poodle Freaked Over Heartbleed?, Mary Ann Davidson Blog https://blogs.oracle.com/maryanndavidson/entry/is_your_shellshocked_poodle_freaked [5] Common Vulnerabilities and Exposures https://cve.mitre.org/ [6] [SE-2012-01] Details of issues fixed by Feb 2013 Java SE CPU http://seclists.org/fulldisclosure/2013/Feb/12 [7] SE-2013-01 Security vulnerabilities in Oracle Java Cloud Service http://www.security-explorations.com/en/SE-2013-01.html [8] SE-2014-01 Security vulnerabilities in Oracle Database Java VM http://www.security-explorations.com/en/SE-2014-01.html [9] SE-2014-01 Vendors status http://www.security-explorations.com/en/SE-2014-01-status.html