Open Web Application Security Project: Core Rule Set: The evolution of an OWASP Project

Let me put one thing straight: there are two things when we talk about ModSecurity. There is the naked ModSecurity engine running inside NGINX or Apache and there is the rule set that instructs the engine what to do. Many different rule sets exist. But the rule set with the largest user base (and longest name) is the OWASP ModSecurity Core Rule Set or CRS for short.

CRS started in 2006 and moved under the OWASP umbrella in 2009. While widely used, it was essentially run as a one man show until late 2015 when Chaim Sanders took over and asked Walter Hop and me to join the project. We formed a three person leadership team with Chaim having the final say. Our new team undertook an informal review of CRS and identified four areas where the CRS sucked:

usability
documentation
ugly code
non-existent community

Additionally, there were also - and still are - blind spots in the rule base. Despite this CRS has always been very good at detecting attacks - easily catching 80% - 90% of the attacks aimed at web applications. If we could solve the four pressing issues, we were sure the project would flourish, so we worked tirelessly throughout 2016 to release CRS v3.0 in November 2016.

False positives really kill the experience when running a web application firewall. The CRS3 release brought a painless out-of-the-box installation that has very few false positives due to our new Paranoia Level. This value lets you decide how aggressive you want the rule set to behave. The default Paranoia Level means you are satisfied with detecting 80% of the attacks - as long as you do not get any false positives. You can raise the level to reach 95% or even 99% of the attacks. But this comes at the expense of a substantial number of false positives and you need to tune away these alerts in order to run your service seamlessly again.

With CRS3, we heard users exclaim that the new CRS just works without much tuning by default. People do a default install on their existing sites and they hardly notice it's there until they try out sql injections on the login screen. As if to underline this, in the year since we released CRS3 we received roughly 20 github issues due to false positives. Having run the old Core Rule Set for many years, 20 false positives used to be what I would eat for breakfast in the old days.

Still, we knew there was room for improvement and so I wrote a series of tutorials in the form of several guides and accompanying scripts to help people streamline the tuning process.

As expected, the CRS3 release brought an increased interest in the project. In order to foster this interest and to build our community we timed the release with a new movie poster style graphic and I began to run ModSecurity / CRS courses in several cities. All these efforts paid off as in October we saw almost 1600 people visit the CRS integration tutorial. On top of that, almost 1000 people visited the tutorial covering false positives. That's a doubling within 12 months. My CRS classes have attracted the interest of internet service providers, appliance manufacturers, big banks, university services and even IT teachers. New developers started to show up with @victorhora leading the way. In early November, we promoted three members of our growing community to active developers and we are now the ten of us. Ten developers is an ass-kicking number and we were able to address the ugly rule base. Developer @fgsch championed a cleanup project in summer. He received immediate supported by @fzipi and @spartantri. This has greatly improved the readability of the rules and spurred a set of coding guidelines that will guide future development.

In late Summer 2017, @franbuehler announced that our legacy of dozens of machine optimized regular expressions were a mess. She proved that the project had lost the readable sources to these patterns a long time ago. So she disassembled the regexes of over 1000 characters width into a list of human readable source patterns. Her solutions were merged to much acclaim in early November. This had the further benefit of galvanizing our community. The monthly CRS community chat evolved into a monthly planning session. We use it to talk about the project and sort out controversial pull requests or issues. This has been very beneficial. It allows us to give positive feedback to new community members. And it lets everybody feel much more like being part of a real community where their input is valued and where they receive guidance with contributing of their own.

This new momentum of the project has not gone unnoticed. The 2017 edition of the OWASP Top Ten includes a reference to the CRS project under A10 Insufficient Monitoring and Logging. And when attending a Hacknight in Berne, Switzerland, one of the board members of the German Open Source Business Alliance approached me and asked to submit our project for the German Open Source Business Awards OSBAR. I gladly complied and on December 6, 2018, CRS 3.0 was invited to receive an OSBAR award together with a handful of other projects including the RUST language. I have never won an award for anything I have done ever, so this felt extremely cool!

With the code base cleanup nearing completion, we are now using our larger developer base to add better coverage and close existing gaps. A hot area in this regard are attacks on Java services. CRS would have detected the Equifax hack, but only at a high paranoia setting. Walter Hop (@lifeforms) and @emphazer are working on a new group of rules to expand the rule coverage for this attack class and include it into the upcoming CRS 3.1 release. Parallel to this, we are also adding unit tests to all our rules. This should protect us from ruining the existing rules with a stupid mistake and from adding false positives when we commit changes. In fact, there is a separate project, FTW, created to support our unit testing process. This is done by providing a Framework for Testing WAFs (hence FTW). It's a great tool and I hope other projects can profit from it's simplicity too. With hundreds of tests merged into OWASP CRS, we are seeing good progress there.

What's next for the project:

Release 3.1.0, possibly also 3.0.3, is on the list for the next few months.

We aim for a CRS summit at one of the European OWASP conferences in 2018. The idea is to meet with various parties using CRS in their setups or as part of their services or products.

We hope to achieve mainstream adoption of CRS with online offerings. Let's encrypt is included with your hosting plan and we think an OWASP ModSecurity Core Rule Set installation should also be added to the package.

Now that CRS is being featured in the OWASP Top Ten (2017), we need to open up to the people looking at OWASP Top Ten. They are looking for guidance on how they can take the best out of CRS with their existing setups. Here is a first blog post with this goal.

If you feel like supporting our project read through our blog post aimed for new project members and check out the CRS website.

Or why don't you join our monthly chat. We meet on the first Monday of a month on freenode IRC, channel ModSecurity at 20:30 CET, and we are very open to newcomers. We’re all newcomers, actually.