Lucent Sky Works with New Relic to Put Off-Line Security Into Run-Time Analytics

Guest author Jim Liu is the co-founder of Lucent Sky, a company that automates how developers find and fix security problems in applications.

When designing an application, there’s often a trade-off between performance and security. For example, using a larger key size for encryption will make it harder to decipher, but it will also take more resources to encrypt and decrypt. When it comes to writing secure source code, however, this performance/security trade-off may not apply.

Application vulnerability mitigation (AVM) solutions, such as Lucent Sky AVM, are designed to help secure applications by automatically identifying vulnerable pieces of source code and replacing them with secured ones. Imagine something like spell check, which both finds and offers a fix for mistakes (in our case, things like cross-site scripting and SQL injection). Because Lucent Sky AVM (like traditional Static Application Security Testing, or SAST, tools) works offline, it’s not immediately apparent what impact these vulnerability fixes have on source code come run-time. Developers and IT professionals will often ask if the security benefits of using AVM will reduce the application’s performance. That’s where Lucent Sky’s integration with New Relic comes in.

The integration between New Relic APM and Lucent Sky AVM is fairly simple: Users set up their New Relic account information in the AVM Server, and the AVM Server links the applications in AVM with those in New Relic.

By using New Relic APM with Lucent Sky AVM, users not only gain performance insights on their applications, but can also monitor how vulnerability mitigation affects application performance.

See for yourself

To demonstrate how linking Lucent Sky AVM and New Relic APM can help customers better see the link between AVM and application performance, we tried the integration on a 3-tier ASP.NET application with around 220,000 lines of code. Multiple instances of Apache Bench were used to simulate 50 concurrent requests. We used New Relic APM to monitor the time it took for the application to process each request, while Apache Bench itself monitored the performance from the client’s perspective.

Here’s a snapshot of the original code (pre-AVM):

AVM original code

And after AVM (secured code):

AVM after code.jpg

Inside the .NET CLR, it took the original application 1.13ms to handle each request, while the secured application used 1.18ms. That’s about 4.4% slower.

Now let’s take a look from a website visitor’s perspective. Here’s a scatter chart of the response time recorded by the test client:


With the original application (grey dots), the average response time is 17.65ms, with 50% of the requests having a response time of 15ms or less and 99% of requests being completed within 31ms. The secured application (blue dots) has a slightly higher average response time of 18.75ms, with the same response time distribution: 50% of the requests have a response time of 15ms, and 99% of the requests were completed within 31ms. Most website users won’t notice any performance difference (unless they’re the lucky 1%).

To put the numbers in perspective, we installed open source @eb application firewall ModSecurity on the server running the original code, enabling the baseline rules from OWASP’s Core Rule Set.


With ModSecurity installed and baseline rules enabled (orange dots), the Web server spent an average of 36.98ms to complete each request, with 50% of the requests having a response time of less than 31ms and 99% of the requests being completed within 47ms. Although comparing AVM secured code with ModSecurity isn’t really apples to apples, it does show how little performance impact AVM secured code has compared to host-based Web application firewalls.

The goal of Lucent Sky AVM is to provide functional actions and insight to developers, resulting in more efficient and secure code releases. By integrating with New Relic APM we can see the specific performance results of implementing automatic fixes to secure source code, and can benchmark AVM secured code against other solutions in run-time.

We’ll continue to test our own and client applications with New Relic, and look forward to seeing how better tools and processes offline can continue to improve run-time results.
For more on using Lucent Sky and New Relic together, visit

This post was written by a guest contributor. Please see their biographical details at the top of the post above. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!