Experimenting with the New Relic Agent SDK, Lua, and Nginx

In the months following our initial release of the Agent SDK, we focused on adding new features, the latest being SSL support. We decided it was finally a good time to take a quick break to experiment a little with the Agent SDK and provide another example of how to get New Relic for apps using frameworks that are not yet supported by our other agents. Since our last example was a big success with helping people get New Relic for their HHVM environments (for example, see Christian Stocker’s HHVM solution), we wanted to reach out to a new group of developers, which also lets us demonstrate how flexible the Agent SDK really is. So, last week, I got started on an Nginx module and Lua example.

nginxOur C API makes it fairly easy to bind to any language, and tools like SWIG make it even easier. One of the biggest time-consumers in building an agent is figuring out where to hook into all the many frameworks. Without any experience using Lua or Nginx, I knew that I might not finish in the short amount of time I had before getting back to implementing the next big Agent SDK feature. I didn’t finish, but by the end of the week, I had an Nginx module that could report request times up to New Relic and a way to instrument Lua code to get more features such as Transaction Traces and Traced Errors. My hope is that people will find what I worked on a good starting point to create their own custom solutions.

Making Use of LuaJit FFI

I started by using SWIG to create a Lua wrapper, but eventually turned to LuaJIT’s FFI library, which allows code to be JIT compiled. It makes calling our API functions faster inside your Lua code and is really easy to use. Since it parses plain C declarations, I could just copy-paste code straight from my C headers. I also wrote Lua wrappers to the C functions. This wasn’t strictly necessary, but it helped clean things up a bit. Then I used Lua’s require function to load and run our library from inside the Lua script that I wanted to instrument.

Working Inside Nginx Config

The next step was to find a better place to run the APM data collector. Nginx uses one master process that manages several worker processes that do the actual request processing. For this example, I decided to start the data collector inside each of these worker processes, and, conveniently, the Nginx HttpLuaModule provides a directive called init_worker_by_lua to run code on worker process startup.

Now that the data collector was running inside the worker process, I was able to capture that start and end of requests by using the HTTPLuaModule directives access_by_lua and log_by_lua to hook into the start and end of a request. The access phase is probably not the best phase to hook into for the start of a request, but I thought this would be sufficient for a proof of concept. Here’s what the code looks like inside the Nginx configuration file:


http {  
    include /usr/local/openresty/nginx/conf/mime.types;

    init_worker_by_lua '
        newrelic = require("newrelic.init")
        newrelic.embed_collector();
        newrelic.init(, "LuaApp", "Lua", "5.1.5")
    ';

    server {
        listen 7747;
        server_name localhost;

        location /test {
            set $transaction_id 1;
 
            access_by_lua '
                ngx.var.transaction_id = newrelic.begin_transaction()
            ';

            content_by_lua_file 'foo.lua';

            log_by_lua '
                newrelic.end_transaction(ngx.var.transaction_id)
            ';
        }
    }
}

Inside foo.lua I was able to access the transaction ID needed to make Agent SDK calls. For instance, here I set the name of the transaction by passing transaction_id and a name string to set_transaction_name.


local transaction_id = ngx.var.transaction_id;
newrelic.set_transaction_name(transaction_id, "foo")

Already I could see request times show up on New Relic after going to localhost:7747/test and pressing refresh a few times. I could also see that transaction traces were triggered when request times were long enough, but if I wanted to see trace data and other useful New Relic features, I would need to add more instrumentation. All of this could be done from within the Lua script.

Making Progress on Building an Nginx Module

I stopped my Lua example at this point and started down the path of building an Nginx module that would automatically record requests. I was able to see request times show up in New Relic using the module, but, due to time constraints, could not finish the work it would take to connect the module to Lua scripts. Overall, I did what I set out to do, which was to provide more starting points for the software development community interested in the Agent SDK and to show a proof of concept (and had lots of fun doing it).

You can see my code samples in our public samples repo. Feel free to make a pull request. Also, your feedback and questions are welcome!

Resources:

Abner Germanow was formerly Senior Director of Partner Marketing & Evangelism at New Relic. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!