Barracuda Application Server C/C++ Reference
 All Classes Functions Variables Typedefs Enumerations Enumerator Groups Pages
Advanced Lua Bindings

Our online Lua Binding Generator is a good starting point when it comes to building your own Lua bindings, thus enabling LSP pages and Lua scripts to call your own C functions.

The following documentation is for writing manually crafted Lua bindings. Manually crafted bindings are preferred for more complex interactions such as interactions with objects on the C side.

The tutorials in this paper explain:

  • How to design a Lua interface to a C object, such as a handle in C code or a C++ class.
  • How to release the global mutex for code that is time consuming, thus enabling concurrent requests.
  • How to call Lua code asynchronously from C code.

Object oriented access and releasing the global mutex

The following tutorial covers how to write a Lua interface to a C object. You can use the same method for exporting a C++ class to Lua. The tutorial also shows how to release the global mutex for C functions that may take some time to complete, thus enabling the web server to accept other HTTP requests while the C function is running.

The tutorial requires that you have an understanding of Lua user data and Lua metatables as described in the Lua book. An older version of the book is available online where you can read the two sections Object-Oriented Access and metatables.

The Lua interface to the C object will be as follows:

Dynamically create a Lua object that represents the C object.

local l2c = Lua2C.open()

The C object is created by calling the open function in the global Lua2C table. The Lua2C table and the open function are all part of this exercise. In other words, we will extend the Lua engine with this functionality.

The dynamically created object will have two functions that can be called.

l2c:fast()
l2c:slow()

The "fast" and "slow" methods increment a counter when called and return the number of times the function has been called. The "fast" function returns immediately and the "slow" function takes 5 seconds to execute. The slow function is therefore designed such that it releases the global mutex while processing, enabling other threads in the server to execute.

Lua can return multiple values so both the "fast" and "slow" functions return its own counter and the counter for the other function.

Creating the Lua2C table and binding the 'open' function to the Lua2C table.

We must start by creating a C function that registers our Lua2C table.

int luaopen_Lua2C(lua_State* L)
{
luaL_requiref(L, "Lua2C", installLua2C, TRUE);
return 1;
}

The luaopen_Lua2C is the only global function in the C file, and it is the function you must call from your startup code – i.e. from the code creating the main Lua state. The name of the function can be any name, but we have opted to use a naming convention that is compatible with the Lua function require. Our C file can be compiled as a standalone module, on operating systems that support DLLs/shared libraries, and loaded on demand by calling require("Lua2C").

Notice how we set the last argument in function luaL_requiref to TRUE. This construction registers the Lua2C table as a global variable and makes it possible to reference this table directly in the global namespace, without having to call require("Lua2C") from the Lua code.

In the typical embedded system, function luaopen_Lua2C will be called just after you created the main Lua state, and our C file with the Lua2C bindings will be compiled and linked with the system. In other words, it will not be loaded dynamically as a dynamic library.

L = balua_create(&blp); /* create the Lua state */
lua_pop(L,luaopen_Lua2C(L)); /* pop and balance stack when installing Lua2C */

Function luaopen_Lua2C returns a value designed for dynamic loading via require and the return value is not needed when we link the code directly with the server code. What we do after creating the main Lua state "L" is to call luaopen_Lua2C and pop the values pushed by luaopen_Lua2C off the stack. This keeps the Lua stack balanced.

In function luaopen_Lua2C, luaL_requiref takes function installLua2C as an argument. The installLua2C function installs the "open" function in a new Lua table and returns this table. Lua will then register this table as "Lua2C".

static int installLua2C(lua_State *L)
{
static const luaL_Reg mdfTable[] = {
{"open", Lua2C_open},
{NULL, NULL}
};
luaL_newlib(L, mdfTable);
return 1;
} /* End */

Lua code can now call Lua2C.open() and a call to this function from Lua ends up in the Lua binding Lua2C_open.

static int Lua2C_open(lua_State *L)
{
Lua2C* l2c = (Lua2C*)lua_newuserdata(L, sizeof(Lua2C));
l2c->slowCnt = 0;
l2c->fastCnt = 0;
/* Get metatable for Lua2C */
if(luaL_newmetatable(L, LUA2C))
{ /* Failed getting meta i.e. first time this func is called. */
static const luaL_Reg Lua2CLib[] = {
{"fast", Lua2C_fast},
{"slow", Lua2C_slow},
{"__gc", Lua2C_gc},
{NULL, NULL}
};
/* Create metatable */
lua_pushvalue(L, -1); /* Push table (t) created by luaL_newmetatable */
lua_setfield(L, -2, "__index"); /* t.__index == t */
/* Set t.fast=Lua2C_fast, t.slow=Lua2C_slow, and t.__gc=Lua2C_gc */
luaL_setfuncs(L, Lua2CLib,0);
}
lua_setmetatable(L, -2); /* Set meta for Lua2C userdata */
return 1; /* Lua2C userdata */
} /* End */

The key to understanding the above code is to read the Lua documentation for luaL_newmetatable and read up on how metatable.__index = metatable works. This function creates a new table the first time it is called and retrieves the table on subsequent calls. The table is set as a metatable for the Lua userdata, i.e. for the C object Lua2C. The metatable binds the 3 Lua functions, "slow", "fast", and "__gc" to the dynamically created Lua2C object, thus making it possible for Lua code to use the userdata and call function "fast" and "slow". The "__gc" function is run by Lua when the garbage collector reclaims the memory for the userdata object. The "__gc" function is the destructor.

Lua code can now call function "fast" and "slow". The following will print out "1 0" and "2 0".

local l2c = Lua2C.open()
print(l2c.fast(l2c))
print(l2c:fast()) -- Syntactic sugar for the above code 'l2c.fast(l2c)'

The above Lua example code triggers the C function Lua2C_fast twice.

static int Lua2C_fast(lua_State *L)
{
Lua2C* l2c = (Lua2C*)luaL_checkudata(L,1,LUA2C);
l2c->fastCnt++;
lua_pushinteger(L, l2c->fastCnt);
lua_pushinteger(L, l2c->slowCnt);
return 2;
}

Function Lua2C_fast verifies that the first stack argument is a Lua2C object, typecasts the Lua userdata object to a Lua2C object, increments the counter, and pushes the two return values onto the Lua stack.

Releasing the Global Mutex

Function Lua2C_slow is similar to function Lua2C_fast and performs the same checking, increments the counter, and pushes the two values onto the stack. The "slow" function also sleeps for 5 seconds, thus simulating a lengthy C call.

static int Lua2C_slow(lua_State *L)
{
Lua2C* l2c = (Lua2C*)luaL_checkudata(L,1,LUA2C);
Thread_sleep(5000); /* 5 secs */
l2c->slowCnt++;
lua_pushinteger(L, l2c->slowCnt);
lua_pushinteger(L, l2c->fastCnt);
return 2;
}

The above code fetches the server's main mutex and releases the mutex just before sleeping for 5 seconds. The mutex is reclaimed just after it sleeps. You must make sure you have the mutex locked when working with the Lua state L. The mutex will always be locked when the Lua binding starts, and the mutex must be locked when it returns.

When an LSP page calls the "slow" function, it will block for 5 seconds. This enables other threads in the server's pool to execute. An introduction to the thread mechanism in the server can be found in the introduction under section Thread Mapping and Coroutines. Many of the Barracuda Application Server's Lua bindings release the mutex when calling code that can block. Examples of such functions are the functions in the SQL library and functions in the response object such as response:write.

Each native thread in the server's thread pool is mapped to one Lua coroutine stack; thus the server can handle many concurrent Lua stacks at the same time. We will explain more on how to map a coroutine stack onto a native thread in the next section, where we explain how to call Lua from C code.

Download Example Code

Download the source code for the above example.

Automatic Mutex Wrapper Generation

It is tedious to create code that releases the mutex during lengthy function calls if a large number of functions must be modified. You may also have existing Lua bindings, or Lua bindings from the Internet that you want to download and quickly modify. Real Time Logic has prepared two scripts that you may use to automate this process.

mutex.lua: Mutex wrapper script generator

This script creates C code that wraps around the original functions. Each wrapper function releases the mutex, calls the original function, and locks the mutex when the original function returns.

Let's say you have the following function and you want to wrap this function into another function that releases the mutex when it is called.

int myfunction(int arg1, const char* arg2);

Create a file such as functions.txt and put all the function declarations you want to wrap into this file. In our case, we put the above function declaration into this file. The next step is to create the wrapper code. Run the mutex script as follows:

lua mutex.lua functions.txt my-generated-wrapper my-header.h

The Lua script mutex.lua parses the function declarations in functions.txt and produces my-generated-wrapper.h and my-generated-wrapper.c. The last argument passed into mutex.lua (my-header.h) is to a header file that provides the original function declarations. This file is included in the output (my-generated-wrapper.h).

my-generated-wrapper.h:

#include "my-header.h"
int myfunction_L(lua_State* L, int arg1, const char* arg2);
#define myfunction(arg1, arg2) myfunction_L(L, arg1, arg2)

my-generated-wrapper.c:

#undef myfunction
int myfunction_L(lua_State* L, int arg1, const char* arg2)
{
int ret;
ThreadMutex* __tm = getDispMutex(L);
ThreadMutex_release(__tm);
ret = myfunction(arg1, arg2);
ThreadMutex_set(__tm);
return ret;
}

fext.lua: Function declaration extraction script

The file that lists all functions (functions.txt) can be manually created or you can use fext.lua to parse all your header files and automatically produce functions.txt.

$lua fext.lua
Usage: lua fext.lua [-p pattern] [-c cdata-blob-file] [-e exclude-file] input-files....

The optional pattern (-p) means that you should only include files that matches this pattern. See Lua patterns for details.

You do not want to create wrappers for functions not being used by your Lua bindings. The optional -c option is for loading a "blob file" that is a concatenation of all C files that are part of your Lua binding library. When a function is found in the header file(s), the "blob file" is searched. The function is only included in files.txt if it is found in the "blob file". You can, for example, create this file as follows: cat *.c > c-data-blob.txt

You may further want to limit the set of functions included. The -e option is for a file that lists all functions that should be excluded.

Download mutex.lua and fext.lua

For additional examples on how to use the above two Lua scripts, see the build scripts (build.sh) for the MongoDB and PostgreSQL Modules.

Calling Lua Code Asynchronously From C Code

The Barracuda Application Server enables multiple threads to call and execute Lua code. In the previous section, we explained how threads in the thread pool can simultaneously execute in the Lua VM instance by using coroutine thread stacks. Only one thread can execute at any time in the VM, but Lua scripts can call Lua bindings (C functions), which may then release the global mutex protecting the Lua VM. This construction enables multiple threads to co-operatively run the Lua VM instance.

The following code snippet is from the Web File Server C code wfs.c, and the function shows how to call an unload handler (Lua function) from C code just before the server terminates. The optional unload handler can, for example, be used to cleanup temporary resources just before the server exits.

/* Run the Lua 'onunload' handlers for all loaded apps when wfs exits.
Function onunload is optional. Lua applications (.preload scripts)
loaded by the wfs can optionally declare an onunload function if
the application requires graceful termination of resources such as
sending a socket close message to peers.
*/
static void onunload(void)
{
/* Run the 'onunload' function in the .config Lua script, which in
* turn runs the optional onunload for all loaded apps.
*/
lua_getglobal(L,"onunload");
if(lua_isfunction(L, -1))
{
if(lua_pcall(L, 0, 1, 0))
{
printf("Error in 'onunload': %s\n",
lua_isstring(L,-1) ? lua_tostring(L, -1) : "?");
}
}
} /* End */

We push the global variable onunload onto the stack and check if the variable pushed onto the stack is a function. We then call the function by using lua_pcall, which runs the Lua function in protected mode and catches any exceptions thrown by the Lua script.

The above code uses the main Lua state created by function balua_create and not a Lua coroutine state. This is okay since we are not running multiple threads just before exiting the Web File Server executable. However, this is not a solution you should use when calling Lua asynchronously from C code in a running system. You should not use the main stack since it could corrupt the system state. Instead, create a Lua coroutine thread for each native thread calling into Lua. You should have a one-to-one mapping between native threads and Lua coroutine threads.

A good place to create the coroutine thread(s) is just after you create the main state. The following code snippet is from the Web File Server code (wfs.c) and shows how to create the main Lua state 'L'.

/* Create a LSP virtual machine.
*/
blp.vmio = (IoIntf*)&vmIo; /* The required IO */
blp.server = &server; /* pointer to a HttpServer */
blp.timer = &timer; /* Pointer to a BaTimer */
L = balua_create(&blp); /* create the Lua state */

Just after the above code, add code similar to the following:

lua_State* Lt = lua_newthread(L);
int threadRef = luaL_ref(L,LUA_REGISTRYINDEX);

The above code creates a new Lua thread Lt and it is the Lt state you will use in your native thread when calling Lua code from your C code. We must keep the main stack balanced at zero so we must pop the new thread off the main stack L. We cannot use lua_pop since this would make the garbage collector terminate the thread and make the Lt pointer invalid, thus causing your program to crash. What we do instead is to call luaL_ref, which pops it off the main stack, but keeps a reference so it is not garbage collected.

The following code snippet from wfs.c shows how the mutex variable of type ThreadMutex is initialized:

/* Initialize the Socket Dispatcher (SoDisp) mutex variable. */
ThreadMutex_constructor(&mutex);
/* Initialize the Socket Dispatcher (SoDisp) */
SoDisp_constructor(&disp, &mutex);

The native thread calling asynchronously into Lua code must keep a reference to the thread state Lt and to the main mutex (SoDisp mutex). The following code shows how you can design a function that uses the Lua thread state Lt and the main mutex when calling any global function.

const char*
callLuaFuncFromThread(lua_State* Lt, ThreadMutex* m, const char* func)
{
const char* emsg=0;
ThreadMutex_set(&mutex); /* Lock (may block) */
lua_settop(Lt, 0); /* unwind any previous error messages */
lua_getglobal(Lt,func);
if(lua_isfunction(Lt, -1))
{
if(lua_pcall(Lt, 0, 1, 0))
emsg = lua_isstring(L,-1) ? lua_tostring(L, -1) : "?";
}
else
emsg = "function not found";
ThreadMutex_release(&mutex); /* Release */
return emsg;
}

Any error message is left on top of the Lua stack and it is safe to use this const pointer when the mutex is released. Any error is automatically erased the second time callLuaFuncFromThread is called by setting the stack top to 0.

The above example does not show how to pass arguments to the Lua function or check for any returned argument(s), except for 'error'. See the function lua_pcall for more information on passing in arguments to the Lua function and checking for return values.