The Virtual File System (URL Routing)

This guide introduces the concept of the Virtual File System (VFS) in the Barracuda App/Web Server (BAS/BWS). It explains that the server requires the user to build a VFS to provide routing and to serve HTTP resources. Any attempt to access resources without a properly configured VFS will result in a 404 error. The VFS allows for the organization and management of various resources such as static files, Lua Server Pages (LSP), and directories, providing a flexible and powerful structure for serving content.

BAS VFS vs. Standard Routing

The Barracuda App Server's Virtual File System provides a more flexible way to manage routing than what is found in traditional web servers, where routing often involves mapping URL paths directly to filesystem directories or predefined controller actions. BAS, however, employs a VFS that offers a more dynamic and programmable approach to routing. This means you can define how URLs are handled at runtime, providing flexibility that surpasses conventional static routing methods.

Key Advantages of BAS's Routing via VFS:

1. Dynamic Route Handling: Instead of static mappings, you can programmatically define how different URL paths are processed, allowing for dynamic responses based on various conditions.
2. Modular Application Structure: The VFS allows you to organize your application into modules or components, each responsible for different parts of the URL space, promoting a clean and maintainable codebase.
3. Runtime Configuration: Routes and their handlers can be modified or extended at runtime without restarting the server, facilitating seamless updates and feature additions. For example, the Mako Server and Xedge provide an application concept, adding routing for each running app. These routes are managed via the application's directory instance and are installed and removed as the application state changes from running to stop.
4. Simplifies Authentication: An authenticator can be installed on a directory node, acting as a protective umbrella that enforces authentication for all resources within that directory, eliminating the need to secure each resource individually.

VFS Introduction

BAS and BWS differ from standard web servers because URLs are not automatically mapped to a file system or to predefined controller actions. Instead, the URL path component is initially processed by the VFS, which decodes it and attempts to find a matching node. If a node is found, the matched path portion is stripped, and the node processes the remaining path component. This allows for a dynamic and programmable VFS structure, constructed at runtime, as illustrated in the following diagram.

Figure 1: A virtual file system with three nodes

After parsing client HTTP requests, the web server delegates the request to the VFS. The VFS then locates and processes the requested resource. If it cannot find a matching node, it returns a "404 not found" error to the HTTP client.

The VFS resources extend beyond managing physical files to include items like Lua Server Pages (LSP), which contain Lua code executed upon request. The VFS supports various directory types, including Resource Reader (depicted as Dir 2 in Figure 1), WebDAV, and others, enhancing its flexibility.

Here's a Lua example to demonstrate setting up a basic web server:

-- Create a "resource reader" (resrdr) using the Lua disk I/O interface.
myapp = ba.create.resrdr(ba.openio("disk"))
myapp:insert() -- Insert as root application

This code initializes a Resource Reader and integrates it into the VFS, enabling browsers to fetch HTML pages from your file system. Detailed explanations of the resource reader will follow later in the guide.

VFS Features:

• Directory Node Response: Returns true if the resource is found and handled, false if not, prompting the VFS to continue searching.
• Root Directory: A node inserted at the root of the VFS, which can have a name or be set to nil to act directly on the root.
• Sibling Directory: A directory without a name, whether root or subdirectory, acts as a sibling rather than a subdirectory.
• Identical Node Names: The VFS can accept nodes with identical names, allowing searches across multiple root directories or subdirectories for the same path. Directory nodes have priorities, with higher-priority nodes being searched first.
• 404 Handler: A directory node can act as a 404 handler for a subdirectory or for all by accepting any requested resource. By installing the directory with a low priority, the VFS only calls this node if no other directory node manages the requested resource, thus making it work as a catch-all if not found.
• Filter or Custom Authenticator: A directory node can be designed as a special filter to analyze, authenticate, authorize, etc., by being installed as a directory with high priority. The filter can return false if it accepts the request, making the VFS continue to search for matching nodes. If the filter handles the request by, for example, returning a 401 Unauthorized to the HTTP client and returning true to the VFS, the VFS stops searching, thus protecting the other nodes.

One of the VFS's unique features is that nodes can be inserted and removed at runtime. Both the Mako Server and Xedge include easy-to-use APIs for dynamically starting and stopping web applications, which basically means inserting and removing VFS directory nodes at runtime. The Xedge introductory video shows how this works.

Constructing a Virtual File System Tree

A Barracuda App/Web Server with an empty Virtual File System (VFS) cannot do anything. Attempting to access the server using a browser will return a "404 resource not found" error message.

The Virtual File System must contain at least one resource. A basic server can be assembled as follows:

local diskio=ba.openio("disk")
myapp = ba.create.resrdr(diskio)
myapp:insert() -- No argument = insert as a root directory.

In the above example, a Resource Reader is created and inserted as a root directory. The resource reader is initialized with the Virtual Machine's I/O interface.

A browser entering http://localhost/index.html loads index.html from the base of the disk I/O interface.

I/O interfaces abstracts the underlying file-system architecture for the server. The Barracuda App/Web Server supports a number of I/O interface types such as DiskIo and ZipIo. A DiskIo maps to a location on a standard file system and a ZipIo maps to the internal structure of a ZIP file. The disk I/O is initialized by the C/C++ startup code.

Directory properties:

• The VFS can contain multiple root directories.
• The processing order of directories is determined by their priority. This priority comes into play when two nodes share the same name or when nodes are unnamed. By default, the priority is set to 0, with a minimum value of -127 and a maximum of 127.
• The VFS passes the request to the directory with the highest priority. If the resource is not found in the first directory, the VFS passes the request to the second directory. This step continues until the resource is found.
• Directories can contain child (sub) directories in the VFS.
• Child directories can have priorities just like root directories.
• Child directories can share the same name. When multiple directories have identical names within a branch, they are searched in order of their assigned priority.
• A directory added as a subdirectory is typically assigned a name. If a subdirectory is inserted without a name, it is treated as a sibling of the parent directory while still being logically associated with it. Removing (unlinking) the parent directory also removes its unnamed sibling.
• Directories can be dynamically inserted and removed in a running system.
• A directory provides the Virtual File System (VFS) with one of the following responses:
  1. Resource found - The search stops. This is the default if no value is returned. You can also return true.
  2. Resource not found - The search continues. The directory function returns false to continue searching the VFS.
• Resources that begin with a dot (e.g., .myhiddendir) cannot be accessed directly by a client. If a client attempts to access such a resource, the server responds with a 404 Not Found error. However, hidden resources can still be accessed indirectly using the response:include() and response:forward() methods. See hidden files for details.

The Lua .config script

The Lua .config script which must be located in the root of the VM's I/O interface is automatically loaded and executed when the C/C++ code creates the Lua VM.

The .config script is typically where you assemble your default Virtual File System. One thing to keep in mind is that Lua garbage collects all objects not being referenced. You must make sure that the directories you create in the .config script are referenced (global or owned by a global variable). An unreferenced directory will be removed from the VFS by the Lua garbage collector.

Example:

The following example shows a practical use of the directory priority mechanism in the VFS. Two resource readers are created: diskapp and zipapp. The diskapp is given priority 1 and the zipapp is given priority 0. The idea behind this construction is that you have designed an application and put the application into a ZIP file. The users can overload any of the files you have in the zip file by putting the files into c:/www. When the browser accesses a resource, the VFS first searches the user's c:/www directory. If the file is not found in the www directory, the ZIP file is searched. Assume that you have the style sheet style.css. The users can copy your style sheet from the ZIP file, change it, and put the file into c:\www. The browser loads the user's changed style sheet since it has a higher priority than the style sheet in the ZIP file.

We also create a Directory function and give it a very low priority. If the VFS cannot find the resource in diskapp or zipapp, the notFoundApp's directory callback function is activated. This function returns a 404 message to the user. The notFoundApp shows how to create customized "global" error handlers in the VFS.

local io=ba.mkio(ba.openio("root"), "c:/www")
diskapp = ba.create.resrdr(1,io) -- Priority 1
diskapp:insert()
io=ba.mkio(ba.openio("disk"),"myzip.zip")
zipapp = ba.create.resrdr(0,io) -- Priority 0
zipapp:insert()
local davm={PROPFIND=true,OPTIONS=true}
local function manage404(_ENV,path)
   if davm[request:method()] then return false end
   print("<html><body><p>",
         path,
         " not found",
         "</p></body></html>")
end
notFoundApp = ba.create.dir(-126) -- Priority -126
notFoundApp:setfunc(manage404) -- Enable 404 handler
notFoundApp:insert()

The Two I/O interfaces "disk" and "root" are initialized by the C/C++ code. The above example assumes that the "root" I/O is initialized to the root in a Windows computer. The base of an I/O interface can, for example, be set to a subdirectory by the C/C++ code.

WebDAV and 404 Page:
Line 7 and 9 are required when a WebDAV instance is installed in the virtual file system and when using Microsoft's WebDAV client.

VFS Node Types

The following introduces you to the Virtual File System (VFS) nodes , which serve as the foundation for managing and organizing resources within BAS. These nodes allow dynamic handling of web requests, enabling developers to map URLs to various functionalities, including serving static files, executing scripts, and integrating custom logic.

1. Directory Node

   Acts as the base type of the Virtual File System and can be extended with custom functionality via callback to handle dynamic routing.

2. Resource Reader

   Enables the server to return static content like HTML, JavaScript, and images. It can be upgraded to execute Lua Server Pages (LSP) , allowing dynamic content generation.

3. Web File Server (WFS)

   Provides an easy way to upload and download files via a web browser. Allows exposing server directories as web-accessible storage.

4. WebDAV Node

   Extends HTTP functionality to allow remote file system access and enables mounting the server as a network drive, which is helpful for editing LSP files remotely.

The resource reader, Web File Server, and WebDAV inherit from the base Directory Node; thus, all methods for the Directory Node are also available for the other nodes, including installing a directory callback function.

Directory

The directory object is the Virtual File System's "base type", also known as HttpDir when writing C code. The directory function is not doing much unless you extend its functionality. The directory type's default implementation is to chain or link in other directories. We will focus more on linking directories when we explain how to construct a virtual file system tree.

Extending the directory function is best illustrated with an example:

local function myservice(_ENV,path)
   -- _ENV is the command environment where the
   -- globals print, request, and response are found.
   print("<html><body><p>The relative path is: ",
         path,
         "<br> The absolute path is: ",
         request:uri(), -- Print the URI component of the URL
         "</p></body></html>")
end

dir = ba.create.dir("mydir") -- Create directory "mydir"
-- Enable the directory service function
dir:setfunc(myservice)
dir:insert() -- Insert "mydir" as a root directory.

In the above code, we create the Lua function "myservice". We register this function as a directory callback function just after creating a directory object by calling ba.create.dir. The directory object created is then inserted as a root directory in the server. We name the directory created "mydir", thus making the directory available at http://localhost/mydir/, where "localhost" is the address to the server.

The directory service callback function "myservice", installed by calling dir:setfunc, is executed when a client accesses the directory. Just like an LSP page, a directory service function has a number of predefined objects in the command environment such as the request object, the response object, and function print. Function print sends data back to the browser.

The third argument to function "myservice" is the relative path to the virtual resource requested. For example, entering http://localhost/mydir/a/b/c in a browser displays the following:

The relative path is: a/b/c
The absolute path is: /mydir/a/b/c

A shopping cart with clean URL's

You have probably noticed that many online shopping carts use dirty URLs such as:

http://example.com/cgi-bin/gen.pl?category=flowers&color=blue

The last part of the above URL contains URL encoded data: ?category=flowers&color=blue. In Barracuda, URL encoded data can be accessed as follows:

local data = request:data() -- Return URL encoded data as a Lua table
print(data.category) -- prints flowers
print(data.color) -- prints blue

Dirty URLs do not promote usability and are difficult to remember. Also, browsers can have a hard time caching pages containing URL encoded data.

Using clean URLs with Barracuda is easy. The above URL converted to a clean URL would look like:

http://example.com/cart/flowers/blue

Our cart implementation:

local function cart(_ENV,path)
   category,color=string.match(path, "(%w+)/(%w+)")
   response:forward("/.cart.lsp")
end
local cartdir = ba.create.dir("cart")
cartdir:setfunc(cart)
dir:insert(cartdir, true)

Download the shopping cart examle from GitHub

In the above example, we create a cart directory and make the directory available at URI /cart/. The cart directory service function extracts the category and the color from the path and then transfers the request to the LSP page ".cart.lsp". We have, for the sake of simplicity, omitted error checking.

The URL http://example.com/cart/flowers/blue uses a combination of physical and logical elements. The base path http://example.com/cart/ is a physical URL to the directory instance installed in the Virtual File System. The URL http://example.com/cart/flowers/blue is a logical URL; in other words, the path flowers/blue does not relate to a physical page.

The two variables, category and color which are extracted from the path, are declared in the temporary command environment. The temporary "command environment" is valid for the length of the request. This means that the Lua Server Page ".cart.lsp" can access these two variables.

Resource Reader

A default resource reader enables the server to return static content to a client. Static content includes HTML files, images, JavaScript Files, etc. A resource reader can also be upgraded to run Lua Server Pages (LSP), thus delivering dynamic content.

-- Create a new ZIP I/O interface by loading "myzip.zip" from another I/O.
-- I/O root directory.
local myzipio=ba.mkio(ba.openio("disk"), "myzip.zip")
zipapp = ba.create.resrdr("myzip",myzipio)
zipapp:lspfilter() -- Upgrade resource reader to execute LSP resources.
zipapp:insert() -- Install as a root directory.

A resource reader instance must be associated with a Barracuda file system, a.k.a. I/O interface. In the above example, we create a new I/O interface by attaching the I/O to a standard ZIP file located in the disk root directory. The created I/O is then associated with a resource reader instance. Finally, the resource reader is inserted into the Virtual File System such that it is available at base URI /myzip/.

Let's assume the ZIP file contains the following files:

• index.lsp: Accessible via: http://localhost/myzip/index.lsp
• directory/index.html: Accessible via: http://localhost/myzip/directory/index.html

By using a ZIP file, BAS allows these files to be accessed just like a regular directory structure on a traditional file system.

• LSP Files: When a browser accesses index.lsp, the LSP page is uncompressed, compiled, and executed by the LSP engine. The LSP page then sends uncompressed dynamically created data to the browser.
• Static Resources: When a browser accesses directory/index.html, the file is sent to the browser without uncompressing it, making storing static resources in ZIP files very efficient.

Web File Server

The Web File Server enables one to easily upload and download files using a Web Browser. Example: local iodisk = ba.openio"disk" wfs=ba.create.wfs("fs", iodisk) wfs:insert()

In the above example, a Web File Server directory is created and made available at base URI /fs/ -- i.e. URL: http://address/fs/, where "address" is the domain or IP address of the server.

Note: When you click the above run button, you will get an accessed denied message due to the cross-origin safety setting. You resolve this by clicking in the browser’s address bar and pressing the enter key.

WebDAV

WebDAV is a set of extensions to the HTTP protocol that enables a web server to appear as a standard network drive. If you are new to WebDAV, take a look at the FuguHub's WebDAV plugin product sheet for a brief overview of WebDAV.

WebDAV is particularly useful when developing LSP pages and when the server is not running on "localhost". By mapping a drive in Windows to the remote server or mounting the remote server on Linux, a standard text editor can be used to directly edit the LSP pages on the remote computer. Refreshing the LSP page in a web browser automatically recompiles the file after it has been changed.

local iodisk = ba.openio"disk"
davd=ba.create.dav("mydav",ba.openio"disk",".LOCK")
davd:insert()

In the above example, a WebDAV directory is created and made available at base URI /mydav/ -- i.e.
URL: http://address/mydav/, where "address" is the domain or IP address to the server.

On Windows, the above can be mapped as a drive from a DOS command window as follows:

net use * http://address/mydav/

Windows should subsequently show information similar to the following:

Drive Y: is now connected to http://address/mydav/.
The command completed successfully.

On Linux, davfs2 is the recommended file system driver.
On Mac, go to the Finder and press "Apple-K". Enter the WebDAV URL and click the connect button.

Tips:

• A WebDAV client that is not operating as a file system driver is not as useful since you cannot use an editor to directly edit files on a remote server. Many WebDAV clients work similar to ftp -- i.e. you first edit the file locally and thereafter upload the file to the remote server.
• The Windows "net use" command uses the Windows WebDAV Mini Redirector which is an integral part of Windows. This tool has some quirks such as not accepting non standard server port numbers. See the WebDAV Mini Redirector tutorial for more information if you experience problems.
• A WebDAV server must not be used without an authenticator in production code.
• It is easier to use a WebDAV server without an authenticator when doing development on a private LAN.
• A WebDAV server can use a ZIP I/O interface, but the files will be "read only". The lockdir parameter can be set to nil.
• Some WebDAV clients such as the Windows WebDAV Mini Redirector may make the operating system hang if the server is stopped. If you stop the server, make sure you do not use the DAV mount or click on any program that has open files before the server is back up and running. Stopping a server is common during development. The HTTP protocol is stateless and everything should be back to normal on the client when the server is back up and running -- i.e. there is no need to reconnect the DAV client.

Hidden Files

Hidden files are files whose names begin with a dot (e.g., .page.lsp). These files can only be accessed by server-side code. If a client requests a hidden file directly, the server responds with a 404 Not Found error.

• Hidden files can be accessed indirectly using the response:forward() and rresponse:include() methods, but not with response:redirect().
• A directory that starts with a dot hides all of its contents from direct client access.
• HTML files with the .shtml extension are treated as hidden. In Barracuda, .shtml files are used for server-side includes (SSI) and are typically employed in the creation of hidden C Server Pages (CSP).

Ready-to-run Virtual File System Tutorials

• Building Industrial Web Applications - A Ready-to-use Foundation
  This tutorial provides a ready-to-use foundation for building industrial web applications. It demonstrates how to create a web-based management interface by leveraging a directory function to dynamically map URLs to files. For a complete walkthrough, see the Interactive Dashboard App Tutorial.
• URL-to-Database Mapping Using a Directory Function:
  This tutorial explains how to implement URL-to-database mapping through a directory function combined with the SQLite API. It provides a step-by-step guide to building a basic wiki engine, demonstrating efficient techniques for linking URLs to database queries. For more details, see the Lua Wiki Engine Tutorial.
• Embedding Sessions in URLs:
  The article Single Sign-On demonstrates how to dynamically embed a session in the URL, a practical approach when using an HTTP client that does not support token-based authentication. By leveraging a directory function, users can sign in through a browser, copy the authenticated URL, and seamlessly use it with a WebDAV client that would otherwise fail to authenticate.