ESP32 WebDAV Server

ESP32 WebDAV Server


WebDAV, an HTTP(S) protocol extension, transforms a web server into a standard network drive, enabling seamless file sharing across various devices like PCs, iPhones, Androids, and Linux systems. With WebDAV, users can effortlessly map or mount a drive, eliminating the need for extra software installations. See the WebDAV server introduction for a more detailed explanation.

You can use WebDAV with your favorite code editor as a substitute for the web-based Xedge IDE. Once the ESP32 is set up and mounted, you can directly manage files - editing, creating, deleting Lua source code, or updating configuration files - right from your favorite editor. However, If you aim to use the ESP32 as a compact WebDAV server, ensure your ESP32 has a micro-SD card slot since the onboard flash file system is too small to be used for file storage. The main focus in this tutorial is to set up a file storage server. We used the Freenove ESP32-S3 CAM board for our tests paired with a 128Gb micro SD card. Alternatively, you can use the XIAO ESP32S3 Sense, as shown in the above image, combined with an SD card expansion board. This setup allows you to create the world's smallest functional WebDAV server.

To follow this step-by-step guide, first, upload the necessary firmware and familiarize yourself with executing xlua files.

Enabling the "sd" IO Interface

After flashing the firmware, inserting the SD card, and powering on the board, proceed to enable the SD card on your ESP32 board. Execute the following command using LuaShell32 or via an xlua file.

 trace(esp32.sdcard(1, 39, 38, 40))

Example 1: Enabling the SD card on the Freenove ESP32-S3 CAM board

The above command is for the Freenove board. Make sure to use the correct settings for your board. See function esp32.sdcard for details. The ESP32 will automatically reboot if the command is successful; otherwise, the error return value is printed using function trace. When the ESP32 has rebooted, enter the following command to verify that the SD card works.

 trace(ba.openio("sd"))

Example 2: Check if SD card IO is enabled

A successful response will display "IO Interface" followed by a number. However, if the response is nil, it indicates that the SD card is not functioning properly.

Testing the SD Card Using the Ready-To-Use WebDAV Server

While we'll guide you on crafting your own WebDAV server object in the next section, let's first ensure the SD card functions using the built-in Xedge Web-File-Manager and WebDAV instance.

To access the SD card using the Web File Manager, either:

  • Directly visit http://ip-address/rtl/apps/sd/
  • Or, start at the root directory http://192.168.1.159/rtl/apps/ and select "sd".

You can now mount this URL as a WebDAV drive. For a step-by-step guide on this process, refer to the video tutorial titled How To Mount WebDAV. Try uploading and downloading a few files.

Creating a combined Web-File-Manager and WebDAV instance

The built-in Xedge Web-File-Manager and WebDAV instance located at /rtl/apps/ are tailored for development purposes and aren't ideal for file server use. This is because the default server incorporates multiple layers, like LuaIo, that process uploads and initiates specific Xedge logic. For instance, when you upload an xlua file to an app within Xedge's built-in file manager, it first terminates the existing xlua program and then runs the newly uploaded xlua program.

Typically, when we talk about a combined Web-File-Manager and WebDAV instance, we use the term file-server or simply 'fs', and the root of the file system, in this case, the SD card, is typically inserted into the server's virtual file system (VFS) at /fs/, making the combined Web-File-Manager and WebDAV instance accessible at http://ip-address/fs/.

The following code creates an 'fs' instance by calling ba.create.wfs() and inserts the instance into the root of the virtual file system (VFS).

local wdio = ba.openio"sd" -- The WebDAV IO : SD card root
-- Create a combined web file manager and WebDAV server called 'fs'
local fsdir=ba.create.wfs("fs",10,wdio,".LOCK")
fsdir:insert() -- Insert as a root node with name 'fs' in the VFS
trace"Web-file-manager and WebDAV installed at '/fs/'"

function onunload()
   trace"Uninstalling 'fs'"
   fsdir:unlink()
end

Example 3: Inserting a combined Web-File-Manager and WebDAV instance at '/fs/'

How to run and test Example 3

  1. Create or use an existing Xedge app.
  2. Generate an xlua file and name it fs.xlua.
  3. Using the Xedge IDE, insert the above Lua code into this file.
  4. The code will be activated upon saving the file, establishing an 'fs' instance immediately.
  5. Use a browser and navigate to http://ip-address/fs/. You should see the Web file Manager.
  6. Using your computer, map/mount the WebDAV server using the URL http://ip-address/fs/

Implementing Authentication and Establishing a User Database

While Example 3 offers convenience, it lacks authentication, meaning anyone can mount the drive and access its contents. To enhance security, we've provided two examples on GitHub:

  1. Basic Authentication (Example 1): This straightforward method ensures that only authorized users can access the server.
  2. User Database with Access Control Lists (ACL) (Example 2): A more advanced approach, this example demonstrates how to set up a user database and implement ACLs for granular access control.

Though these two examples are designed for the Mako Server, they are also compatible with Xedge32. To use them, either upload the respective .preload script from the example or copy its content into an xlua file. It's possible to run both examples concurrently. However, if you choose to do so, ensure you modify the VFS base URL '/fs/' for one of them to avoid conflicts.

The two examples are explained in detail in the tutorial How to Create a WebDAV Server.

Important: If you require authentication, it's crucial to also secure the built-in file-server located at '/rtl/apps/'. You must ensure that unauthorized access is prevented by either setting a password for Xedge or by entirely disabling the Xedge IDE.

WebDAV Single Sign On

Single Sign-On (SSO) is a user authentication method that allows individuals to access multiple applications with a single set of credentials. Its benefits include simplified user management, enhanced user experience by reducing password fatigue, improved security with fewer passwords to manage, and quicker access to multiple platforms without repeated logins.

The Xedge IDE offers SSO in addition to traditional authentication, but did you know that it is also possible to use SSO with WebDAV? Here is how it can be implemented: A user signs in via a browser, and upon authentication, the server issues a session ticket. This ticket can then serve as the base URL for a WebDAV server. This mechanism is secure when an HTTPS connection is used. For a hands-on example, check out the WebDAV SSO Example on GitHub. While crafted for Mako Server, it can be adapted to Xedge32. If you're unfamiliar with SSO, explore the tutorial Benefits of Active Directory Single Sign-On.

ESP32 WebDAV Speed Test

The ESP32 can effectively function as a WebDAV server for smaller files, such as Word documents and photos. However, it's essential to understand its limitations: serving videos or handling other high-bandwidth files is impossible. We conducted several speed tests with the Freenove ESP32-S3 CAM board, using Wi-Fi and Ethernet through a W5500.

The following shell script creates a one-gigabyte file, uploads it to the file server at IP address 192.168.1.156, and measures the time it takes to upload the file. Using a one-gigabyte file in the test is purely for benchmarking and comparison purposes, and it's not a recommended practice to store such large files on an ESP32 WebDAV Server, given its severe limitations in terms of transfer speed.

#!/bin/bash
# Create a 1GB file
dd if=/dev/zero of=file.bin bs=1G count=1
# Measure the time taken for the upload using curl
TIME_TAKEN=$( { time curl -F "data=@file.bin" http://192.168.1.156/fs/; } 2>&1 )
# Extract real time from the output
REAL_TIME=$(echo "$TIME_TAKEN" | grep real | awk '{print $2}')
echo "Time taken to upload the file: $REAL_TIME"

The following script, which can be run after uploading file.bin, measures the time it takes to download the 1GB file.

#!/bin/bash
# Measure the time taken for downloading using curl
TIME_TAKEN=$( { time curl http://192.168.1.156/fs/file.bin -o file2.bin; } 2>&1 )
# Extract real time from the output
REAL_TIME=$(echo "$TIME_TAKEN" | grep real | awk '{print $2}')
echo "Time taken to download the file: $REAL_TIME"

Speed Test Results:

  • Uploading Times:
    • Wi-Fi: 28 minutes and 50 seconds
    • Ethernet: 26 minutes and 39 seconds
  • Downloading Times:
    • Wi-Fi: 35 minutes and 43 seconds
    • Ethernet: 26 minutes and 53 seconds

The results show a little difference between Wi-Fi and Ethernet. However, it's worth noting that the SD card's performance also impacts transfer speed. For instance, when we uploaded a 1GB file to a service that discarded the incoming data, the upload via Ethernet took only 19 minutes and 47 seconds, achieving a transfer rate of 534 MB/s.

In summary, the ESP32 can serve as a WebDAV server for smaller files, but it has limitations when it comes to handling large files or high-bandwidth applications. The choice between Wi-Fi and Ethernet also affects the speed of file transfers, with Ethernet providing slightly better performance. If you're in search of a speedy WebDAV server, think about deploying Mako Server on devices like a Raspberry Pi. Think of Mako Server as the "big brother" of Xedge32; it's versatile and can be compiled for any embedded Linux system.

Xedge32 Tutorials:

Continue your journey and expand your expertise with our comprehensive Xedge32 tutorials.


Navigating Complexities? We've Got You!

Navigate the world of embedded web servers and IoT effortlessly with our comprehensive tutorials. But if time isn't on your side or you need a deeper dive, don't fret! Our seasoned experts are just a call away, ready to assist with all your networking, security, and device management needs. Whether you're a DIY enthusiast or seeking expert support, we're here to champion your vision.



OPC-UA

OPC-UA Client & Server

An easy to use OPC UA stack that enables bridging of OPC-UA enabled industrial products with cloud services, IT, and HTML5 user interfaces.

Edge Controller

Edge Controller

Use our user programmable Edge-Controller as a tool to accelerate development of the next generation industrial edge products and to facilitate rapid IoT and IIoT development.

On-Premises IoT

On-Premises IoT Platform

Learn how to use the Barracuda App Server as your On-Premises IoT Foundation.

Embedded Web Server

Barracuda Embedded Web Server

The compact Web Server C library is included in the Barracuda App Server protocol suite but can also be used standalone.

WebSocket Server

Microcontroller Friendly

The tiny Minnow Server enables modern web server user interfaces to be used as the graphical front end for tiny microcontrollers. Make sure to check out the reference design and the Minnow Server design guide.

WebDAV Server

Network File System

Why use FTP when you can use your device as a secure network drive.

HTTP Client

Secure HTTP Client Library

PikeHTTP is a compact and secure HTTP client C library that greatly simplifies the design of HTTP/REST style apps in C or C++.

WebSocket Client

Microcontroller Friendly

The embedded WebSocket C library lets developers design tiny and secure IoT applications based on the WebSocket protocol.

SMTP Client

Secure Embedded SMTP Library

Send alarms and other notifications from any microcontroller powered product.

Crypto Library

RayCrypto C Library

The RayCrypto engine is an extremely small and fast embedded crypto library designed specifically for embedded resource-constrained devices.

Embedded PKI Service

Automatic SSL Certificate Management for Devices

Real Time Logic's SharkTrust™ service is an automatic Public Key Infrastructure (PKI) solution for products containing an Embedded Web Server.

Modbus

Modbus TCP client

The Modbus client enables bridging of Modbus enabled industrial products with modern IoT devices and HTML5 powered HMIs.

Posted in Xedge32