Barracuda Application Server C/C++ Reference
 All Classes Functions Variables Typedefs Enumerations Enumerator Groups Pages
JSON and UBJSON

JSON has become a popular inter-process communication (IPC) data interchange format for a variety of computer languages. UBJSON (Universal Binary JSON), the binary version of JSON, provides a more compact encoding format than standard JSON. Although the JSON and UBJSON encoding and decoding libraries are part of the Barracuda Application Server, they can also be used standalone. The libraries are implemented in C code and provide a C API and C++ API. The C++ API consists of inline C++ methods in the header files. The C libraries conform to the object oriented C APIs in the Barracuda Application Server.

The JSON and UBJSON C libraries are designed such that they can function without dynamic memory allocation, which is a requirement found in many embedded/microcontroller solutions. Consequently, the libraries are ideal for use "standalone" in small microcontrollers, where JSON or UBJSON can be used for encoding and decoding data.

JSON and UBJSON Use Cases

A typical JSON use case is to send encoded data over HTTP or other similar Remote Procedure Call (RPC) based protocols. The JSON and UBJSON libraries we provide extend well beyond the typical or traditional JSON use cases. For example, the encoders/parsers can be used for encoding/decoding data on a fully asynchronous bi-directional communication stream such as a real-time Machine-2-Machine (M2M) channel. Both parsers are stream and event based, thus capable of parsing data as it trickles in on a continuous data stream. The parsers do not need any start/end marker on an incoming multi JSON object stream and can be connected directly to a raw socket connection. Event based means that the parsers do not block and wait for new data. As such, the parsers can be used in an event driven bare metal embedded system as a means for providing encoding/decoding of real time data sent over the Internet.

http-vs-async.jpg
Figure 1 shows an RPC protocol versus asynchronous communication.

Use cases include, but are not limited to:

  • Encode/decode messages sent on a continuous data stream such as on a raw TCP/IP connection
  • Encode/decode messages sent on a WebSocket connection
  • Encode/decode messages sent via an MQTT topic
  • Encode/decode messages sent/received via RPC protocols such as HTTP
  • Encode/decode data for persistent storage on disk (mini database)

Using JSON or UBJSON:

UBJSON is friendlier to C/C++ design than JSON, is more compact, and supports sending binary data in objects without having to resort to B64 encoding, however, JSON is more ubiquitous and it is easier to find a library for other platforms that supports JSON. We support JSON and UBJSON via the C libraries and via the Lua scripting language in the Barracuda Application Server and derivative products such as the Mako Server. The UBJSON library is smaller and uses less resources than the JSON library.

Introduction

As we mentioned above, the JSON and UBJSON libraries are implemented in C code, with a C++ wrapper API defined in the header files. We will use a mixture of C and C++ in the examples that follow. In general, you can easily switch between C and C++ as the following example illustrates:

JVal* child = JVal_getArray(parent); /* C code example */
JVal* child = parent->getArray(); // C++ code example

We will focus on using JSON and UBJSON for encoding and decoding C/C++ data structures. You can apply the information you learn here and use it for encoding and decoding data for various purposes such as encoding messages sent over a Machine to Machine connection. The following figure provides a high level view of how a C structure can be encoded and decoded by using method "set" and method "get".

JC-Struct.jpg
Figure 2: Encoding and decoding C/C++ structure to/from JSON format (using C++ API).

In Figure 2, we use method "set" for encoding a C structure as a JSON stream and method "get" for decoding a JSON stream and converting it into a C structure instance. Method "set" is similar in functionality to function printf in that it provides formatted output. The first argument specifies the types and length of the variable number of arguments that follows. Method "get" is similar to function scanf in that it performs formatted input. We are also using a few optional helper macros in the above example. We will get more into how these macros are used later. The UBJSON API is similar, however, the format specifiers differ from the JSON API.

Introduction to Encoding Data

Encoding data is straightforward. You serialize the C structure by using method "set" or by using primitive methods that enable you to dynamically serialize data and/or C structures. The following C++ example copied from Figure 2 shows how to encode a C struct instance and it's 3 members:

encoder.set("{dsf}", // flags: start object, int, string, float, end obj
JE_MEMBER(&out, myInt), // Macro: JSON Encoder Member
JE_MEMBER(&out, myString),
JE_MEMBER(&out, myFloat));

The first argument provided to method "set" includes format flags that indicate the subsequent number of dynamic arguments and their type. The format flags for the "set" method closely matches the format flags for the "get" method. See the "get" method for how to use the format flags.

You can also encode data by using the primitive methods used internally by the "set" method. The following example shows how to use these methods to encode the data in the above example:

C++C Code
encoder.beginObject();JEncoder_beginObject(&encoder);
encoder.setName("myInt");// Member nameJEncoder_setName(&encoder,"myInt");
encoder.setInt(myInt);JEncoder_setInt(&encoder,myInt);
encoder.setName("myString");JEncoder_setName(&encoder,"myString");
encoder.setString(myString);JEncoder_setString(&encoder,myString);
encoder.setName("myFloat");JEncoder_setName(&encoder,"myFloat");
encoder.setDouble(myFloat);JEncoder_setDouble(&encoder,myFloat);
encoder.endObject();JEncoder_endObject(&encoder);

Figure 3 shows how to dynamically encode the C structure in Figure 2.

Introduction to Decoding Data

Decoding data is more complex than encoding data since the incoming JSON or UBJSON stream must be parsed and broken up into tokens. The decoder logic consists of a parser and a decoder that plugs into the parser. The JSON parser is designed for parsing a continuous JSON text stream, and the UBJSON parser is designed for decoding a continuous binary UBJSON stream.

JParser.jpg
Figure 4 shows how the event driven parser callback API functions.

Both parsers provide an event based callback API, where you can install your own callback function or use one of the ready to use decoders for JSON (JParserValFact or JDecoder) or UBJSON (UBJDecoder).

The JDecoder (JSON) and the UBJDecoder (Universal Binary JSON) are designed to make it easy to decode JSON/UBJSON and convert the incoming data directly to data represented by a C structure and/or C array.

JParserValFact is an optional JSON value factory builder that can be used instead of the JDecoder. JParserValFact provides an API that is much more advanced than the API provided by JDecoder and is more suitable for decoding complex types. Note: A value factory builder is not provided for the UBJParser.

The JSON and UBJSON Lua Value Factory builders are part of the Barracuda Application Server's LSP bindings and are designed to automatically build Lua tables and primitive values from the parsed data.

Using the JDecoder and UBJDecoder

JDecoder and UBJDecoder are designed for memory constrained systems where a value factory such as JParserValFact is not suitable. Note: JDecoder and UBJDecoder do not use dynamic memory allocations, but they do need a buffer as explained later. Method "get" builds a pointer value tree where the leaf nodes contain pointers to the primitive values in your C structure. The following figure shows how this works.

JDecoder.jpg
Figure 5 shows how the JDecoder::get method builds a pointer value tree.

It is important to understand that the "get" method does not actually update any values in your C struct instance. What it does is to build a tree where the leaf nodes have pointers to the primitive values in your struct instance. For example, in the above figure, the green 'id' leaf node above contains a pointer to ld.green.id. When the parser calls the callback function in the JDecoder or UBJDecoder instance, the correct leaf node is found, and this is when the value in the struct instance is set.

There are some design limitations imposed by the JDecoder and UBJDecoder. JSON and UBJSON maps more naturally to dynamic data represented by scripting languages. The mapping to strict C structures and/or C arrays as managed by the "get" method limits the dynamic features of JSON and UBJSON. You must also know the representation layout of the parsed JSON and UBJSON data before parsing since the "get" method, which builds the pointer value tree, must be called before parsing. However, if you define a strict API, this should not be a problem. As an example, say you know the peer side will send you the exact command sequence: LED, MOTOR, TEMPERATURE. From this knowledge, you can call the "set" method and parse as the following pseudo code shows:

decoder.get(<<LED>>) /* Use as shown in Figure 5 */
until complete: parser.parse(stream data)
decoder.get(<<MOTOR>>)
until complete: parser.parse(stream data)
decoder.get(<<TEMPERATURE>>)
until complete: parser.parse(stream data)

Every time we call method "get", we re-arrange the internal pointer value tree in the decoder to represent the new struct instance thus making the decoder ready for the next object parsed by the parser.

The JDecoder and UBJDecoder "get" methods use a variable number of arguments. The first argument specifies the subsequent arguments.

  • An object starts with '{' and ends with '}'
  • An array starts with '[' and ends with ']'

Objects must include the member name; for example, the following C++ code example from Figure 2 includes the member name via the preprocessor expansion of the support macros.

decoder.get("{dsf}",
JD_MNUM(&in, myInt), // Macro: JSON Decoder Member Number
JD_MSTR(&in, myString), // Macro: JSON Decoder Member String
JD_MNUM(&in, myFloat));

The following C++ example shows how to call the "get" method without using the helper macros:

decoder.get("{dsf}",
"myInt", &in.myInt,
"myString", &in.myString, sizeof(in.myString),
"myFloat", &in.myFloat);

Notice that a string also requires the buffer size. The parser callback (in JDecoder and UBJDecoder) checks the size of the buffer when copying the decoded string to the buffer. The parser callback will return an overflow error to the parser if the decoded string is larger than the provided buffer. The maximum string length is "buffer size - 1".

The JSON Decoder Member Number and Member String macros:

#define JD_MNUM(o, m) #m, &(o)->m
#define JD_MSTR(o, m) #m, &(o)->m, sizeof((o)->m)

The member name must also be included when with nested structures. The example in Figure 5 includes the member name "green" and "blue" when calling the "get" method.

You should note that if your format flag does not correspond to the arguments provided, the code will crash when calling method "get".

Arrays are easier to manage. The following C++ example shows how to decode an array of 3 elements:

int array[3];
decoder.get("[ddd]",&array[0],&array[1],&array[2]);

As you can see from the above example, using format flags for extracting an array is easy, but not very efficient since we need to specify each element in the array. For large arrays or dynamic length arrays, you can either install your own parser callback directly in the parser or use the 'X' format flag, which enables you to install a callback for sections of the data being parsed. The following example illustrates how this works:

JParserIntf intf(myParserCallbackFunction);
decoder.get("[X]", &intf);

In the above example, the Decoder calls the chained parser callback for each element in the decoded array. The following example shows a more complex use, where an object includes two member arrays and where the member names in the object is called "array1" and "array2".

JParserIntf array1(myParserCallbackFunction1);
JParserIntf array2(myParserCallbackFunction2);
decoder.get("{[X][X]}",
"array1", // Member name
&array1, // callback
"array2", // Member name
&array2); // callback

Using JParser and JDecoder

(Creating a JParser object and initializing it with a JDecoder callback object)

The following example shows how to create a JSON parser and decoder, in C++ and in C code.

JSON parser and decoder in C++ code:

char memberName[10];
unsigned char buf[1024];
JDecoder decoder(buf, sizeof(buf));
JParser parser(&decoder, memberName, sizeof(memberName),
AllocatorIntf_getDefault());

JSON parser and decoder in C code:

char memberName[10];
unsigned char buf[1024];
JDecoder decoder;
JParser parser;
JDecoder_constructor(&decoder, buf, sizeof(buf), 0);
JParser_constructor(&parser, (JParserIntf*)&decoder,
memberName, sizeof(memberName),
AllocatorIntf_getDefault(), 0);

Figure 6 shows creating a JSON parser and decoder in C++ and C code.

The JParser and JDecoder do not require dynamic memory allocation, but they do require temporary memory for processing the data.

The buffer used by the JDecoder is initialized and used when building a pointer value tree when the "get" function is called. The buffer must remain valid as long as the JDecoder is used. The buffer must be no less than N* 16 bytes on a 32 bit processor, where N is the number of format flag parameters used when calling method "get". See the reference manual for details.

The parser requires two buffers. The object member name buffer must be no less than the largest expected member name size, e.g. the member name "myFloat" in figure 2 requires 8 bytes (7+1). The second buffer is used for internal housekeeping such as temporarily storing parsed data including strings, if used. The parser will return an error code if it parses JSON data that does not fit into the provided buffers. In the above example, memory for the second buffer is provided by a dynamic memory allocator that can grow the buffer for large strings. An embedded system without an allocator can set a fixed size buffer, but the buffer must not be smaller than 256 bytes and no less than the largest expected string.

Creating a UBJparser and UBJDecoder is similar to creating the JSON version shown above. The UBJParser uses less memory than the JParser, especially for strings. The JParser must assemble and temporarily store a parsed string. The string is passed in whole to the parser callback. However, the UBJParser does not need to process string data and a large string may be sent as fragments to the parser callback as the data trickles in on the stream.

Using the Parser

As we mentioned above, the JParser and UBJParser are stream based and can parse data as it trickles in on a stream such as on a socket connection. In the following section, we will explain how this works, but before doing so, let's look at a more traditional way of using a JSON parser. In the following example, the file "oneobject.json" contains one JSON object. We have omitted error checking on reading the buffer.

fp = fopen("oneobject.json","rb");
size=fread(buf, 1, sizeof(buf), fp);
fclose(fp);
parser->parse(buf, size);
if(parser->getStatus() != JParsStat_DoneEOS)
{
printf("Parse err\n");
return -1; // Error
}
return 0; // OK

Figure 7 shows how to parse one JSON object.

In the above example, a complete JSON string is loaded into a buffer and passed into the parser. Since we load one complete object, we expect the parser to be in state JParsStat_DoneEOS (Done parsing and end of stream) after parsing the data.

Now, let's assume that we instead load a buffer with multiple JSON objects as shown in the following example:

fp = fopen("manyobjects.json","rb");
size=fread(buf, 1, sizeof(buf), fp);
fclose(fp);
for(parser->parse(buf, size) ;
parser->getStatus() == JParsStat_Done ;
parser->parse(buf, size) )
{
// Use the parsed and decoded object
}
if(parser->getStatus() != JParsStat_DoneEOS)
{
printf("Parse err\n");
return -1; // Error
}
return 0; // OK

Figure 8 shows how to parse multiple JSON objects in one buffer.

In the above example, we continuously call the parser until the state is not JParsStat_Done (Done parsing, but not end of stream). When the parser state no longer is JParsStat_Done, we expect it to be JParsStat_DoneEOS (done and end of stream).

JParsStat_Done means that the parser parsed a complete JSON object, but that the buffer passed into the parser contains additional data. When the parser returns JParsStat_Done, the parser must be called again with the same buffer unmodified.

For the final example, we will show how to read data from a stream as it trickles in. The data could for example come from a raw socket connection. The following example emulates a data stream by reading one byte at a time from a JSON data file containing multiple JSON objects.

fp = fopen("manyobjects.json","rb");
for(;;) // Until EOF stream or error
{
// Read one byte
unsigned char c = (unsigned char)fgetc(fp);
if(feof(fp)) break; //No more data
int status = parser->parse(&c, 1); //One byte
if(status)
{
if(status < 0)
break; // Parse error
// else: status == 1. We parsed a complete object
// Use the parsed and decoded object
}
//Else: status == 0. Parser needs more data
}
fclose(fp);
if(parser->getStatus() != JParsStat_DoneEOS)
{
printf("Parse err\n");
return -1; //Error
}
return 0; // OK

Figure 9 shows how to parse data as it trickles in on a stream.

We do not check the return value from the parser in figure 7 and 8, but instead check the parser status. In addition to using the status, the return value from the parser can be used as shown above. The parser returns zero if the provided chunk of data did not include a complete JSON object, but parsed successfully – in other words, the parser needs more data in order to complete parsing the JSON. The parser returns minus one on any error and one when a JSON object is completely parsed.

In the above example, we feed the parser only one byte at a time, thus the parser state can only be one of JParsStat_DoneEOS or JParsStat_NeedMoreData as long as no errors occur. The state cannot be JParsStat_Done, which means that the buffer contains more data to be parsed. In a real case scenario, you would not read one byte at a time from a stream, but say 1024 bytes. In such an application, you must also check for the JParsStat_Done state and make sure you call the parser again with the same buffer unmodified. Figure 11 includes an example that shows how this is done.

Using JParserValFact

The JParserValFact can be used as an alternative to JDecoder. We do not have an equivalent to JParserValFact for UBJSON, however, it is possible to also create one for UBJSON.

The JParserValFact builds a tree structure of JVal objects. A JVal instance represents one JSON value.

JParserValFact.jpg

The difference between the JParserValFact and JDecoder is that the JParserValFact can dynamically build and represent any JSON type. However, the JDecoder can only decode JSON that exactly matches the pointer value tree created by calling method "get".

An instance of JParserValFact is initialized as follows (using C code):

JParser p; /* JSON parser */
JParserValFact pv; /* JSON parser value factory. Used by the parser */
AllocatorIntf* alloc = AllocatorIntf_getDefault();
JParserValFact_constructor(&pv, alloc, alloc);
JParser_constructor(&p, (JParserIntf*)&pv
memberName, sizeof(memberName),
alloc, 0);

Figure 10 shows how to Create a JParserValFact and a JParser.

The JParserValFact constructor takes two allocators as arguments, where one allocator is used for allocating JVal types and the other for allocating strings. We previously mentioned that the JParserValFact requires dynamic memory allocation, but it is possible to use it with static buffers. The reason for using two allocators is to make it easier to use static buffers. The JVal pointers must typically be aligned on word boundaries on most CPUs, but not strings, which can contain non aligned lengths. Releasing memory is not done before terminating the JParserValFact instance; thus, the allocator does not need to implement releasing or reallocating memory. An allocator for the JParserValFact can for this reason simply advance an internal pointer in a static buffer for each new allocation.

The following C example illustrates how one can use the JParser and JParserValFact for receiving and decoding asynchronous JSON messages received over a socket connection. Each JSON message includes a message number that uniquely identifies the message type received. All JSON messages include this message number.

#define ALARM1 4 // Message number
typedef struct
{
char* msg;
int level;
} Alarm1;
int status,n;
char* buf[1024];
JErr err;
JErr_constructor(&err);
for(;;) // forever
{
// More data to be parsed in buffer if state is JParsStat_Done
if(JParser_getStatus(&p) != JParsStat_Done)
{
n=socketread(sock,buf,sizeof(buf)); // Read from stream
if(n <0 ) break; // Stop: No more data or stream err.
}
status = JParser_parse(&p, buf, n);
if(status < 0)
break; // Stop: Parse error.
else if(status > 0)
{ // A Complete JSON object
int msgno;
JVal* v = JParserValFact_getFirstVal(&vp); // Root tree node
// Extract the message number that is common for all structures.
// Typically used with M2M asynchronous message transfer,
// but not with JSON over HTTP.
JVal_get(v, &err, "{d}", "msgno", &msgno);
if(msgno == ALARM1)
{
Alarm1 a1; // struct alarm
// Extract data from JSON tree
JVal_get(v, &err,"{sd}",
" message ",&a1.message,
"level",&a1.level);
if(JErr_noError(&e))
{
// Use data in a1 (Alarm1)
}
}
// else, other message types
/* Delete the JSON value tree, thus making the
* JParserValFact ready for the next object in the stream. */
JParserValFact_termFirstVal(&pv);
}
// Else: need more data. Continue loop.
}

Figure 11: Continuously parsing data on a socket stream (C code example)

When using the parser with HTTP, you typically read the complete HTTP input, create a parser, parse all data at once, and then terminate the parser. The parser can also be used when receiving HTTP chunk encoded data with unknown length. You would use the parser similar to above; i.e. parse data as it streams in on a HTTP chunk encoded stream.

Notice how we skip reading from the data stream if we parsed a complete object (state != JParsStat_Done). The reason for this construction is that a chunk read from the stream may contain parts of a new JSON object or contain multiple objects. We must call the parser again, without reading more data since the parser is at this point still using our input buffer.

In the above code, the JSON data is extracted from the JSON tree by the following code:

JVal_get(v, &err,"{sd}",
"msg",&a1.msg, //Pointer to string pointer
"level",&a1.level); // Pointer to int

The function is the inverse of JEncoder_set(). The function is similar to scanf. A format flag specifies the data structure to extract from the JSON tree. The function is internally parsing the format string and using the primitive JVal_XX functions. For example, the formatting "{sd}" instructs the function to do the following in sequence on the JVal tree:

  1. getObject(): The curly bracket { instructs JVal_get to go one level down in the JVal tree.
  2. getString(): Set the pointer a1.msg to the pointer in the JVal string member.
  3. v=JVal_getNextElem(v): Advance to next element
  4. getInt():Copy data into the pointer to integer &a1.level

The sender of ALARM1 can use the JEncoder set method when sending the message as illustrated below:

Alarm1 a1;
// Initialize/set a1.msg and a1.level
JEncoder_set(encoder,"{dsd}",
"msgno", ALARM1,
"msg",a1.msg,
"level",a1.level);
JEncoder_commit(encoder); /* Commit JSON data i.e. flush and send to peer. */

M2M LED JSON Client

The M2M LED JSON edition provides the same functionality as the M2M LED demo on our web site, except we use JSON instead of the primitive binary protocol that was designed for the original demo.

The benefit with using JSON is that it is easy to scale up the message exchange types in any way you want, and you will not have the restrictions found in the primitive binary protocol used in the original demo. The source code for M2M LED JSON edition can be used as a template for designing IoT applications where JSON is used for message exchanges.

The code consists of the device side implemented in C code and the server side code implemented in the Lua scripting language.

The JSON protocol used for exchanging messages is defined as follows:

[ // Start array
messageNumber, // ASCII value of the commands used such as 'S', 'R', and 'C'
messageObject // The messageNumber controls the type of this object
] // End array

The message number enables us to decode the message objects used. The message number is used in a C "switch" statement as illustrated in the following example:

switch(messageNumber)
{
case 'S':
/* decode the 'S' JSON messageObject */
break;
case 'C':
/* decode the 'C' JSON messageObject */
break;
}

M2M LED JSON Message Types:

Message 'A':

A for Accept client connection request. The server sends this command to the client if the connection request was accepted.

messageObject: none. The accept message does not require a message body.

Message 'R':

R for Register device. The device client sends LED information to the server, such as number of LEDs and LED colors, after receiving the 'A' message.

messageObject:

{
'devname': string // Device name
'leds': dynamic-length-array
}

The size of the dynamic-length-array depends on the device and how many LEDs the device has. This array contains object(s) of type:

{
'id': number, // LED ID number
'name': string, // LED Name
'color': string // yellow, green, blue
}

The 'id' is used in message 'S' when switching LEDs on or off. The name and color is used by the server logic when building the web pages for browsers connecting to the server.

Message 'S':

S for Switching LED on or off. Both client and server can send this command. A device client sends this command if a button for the LED was clicked on the device. The server sends this command if a connected browser either switches the LED on or off.

messageObject:

{
'id': number, // LED ID number
'on': boolean // Switch LED on if true and off if false
}

Message 'P':

P for Ping or Pong. Device sends ping and server sends pong. The purpose with ping and pong is to keep the persistent connection alive when communicating via network devices such as NAT routers that will close idle connections after a certain time.

messageObject: none

Message 'C'

C for gracefully Closing the connection. Server sends this message if it wants the client to close the connection.

messageObject:
{
'msg': string // Reason for closing the connection
}

Using JDecoder or JParserValFact in the device client

When designing the device side we have to decide if we should use JDecoder or JParserValFact for decoding the parsed JSON stream. JParserValFact is easier to use since a value tree is dynamically built for any received JSON data. The C code can then simply use the value tree as soon as a complete JSON object is parsed. However, the JParserValFact uses more memory than JDecoder; thus JDecoder is favorable in a design where memory is limited. In this example, a JDecoder is selected since we want the code to use as little memory as possible.

Using JDecoder with multiple message types

The JDecoder requires that we build a pointer value tree before parsing which creates a problem since we can receive multiple message types from the server. However, this problem can easily be overcome by creating a parser callback function that chains the parser callback function in JDecoder.

ChainedParserCB.jpg
Figure 12 shows how to set up a chained callback.

The chained parser callback function is designed for our protocol, where it decodes the message number and instructs the JDecoder to build a pointer value tree for the message object.

[ messageNumber, messageObject ]

The chained parser callback decodes the messageNumber and delegates decoding of messageObject to a JDecoder

The following code snippet shows the source code for the chained callback. The nesting level (nLevel) will be 0 for start array '[' and end array ']', The nesting level will be one for messageNumber and any value between 1 and N for messageObject. The callback can be in two states, decoding messageNumber or delegating the decoding of messageObject to the JDecoder.

M2MLED_parserCallback(JParserIntf* super, JParserVal* v, int nLevel)
{
M2MLED* o = (M2MLED*)super;
if(nLevel) /* outer '[' and ']' will be at level 0 */
{ /* We get here for messageNumber and messageObject */
if(o->messageNumber) /* State: get messageObject */
{
/* Delegate to the JDecoder parser callback.
Notice how we adjust the nesting level by removing outer array.
*/
return JParserIntf_serviceCB((JParserIntf*)&o->decoder,v,nLevel-1);
}
else /* State: get messageNumber */
{
if(v->t != JParserT_Int)
{
xprintf(("Expected type 'int' for messageNumber\n"));
return -1; /* Incorrect data from server */
}
o->messageNumber = v->v.d; /* Get message number */
switch(o->messageNumber)
{
case 'A': /* Server Accepted the HTTP connection */
/* No messageObject i.e. no payload */
break;
case 'S': /* Turn LED on/off */
VALIDATE(JDecoder_get(&o->decoder,"{db}",
JD_MNUM(&o->msg.S, id),
JD_MNUM(&o->msg.S, on)));
break;
case 'P': /* Pong (response to ping) */
/* No messageObject */
break;
case 'C': /* Server wants client to Close the connection */
VALIDATE(JDecoder_get(&o->decoder,"{s}",
JD_MSTR(&o->msg.C, msg)));
break;
default:
setProgramStatus(ProgramStatus_InvalidCommandError);
xprintf(("Received unknown command\n"));
return -1;
}
}
}
else /* Basic validation of outer array */
{
{
xprintf(("Expected '[' or ']'\n"));
return -1; /* Incorrect data from server */
}
}
return 0; /* OK */
}
/* end */

Figure 13 shows how to decode multiple object types based on message number

The message types we can receive from the server are 'A', 'S', 'P', and 'C', where 'S' and 'C' contain a messageObject. Notice how we build a JDecoder value tree for the two commands 'S' and 'C', thus preparing the JDecoder for decoding the messageObject. We delegate the decoding of the messageObject when the callback is in the state "messageObject" – i.e. when the messageNumber is non zero.

Notice that you can eliminate the chained parser callback if you instead use the JParserValFact since a JParserValFact instance can dynamically build any type of JSON value tree.

Parsing Stream Data

The following code snippet shows how we parse a chunk of data received from the socket connection. The code would have been almost identical if a JParserValFact had been used instead of a JDecoder.

M2MLED_manage(M2MLED* o, U8* data, U32 dsize)
{
int status;
do
{
/*Method 'parse' triggers 'M2MLED_parserCallback' */
status = JParser_parse(&o->parser, data, dsize);
if(status) /* Complete object or error */
{
if(status > 0) /* A complete object parsed */
{
switch(o->messageNumber)
{
case 'A': /* Server Accepted the HTTP connection */
if( (status = M2MLED_sendDevInfo2Server(o)) == 0)
{
xprintf(("Browse to: "
"http://" SERVERNAME "/IoT/led/json/\n"));
}
break;
case 'S': /* Turn LED on/off */
setLed(o->msg.S.id, o->msg.S.on);
status = 0; /* OK */
break;
case 'P': /* Pong */
status = 0; /* OK */
break;
case 'C': /* Server wants client to Close the connection */
setProgramStatus(ProgramStatus_CloseCommandReceived);
xprintf(("Connection close request from server: %s\n",
o->msg.C.msg));
/* status is 1: exit program in mainTask */
break;
default:
baAssert(0); /* Managed in M2MLED_parserCallback */
}
/* Reset state machine in M2MLED_parserCallback */
o->messageNumber = 0;
}
else /* Error */
{
/* Check 'enum JParsStat' in doc for actual values */
xprintf(("JParser or parser callback error: %d\n",
JParser_getStatus(&o->parser)));
}
}
/* else: Parser needs more data */
} while(status == 0 && JParser_getStatus(&o->parser) == JParsStat_Done);
/* Status: anything but 0 indicates that the connection must be closed */
return status;
}
/* end */

Figure 14 shows parsing and managing multiple message types

The M2MLED_manage method is called for each chunk of data received on the socket. Since a TCP/IP connection is stream based, we may not receive a complete JSON object or we may receive more data than one JSON object. The M2MLED_manage method is designed to manage these conditions. The do while loop checks if we received more than one object by checking the parser state. As we explained in Using the Parser, the parser must be called again with the same buffer if the state is JParsStat_Done.

M2M LED JSON Server

In the following section, we will take a look at some of the design elements of the server side code that is implemented in the Lua scripting language. The complete code can be downloaded and run on a Barracuda Application Server and any derivative product such as the Mako Server. For example, the server code can be run by the Mako Server as follows:

After starting the server code, download, modify, and compile the device code as follows:

  • Download the JSON C library, which includes the device client C code inside the examples directory.
  • Open the device client m2m-led.c in an editor, locate the macro SERVERNAME, and change the defined value to the IP address of the computer where the Mako Server is running. Use "localhost" if the device code runs on the same computer as the Mako Server.
  • Compile and run the device client code.
M2M-LED-multi.jpg
Figure 15: multiple browsers controlling one device.

The server code is, from a design perspective, more complex than the client C code since it must be able to manage multiple devices and multiple connected browsers. In addition, the server code must handle resource management such as multiple browsers controlling one device. The image to the right shows three browsers controlling one device. When a browser clicks on an LED button, the message is sent to the server. The server sends the command to the device and informs all connected browsers about the LED changing state.

The server code may be more complex from a design perspective than the client C code, however, the code is actually smaller since we use the Lua scripting language and not C code for the server side code. This is just one of the many benefits when using a scripting language versus native C code.

The server code uses a standard TCP connection when communicating with the device where the TCP connection is used for exchanging JSON messages between the server and device. The server code uses the EventHandler for maintaining a persistent connection with JavaScript code running in a browser window.

Like most scripting languages, Lua provides easy lookup of elements stored in a table. The following figure shows the main structural components of the server code:

m2m-led-server-logic.jpg
Figure 16 shows the main structural components of the server code.

The "devices" table stores device-info tables and the key used for looking up a device-info table is the device IP address. The device IP address is used as a unique key and identifies one device. The limitation with this design is that it limits the use of devices where the IP address is not unique such as for devices behind a NAT router.

The "cids" table (short for Connection ID table) includes a key (the cid) for all connected browsers. The value in this table is the device IP address. One browser window/connection can control one device. A cid is the value provided by the EventHandler and uniquely identifies one connected browser window.

The red "Logic" block in Figure 16 is the "glue" code that makes it possible for a browser to communicate with a device and vice versa. For example, when a browser sends a message to the server, the server looks up the device-info table as follows:

-- Lookup device-info table by using the EventHandler 'cid' (a unique number)
function lookupDev(cid)
local ipaddr = cids[cid] -- cid to ipaddr lookup
return ipaddr and devices[ipaddr] -- Returns dev-info if found
end

The above Lua code snippet shows how easy it is to use Lua tables for looking up elements. First, the device IP address is found by looking up the IP address in the "cids" table by using the cid as the key. If the IP address is found, the IP address is used as the key in the "devices" table. The function returns the "device-info" table if found or nil if not found.

The device-info table

The device-info table (shown in Figure 16) is created and inserted into the "devices" table when a new device client connects. This table stores vital information about the device and the browsers using the device.

  • js is a JSON stream parser instance, which is the Lua equivalent of the JSON parser logic used by the device client C code. The JSONS instance is used when receiving JSON messages from the device and when sending JSON messages to the device.
  • cids is a table similar to the global "cids" table, but this table only includes the browser windows connected to this particular device. The global 'cids' table include all connected browsers. As shown in Figure 15, the server code must update all connected browsers when a LED changes state, thus a table with all connected browsers is required.
  • name and leds are assembled by using data received from the device, when the device connects. The device sends message R after being accepted by the server. The name string and the leds table are used when dynamically building the web interface for the connected browsers.

The R message sent from the device when it connects is the most complex message in our JSON protocol so let's take a look at how the server code reads and decodes this message.

local device -- device-info table is set inside the pcall below (closure)
local ok,err = pcall(function()
local acceptedColors = {red=true, yellow=true, green=true, blue=true}
R = js:get() -- Get 'R' command: Register LED with server
assert(R[1] == string.byte'R',"Expected message R")
R = R[2] -- msg body
local leds={} -- Build table with device LED info (Ref-B)
for _,led in pairs(R.leds) do
local color = acceptedColors[led.color] and led.color or "red"
leds[led.id] = {color=color, name=led.name, on=false}
end
device = { -- build device-info table
js = js, -- JSONS instance
name=R.devname,
leds=leds, -- (Ref-B)
cids={} -- EventHandler ID list
}
end)
if not ok then
logerr("Command 'R' failed for dev %s: %s", ipaddr, err)
......

Figure 17 shows how to parse and decode the JSON message 'R'

In the above Figure, the code snippet R = js:get() uses the JSON stream parser instance to wait for and parse the complete R message. The data returned is a Lua table representing the data sent from the device client C code. In our JSON protocol, we have defined what message R must contain. However, Lua is a dynamic language and the R table could represent any value, including incorrect data. We must therefore validate table R. Validating the table can be done by checking each attribute of the table. However, the method we have selected is to assume that the data is correct. The code will then throw a Lua exception if it's not what it is expected to be so we wrap the code execution into a protected call (by using pcall). An exception thrown inside a protected call will not propagate beyond "pcall".

Server Source Code

The above provided an introduction to how the server side code functions. You can use the device client without getting into the details of how the server code works, but we suggest that you study the server code in detail if you plan on making your own JSON based real-time IoT framework for your own devices. Download instructions can be found above. The following lists the server components:

│   .preload -- Loads led-json.lua at startup
│
├───.lua
│       led-json.lua -- The core server code
│
└───led
    └───json
            device.js -- JavaScript code for device.lsp
            device.lsp -- Dynamically builds UI for device
            index.lsp -- Entry point (URL) for browsers and devices