A first system

We'll take a simple application, 'slip', through various stages, meeting most of the features and functionality of WhatOS along the way. All code may be found in doc/slip/.

Serial Line IP (SLIP) is a simple protocol for point to point serial communication using a wire. You will find many resources on the internet explaining it. Basically, you need to transmit a packet of arbitrary data, one byte at a time. Byte 192 marks the beginning and the end of the packet. If the packet data contains the number 192, it has to be escaped so that the receiver doesn't think it's the end of the packet. I won't go in the details of the escape mechanism; in this tutorial we'll just transmit packets that don't contain 192.

A system is a group of tasks communicating through signals. Tasks are not aware of each other; they simply receive and emit signals. When a signal is emitted, it is broadcast to all tasks listening for that signal. A signal can be pure or valued. A valued signal carries with it a certain value of a certain type while a pure signal carries no value -- it's mere presence is all of the information it carries.

We'll implement a system containing a Python SLIP transmitter task (pysliptx) and a C SLIP receiver task (csliprx).

Feature: Write tasks in Python or in C with invisible communication barriers.

from whatos.core import *
from whatos.embedded import packet      # we'll borrow packet type for now

END =            0xc0                   # indicates end of packet
ESC =            0xdb                   # indicates byte stuffing
ESC_END =        0xdc                   # ESC ESC_END means END data byte
ESC_ESC =        0xdd                   # ESC ESC_ESC means ESC data byte

# clone pytask and customize
class pysliptx(pytask):
    'SLIP transmitter Python task.'
    # define signal types:
    packet = packet()
    wuint8 = wuint8()
    # inputs/outputs:
    inpacket = input('inpacket', type = 'packet')
    outbyte = output('outbyte', type = 'wuint8')
    # what to do when I receive a signal:
    def process(self, input, inputval):
        # send an initial END character to flush out any data that may
        # have accumulated in the receiver due to line noise
        self.outbyte.emit(END)
        # for each byte in the packet, send the appropriate character
        # sequence
        for b in self.packet.to_list(inputval):
            if  b == END:
                self.outbyte.emit(ESC)
                self.outbyte.emit(ESC_END)
            elif b == ESC:
                self.outbyte.emit(ESC)
                self.outbyte.emit(ESC_ESC)
            else:
                self.outbyte.emit(b)
        self.outbyte.emit(END)

# clone ctask and customize
class csliprx(ctask):
    'SLIP receiver C task.'
    packet = packet()
    wuint8 = wuint8()
    inbyte = input('inbyte', type = 'wuint8')
    outpacket = output('outpacket', type = 'packet')
    # the following synthesize to C code: e.g. #define idle (0)
    defines = {'OVERFLOW' : 10,
               'idle' : 0,
               'receiving' : 1,
               'receiving_esc' : 2,
               'END' : END,
               'ESC' : ESC,
               'ESC_END' : ESC_END,
               'ESC_ESC' : ESC_ESC}
    process = """
    static wuint8 state = idle;
    static packet * p;
    wuint8 byte;
    byte = (wuint8) (unsigned int) inputval;
    await(-1);
    if (state == idle) {
        if (byte != END) {
            /* begin new reception */
            p = acquire(t_packet);
            if (!p) {
                TRACE1("rejecting byte -- no room for packet");
                await(0);
                require(t_packet);
                reject();
            }
            p->len = 0;
            if (byte == ESC) {
                state = receiving_esc;
                accept();
            }
            else {
                state = receiving;
                goto append;
            }
        }
        else {
            accept();
        }
    }
    else if (state == receiving_esc) {
        if (byte == ESC_END) {
            byte = END;
            state = receiving;
            goto append;
        }
        else if (byte == ESC_ESC) {
            byte = ESC;
            state = receiving;
            goto append;
        }
        else {
            /* protocol violation, drop packet */
            release(p);
            state = idle;
            accept();
        }
    }
    else { /* receiving */
        if (byte == END) {
            state = idle;
            emit(outpacket, p);
            accept();
        }
        else if (byte == ESC) {
            state = receiving_esc;
            accept();
        }
        else {
            goto append;
        }
    }
    /* reject signal if got an unexpected signal */
    reject();
    append:
        if (p->len == sizeof(p->data)) {
            /* overflow, drop packet */
            MTRACEE(OVERFLOW);
            release(p);
            accept();
        }
        p->data[p->len] = byte;
        p->len++;
        accept();
    """

We have created pysliptx by cloning object whatos.core.pytask and customizing it; similarly we have created csliprx by cloning object whatos.core.ctask and customizing it. Although these seem to be classes, they are actually objects. WhatOS is written using a style of programming called prototype-based programming where only objects exist (no classes) and new objects are created by cloning existing objects and customizing them.

To bypass the discussion on how to create new signal value types (documented in whatos.core) we have imported packet from whatos.embedded, a python module containing useful tasks for embedded system design. (Actually, whatos.embedded already contains the Python and C tasks for SLIP communication which we are developing in this tutorial.) wuint8 on the other hand, along with other simple integer and floating point types are standard types found in whatos.core.

Attribute process is where the state machine executes. This implementation of pysliptx is not much of a state machine at all, since it emits all bytes in one shot and doesn't change state. csliprx is a more typical state machine example. It changes state when receiving bytes. In pysliptx, method process is called with input = signal and inputval = value of signal; process is implemented directly in Python. In csliprx, process is a string containing the C code which gets executed when an input, inputval are received. accept() and reject() are exit points from this block of code, indicating that the signal was accepted or rejected. These two macros, along with emit(), await(), require(), acquire() and a few others, are defined by the source code generator (see C API internal task interface).

Let's make our system containing these two tasks. When connections between tasks are made, all task inputs and outputs with the same name are connected together by signals. Since pysliptx emits outbyte and csliprx received inbyte, we rename csliprx.inbyte to outbyte.

Feature: Customize interfaces of tasks to match together.

from slip1 import *

# clone system and customize
class system(system):
    # code to go in the header file (wossystem.h)
    header = """
    #define WOS_DBG_SIM          /* enable whatos simulation */
    #define WOS_ASSERT_INTERNALS /* enable catching internal errors */
    """
    # tasks
    pysliptx = pysliptx()
    # clone and customize csliprx, renaming inbyte to outbyte
    csliprx = csliprx(inbyte = csliprx.inbyte('outbyte'))

# specify number of units of packet and wos_Signal to allocate
system.wos_Signal.allocate = 10         # signal queue length = 10
system.packet.allocate = 2

Let's use this system in a Python interactive session. First we build the dynamic link library containing the C portion of the system (csliptx, memory allocator, scheduler, etc...), then we load this dll, and then we simulate by emitting signals. Instead of invoking the Python interpreter with a command like "python" we invoke "wospython". This is a script (bin/wospython) created by WhatOS during the installation which takes care of setting environment variables (such as PYTHONPATH, LD_LIBRARY_PATH, etc.) and then invokes "python":

$ wospython
>>> from system1 import *
>>> system.dllbuild()
>>> system.dllload()
>>> system.inpacket.emit([1, 2, 3])
inpacket([1, 2, 3]) -> pysliptx
outbyte(192) -> csliprx
outbyte(1) -> csliprx
outbyte(2) -> csliprx
outbyte(3) -> csliprx
outbyte(192) -> csliprx
outpacket([1, 2, 3]) -> environment

Reminder: Invoking wospython instead of invoking Python directly saves you the pain of setting environment variables and mistakenly calling the wrong Python version. Call wospython with --interactive option if using Tkinter applications; when invoked like this, wospython uses a separate thread to execute Tkinter.mainloop(), which does not block the console from executing commands interactively.

Feature: Develop and test interactively using Python.

Notice that csliprx has correctly emitted the right packet ([1, 2, 3]). This is one test case which we know is correct. After applying many more test cases like this one, we can save the output (cut and paste) and write a script which will automatically retest pysliptx and csliprx whenever we want (e.g. see Python's standard module doctest). Although most of the time it is impossible to write an exhaustive set of test cases, the more test cases you have the better the confidence in the correctness of the system.

Feature: Use Python to automate testing of written and verified code.

That covers the creation and simulation of a complete system.

Using the generated C files in a stand-alone/embedded application

Until now we have used WhatOS generated C files in the background by creating and using a Python extension module (dll) for simulation. Here we'll use the generated files wossystem.h and wossystem.c to create a stand-alone command-line application. This same procedure applies equally well when using the generated C files in an embedded system application; the generated C files contain a complete operating system including signal routing, priority scheduling and memory management. Hello WhatOS contains a complete demo session with an embedded system.

This sample stand-alone application, called sliprx, is a command-line application which receives a set of bytes (passed on the command line) and prints out a packet if those bytes form a complete packet; if the bytes do not form a complete packet it prints nothing.

from slip1 import *

class system(system):
    # code to go in the header file (wossystem.h)
    # define a hook to execute when outpacket is emitted
    header = """
    void print_packet(packet * p);
    #define hook_outpacket(src, val) print_packet((packet *) val)
    """
    pysliptx = pysliptx()
    csliprx = csliprx(inbyte = csliprx.inbyte('outbyte'))

system.wos_Signal.allocate = 10
system.packet.allocate = 2

def gencode():
    'Generate source code: wossystem.h and wossystem.c.'
    open('wossystem.h', 'w').write(system.genh())
    open('wossystem.c', 'w').write(system.genc())

import sys
if (__name__ == '__main__') and (len(sys.argv) > 1):
    exec sys.argv[1] + '()'

Notice that it's ok to have Python tasks in the system; they will be ignored during the generation of the C source code files. We define the macro hook_outpacket to be executed whenever a task emits the signal outpacket. Using system3.py we generate wossystem.h and wossystem.c with the following command:

$ wospython system3.py gencode

#include <stdlib.h>
#include <stdio.h>
#include "wossystem.h"

void
print_packet(packet * p)
{
    int i;
    printf("[");
    for (i = 0; i < p->len; i++) {
        if (i) printf(", ");
        printf("%i", (p->data[i]));
    }
    printf("]\n");
}

int
main(int argc, char* argv[])
{
    int i;
    /* initialize the system */
    wos_init();
    for (i = 1; i < argc; i++) {
        /* put the byte in the signal queue */
        wos_emit(wos_sig_outbyte, (void *) atoi(argv[i]), WOS_ENVIRONMENT);
    }
    /* process the signal queue, returns when queue is empty */
    wos_process(0);
    return(0);
}

In sliprx.c we use wossystem.h and wossystem.c by adhering to the C API external interface. Let's build the executable and try it out:

$ cc -o sliprx sliprx.c wossystem.c
$ ./sliprx 1 2 3
$ ./sliprx 192 1 2 3 192
[1, 2, 3]

In the first case we get no output because the sequence of bytes 1, 2, 3 do not define a packet. In the second case we get the desired output packet [1, 2, 3].

This simple example did not visibly exploit the full embedded system features like scheduling. In an interrupt driven embedded system application, parallel execution of tasks is needed for real-time applications and that is where this functionality can be fully appreciated.

Feature: Generated C files contain complete stand-alone operating system environment.