# Testing
[NB: The current content is aspirational.  Not fully implemented yet.]

## Overview

The core purpose of DAWN is the processing of information relating
to the set of access points (AP) and clients (STA - meaning stations in 802.11
vernacular) that form the wi-fi network under DAWN's management.

The required data storage and processing capabilites are generally held in the 'storage'
part of DAWN's source code tree.  The remaining parts are mainly for
interfacing with resources of the AP environment rhat provide updated information and allow
DAWN to indicate to stations what they should do next following processing of the latest data.
Specifically, this means components such as iwinfo, ubus and uci that are commonly found on an
AP running OpenWRT variant of Linux.

## Testing Approach
The principal focus of DAWN's test harness is on these storage and processing components.  This is achieved by
having a build target named test_storage which builds these parts
without the environment interaction dependencies.  This "test harness" can then be executed on the
build host (e.g. a Linux desktop development environment) or on the target AP.

To configure the imaginary AP (SUT, or "system under test" in testing parlance)
a script that mimics many of the messages from ubus and other sources is created to represent
the simulated network by describing APs, connected STAs and DAWN's own configuration parameters.
The data evaluation algorithms are then executed, and resultant outputs to stations are simulated as
text output.

For example consider a simple network of two AP with 2 stations.  The following (simplifed) script is
used to configure what one AP will be aware of:

    CONFIG RSSI=10db
    SELF 01:01:01:01:01:01
    AP 02:02:02:02:02:02
    CONNECT 99:99:99:99:99:99 01:01:01:01:01:01 -78dB
    HEARING 99:99:99:99:99:99 02:02:02:02:02:02 -65dB
    CONNECT 88:88:88:88:88:88 01:01:01:01:01:01 -65dB
    HEARING 88:88:88:88:88:88 02:02:02:02:02:02 -65dB
    KICK CONSUME
    KICK

This means our test AP has the BSSID 01:..., and there is another AP in the network with BSSID 02:.... 
Two stations with MAC 99:... and 88:... are in the network, both connected to AP01:... but also
able to see AP02:....  The dB values indicate RSSI levels, and will be evalutated to determine if stations
are connected to an appropriate AP.  We'ed also configured
AP01:... to have an RSSI transition threshold of 10dB.  When "kicking evaluation" is performed
STA99:... can improve its RSSI by over 10dB by switching to AP02:..., so will be instructed to do so,
resulting in the test actions:

    REMOVE  99:99:99:99:99:99 01:01:01:01:01:01
    CONNECT 99:99:99:99:99:99 02:02:02:02:02:02 -65dB

Note that this is also valid input to the test_harness, and the parameter CONSUME
on the KICK action will cause it to be reinjested for consideration when the
second KICK action is evaluated.
	
## Types of Testing
Three main areas of testing are performed by the supplied test scripts:
* Data management: Ensuring data is stored correctly and soes not cause buffer overruns, etc
* Algorithm functionality: Review the outcomes of evaluation for somple and complex network data
* Scalability: Evaluate the ability of DAWN to sacle linearly to hundreds and thousands of AP and STA

A number of scenarios are defined for each type of testing, along with scripts to execute them.

### Data Management Scenarios
Data management scenarios excercise DAWN's internal data structures by filling and emptying them to ensure no
overflow conditions occur, and where appropriate that sorting is applied correctly.  They are independent
of any functional testing, so each data structure is excercised alone in a way that would never occur in
real usage.

#### Test DM001: AP list
Fill, print, empty and print

#### Test DM002: Client list
Fill, print, empty and print

#### Test DM001: MAC list
Fill, print, empty and print

#### Test DM001: Hearing list
Fill, print, empty and print

### Algorithm Scenarios
Algorithm scenarios are used to ensure that DAWN creates the intended outputs to manage which stations
connect to APs.  They require more intricate crafting of the scripted synthetic data to represent the
input to DAWN's evaluation algortithms to blend situation data with decision metrics.

#### Test AL001: 2+2 Stable
Two AP with two stations connected to AP1, and will remain there.

#### Test AL002: 2+2 Cross
Two AP with two stations connected to AP1, both switching to AP2.

#### Test AL003: 2+1+1 Cross
Two AP with one station connected each, and will cross to the other.

#### Test AL003: Load balance
Three AP with 40 stations connected to AP1, all with same metrics.  DAWN should balance load to other APs.


### Scalability Scenarios
Scalability scenarios explore how well DAWN can function in environments with many APs and clients.
Using commodity equipment this might mean tens of APs with a total of a thousand or so connected stations.
Subject to use of appropriate hardware (eg small PC rather than SoC platform) there is an aspiration for
DAWN to be stable while managing hundreds of APs each with tens of connected stations, allowing networks
to support 10,000 or more concurrent stations.

Scalability requires a number of design aspects to function together:
* Data storage scalability: To allow efficient use of memory the amount required should be approximately linear, so simple networks on small devices require small amounts of memory while more complex environments can function on moderately scaled hardware.
* Algorithm scalability: Similar to memory usage, ensuring that calculation times are more linear than exponential as numbers of APs and stations grow.
* Algorithm efficiency: As well as being scaleable the processing of data must fit within absolute limits to ensure that APs can keep up with the rate of data arriving from the network.
