How to extend the built-in function set

We are going to show, step by step, how to add a new built-in function to nuBASIC native API.
Just for example let us assume we want to add a 1d-convolution math function which returns a convolution of two vectors. If they are vectors of polynomial coefficients, convolving them is equivalent to multiplying the two polynomials.
For example, let's image to have two vectors u=(1,0,1) and v=(2,7) representing the coefficients of the polynomials x2+1 and 2x+7, in order to convolve them we want to use a nuBASIC program which employees a new function Conv as shown in the following example:

Dim u(3) as Double
Dim v(2) as Double
Dim w(4) as Double

u(0)=1
u(1)=0
u(2)=1

v(0)=2
v(1)=7

w = Conv(u,v)

For i=0 To 3
  Print w(i);" ";
Next i


and the expected output is the following:
2 7 2 7

which means the vector w contains the polynomial coefficients for the polynomial 2x3+7x2+2x+7.

nuBASIC Conv prototype

Such function Conv will have a prototype defined as follows:

function conv( v1(n) as Double, v2(m) as Double) as Double(n+m-1)

We could also have an additional two (optional) parameters representing the size of vectors in order to allow to reuse the same array for representing different vectors.
In other words supporting a prototype like the following:


 function conv( v1(arraySize1) as Double, v2(arraySize2) as Double, n as Integer, m as Integer) as Double(n+m-1)


Where parameters arraySize1>=n>0 and arraySize2>=m>0.

Implementation of conv function in C++

A possible implementation in C++ of conv function could be the following:

template<typename T>
std::vector<T> conv(const std::vector<T> &v1, const std::vector<T> &v2) {

    const int n = int(v1.size());
    const int m = int(v2.size());
    const int k = n + m - 1;

    std::vector<T> w(k, T());

    for (auto i=0; i < k; ++i) {
        const int jmn = (i >= m - 1) ? i - (m - 1) : 0;
        const int jmx = (i < n - 1) ? i : n - 1;

        for (auto j=jmn; j <= jmx; ++j) {
            w[i] += (v1[j] * v2[i - j]);
        }
    }

    return w;
}

We don't need to analyse in detail such function. It basically returns a new vector which is the convolution of two given vectors v1 and v2.
We can just assume it works fine for our purpose.

Extending the global function set

In general, to add a function to existing built-in API set we need to modify lib/nu_global_function_tbl.cc, and more in detail the static function of class global_function_tbl_t.

global_function_tbl_t& global_function_tbl_t::get_instance()

First time the static class function get_instance() is executed, populates a global map, which associates to each nuBASIC function name to a C++ functor (in other words a callback function) having the follow prototype:

variant_t functor_name( rt_prog_ctx_t& ctx, const std::string& name, const nu::func_args_t& args );

Such functor takes as parameters: 
  • a runtime context (ctx) as defined in include/nu_rt_prog_ctx.h
  • a function name (take into account that multiple names could be mapped to the same functor) and 
  • a vector of arguments, each of them is a nuBASIC expression.
A generic implementation of such functor requires to:
  • Get the number of input arguments (args size) and validate it.
  • Convert and validate any input args into C++ equivalent objects.
  • Call the C++ function for doing the actual job (that could be implemented inline as well).
  • Convert the result into nu::variant_t object, which is how at the end nuBASIC interpreter represent its data.
  • Return the result.

Defining new functor conv

To proceed creating a conv_functor we need to modify the file lib/nu_global_function_tbl.cc. A skeleton of conv_functor could be the following:

variant_t conv_functor(
    rt_prog_ctx_t& ctx,
    const std::string& name,
    const nu::func_args_t& args) 
{
    // Get number of arguments
    // TODO

       // Validate the number of arguments
    // TODO

       // Process and validate the arguments
    // TODO 
    
       // Convert the input parameters into C++ parameters
    // TODO

       // Compute the result
    // TODO

       // Convert the result into nu::variant_t object
    // TODO

    return result;
}


Get and validate the number of input arguments

According the two nuBASIC prototypes of Conv, we want to implement a function which accepts either 2 or 4 arguments. 
So, first implementation step is to get the number of the arguments, which means we need to inspect the args parameter size.
We also need to check if such size is valid.
Knowing args is a vector of nuBASIC expressions (see also include/nu_expr_any.h), its size() method will return the number of arguments of the nuBASIC function.

variant_t conv_functor(
   rt_prog_ctx_t& ctx,
   const std::string& name,
   const nu::func_args_t& args) 
{
       // Get number of arguments
    const auto args_num = args.size();

       // Validate the number of arguments
    // TODO

       // Process and validate the arguments
    // TODO

       // Convert the input parameters into C++ parameters
    // TODO

       // Compute the result
    // TODO

       // Convert the result into nu::variant_t object
    // TODO

    return result;
}

To validate the number of arguments we have to check if it is 2 or 4, otherwise we have to generate a run-time error.
For generating an error we can call the method throw_if() of the rt_error_code_t singleton object which generates an error depending on a specific error code.


variant_t conv_functor(
   rt_prog_ctx_t& ctx,
   const std::string& name,
   const nu::func_args_t& args) 
{
       // Get number of arguments
    const auto args_num = args.size();

    // Validate the number of arguments
    rt_error_code_t::get_instance().throw_if(
        args_num != 4 && args_num != 2, 0, rt_error_code_t::E_INVALID_ARGS, "");

    // Process and validate the arguments
    // TODO

       // Convert the input parameters into C++ parameters
    // TODO

       // Compute the result
    // TODO

       // Convert the result into nu::variant_t object
    // TODO

    return result;
}

In the previous source code throw_if() generates a runtime error by throwing an exception rt_error_code_t::E_INVALID_ARGS whenever the predicate args_num != 2 and args_num != 4 is true.
See the rt_error_code_t class implementation (lib/nu_error_codes.cc) for more information on run-time error codes and related messages.

Converting the input args into C++ equivalent parameters

Because we have validated the number of nuBASIC function arguments which is at least 2, we can now pick the first two of them up.
To do that we have to get the the first two element of vector args (which are shared pointers to instance of nuBASIC expression) and call the method eval() against the runtime context related to executing nuBASIC function ctx, as shown in the following source code:

variant_t conv_functor(
   rt_prog_ctx_t& ctx,
   const std::string& name,
   const nu::func_args_t& args) 
{
    // Get the number of input arguments
    const auto args_num = args.size();

    // Validate the input arguments (args).
    rt_error_code_t::get_instance().throw_if(
       args_num != 4 && args_num != 2, 0, rt_error_code_t::E_INVALID_ARGS, "");

    // Process and validate the arguments
    auto variant_v1 = args[0]->eval(ctx);
    auto variant_v2 = args[1]->eval(ctx);
    // ...
       // TODO

    // Call the C++ function conv for processing the input data and doing the actual job.
    // TODO

    // Get the result and convert it into nu::variant_t which is how internally the result is represented.
       // TODO

    return result;
}

Note that the execution order matters for respecting the original semantics which evaluates expression arguments from left to right, and the arguments are listed from left to right using the lowest index 0 to the highest index which is given by the number of arguments - 1. We have to evaluate the arguments respecting such order, otherwise we could violate the semantics introducing unexpected behaviours. Each expression, in fact, could execute functions with side effect on the global context, and the order of such side effects must be preserved.

As result of the previous two highlighted statements we get two variant objects that we can assume to be vector of numbers (no matters if Integer, Float, Double, ... etc, because all of them have a safe conversion into Double). 
Checking the method vector_size() of a variant object we can verify if they actually represent vectors.

Processing and validation of all the input arguments

Now we can proceed to check the actual vectors size. If we have just two parameters we can assume the vector size is coincident with size of array.
If the number of arguments is 4, we have to process the additional two arguments assuming them integer numbers.
We have to generate a runtime error whenever the first two arguments are not vectors, or in case any additional two arguments are invalid because greater than the array size.

In other words, we can implement something like the following statements:

variant_t conv_functor(
    rt_prog_ctx_t& ctx,
    const std::string& name,
    const nu::func_args_t& args) 
{
    // Get number of arguments
    const auto args_num = args.size();

       // Validate the number of arguments
    rt_error_code_t::get_instance().throw_if(
        args_num != 4 && args_num != 2, 0, rt_error_code_t::E_INVALID_ARGS, "");

       // Process and validate the arguments
    auto variant_v1 = args[0]->eval(ctx);
    auto variant_v2 = args[1]->eval(ctx);

       const auto actual_v1_size = variant_v1.vector_size();
    const auto actual_v2_size = variant_v2.vector_size();

    const size_t size_v1 =
       args_num == 4 ? size_t(args[2]->eval(ctx).to_long64()) : actual_v1_size;

    const size_t size_v2 =
       args_num == 4 ? size_t(args[3]->eval(ctx).to_long64()) : actual_v2_size;

    rt_error_code_t::get_instance().throw_if(
       size_v1 > actual_v1_size || size_v1<1, 0, rt_error_code_t::E_INV_VECT_SIZE, args[0]->name());

    rt_error_code_t::get_instance().throw_if(
       size_v2 > actual_v2_size || size_v2<1, 0, rt_error_code_t::E_INV_VECT_SIZE, args[1]->name());

       // Convert the input parameters into C++ parameters
    // TODO

       // Compute the result
    // TODO

       // Convert the result into nu::variant_t object
    // TODO

    return result;
}


Calculate the result and convert it into nu::variant_t which is how internally the result is represented.

At this point, in absence of exceptions, we will have two variant objects, the input vectors, and two respective sizes: size_v1 and size_v2.
We need to transform them into std::vector<double>, and we can proceed as follows:

std::vector<double> v1(size_v1);
bool ok = variant_v1.copy_vector_content(v1);

rt_error_code_t::get_instance().throw_if(
   !ok, 0, rt_error_code_t::E_INV_VECT_SIZE, args[0]->name());

std::vector<double> v2(size_v2);
ok = variant_v2.copy_vector_content(v2);

rt_error_code_t::get_instance().throw_if(
   !ok, 0, rt_error_code_t::E_INV_VECT_SIZE, args[1]->name());

v1 and v2 represent C++ vectors of double (initialised to 0).
The method copy_vector_content() will copy the variant_v1 content into a vector of double precision floating point numbers.
If the conversion is impossible the method would fail and a runtime error will be generated as result.

If we don't have any error at this point, we can call the C++ function conv which does the actual work:

auto vr = conv(v1, v2);

Then we can convert the result into a variant (internally typed as a vector of Double)

nu::variant_t result(std::move(vr));

Finally, we can return the result:

return result;


And putting all together:
variant_t conv_functor(
    rt_prog_ctx_t& ctx,
    const std::string& name,
    const nu::func_args_t& args) 
{
    // Get number of arguments
    const auto args_num = args.size();

    // Validate the number of arguments
    rt_error_code_t::get_instance().throw_if(
       args_num != 4 && args_num != 2, 0, rt_error_code_t::E_INVALID_ARGS, "");

    // Process and validate the arguments
    auto variant_v1 = args[0]->eval(ctx);
    auto variant_v2 = args[1]->eval(ctx);

    const auto actual_v1_size = variant_v1.vector_size();
    const auto actual_v2_size = variant_v2.vector_size();

    const size_t size_v1 =
       args_num == 4 ? size_t(args[2]->eval(ctx).to_long64()) : actual_v1_size;

    const size_t size_v2 =
       args_num == 4 ? size_t(args[3]->eval(ctx).to_long64()) : actual_v2_size;

    rt_error_code_t::get_instance().throw_if(
       size_v1 > actual_v1_size || size_v1<1, 0, rt_error_code_t::E_INV_VECT_SIZE, args[0]->name());

    rt_error_code_t::get_instance().throw_if(
       size_v2 > actual_v2_size || size_v2<1, 0, rt_error_code_t::E_INV_VECT_SIZE, args[1]->name());

    // Convert the input parameters into C++ parameters
    std::vector<double> v1(size_v1);
    std::vector<double> v2(size_v2);

    bool ok = variant_v1.copy_vector_content(v1);

    rt_error_code_t::get_instance().throw_if(
       !ok, 0, rt_error_code_t::E_INV_VECT_SIZE, args[0]->name());


    ok = variant_v2.copy_vector_content(v2);

    rt_error_code_t::get_instance().throw_if(
       !ok, 0, rt_error_code_t::E_INV_VECT_SIZE, args[1]->name());

    // Compute the result
    auto vr = conv(v1, v2);

    // Convert the result into nu::variant_t object
    nu::variant_t result(std::move(vr));

    return result;
}


Still we have to map the function inserting the functor into the fmap defined inside the function global_function_tbl_t::get_instance() which is declared in the file nu_builtin_help.cc:
fmap["conv"] = conv_functor;

To complete our integration we can extend the console inline help and the list of reserved words.
To extend the inline help we need to add a specific entry to the _help_content collection, as follows:

static help_content_t _help_content[] = {

// ... 

{ lang_item_t::FUNCTION, "conv",
        "Returns a vector of Double as result of convolution 2 given vectors of numbers",
        "Conv( v1, v2 [, count1, count2 ] ))" },

// ...
};


That will allow to use help and apropos commands from nuBASIC console to get information about the new function conv.

Final step is extending the list of reserved words by modifying the file nu_reserved_keywords.cc which defines the set of strings named reserved_keywords_t::list:

const std::set<std::string> reserved_keywords_t::list = {
    // ...
   "conv", 
    //
};