joinAppend

joinAppend performs a join operation on an input range, appending the results to an output range.

joinAppend was written as a performance enhancement over using std.algorithm.joiner or std.array.join with writeln. Using joiner with writeln is quite slow, 3-4x slower than std.array.join with writeln. The joiner performance may be due to interaction with writeln, this was not investigated. Using joiner with stdout.lockingTextWriter is better, but still substantially slower than join. Using join works reasonably well, but is allocating memory unnecessarily.

Using joinAppend with Appender is a bit faster than join, and allocates less memory. The Appender re-uses the underlying data buffer, saving memory. The example below illustrates. It is a modification of the InputFieldReordering example. The role Appender plus joinAppend are playing is to buffer the output. BufferedOutputRange uses a similar technique to buffer multiple lines.

Note: The original uses joinAppend have been replaced by BufferedOutputRange, which has its own joinAppend method. However, joinAppend remains useful when constructing internal buffers where BufferedOutputRange is not appropriate.

int main(string[] args)
{
    import tsvutil;
    import std.algorithm, std.array, std.range, std.stdio;
    size_t[] fieldIndicies = [3, 0, 2];
    auto fieldReordering = new InputFieldReordering!char(fieldIndicies);
    auto outputBuffer = appender!(char[]);
    foreach (line; stdin.byLine)
    {
        fieldReordering.initNewLine;
        foreach(fieldIndex, fieldValue; line.splitter('\t').enumerate)
        {
            fieldReordering.processNextField(fieldIndex, fieldValue);
            if (fieldReordering.allFieldsFilled) break;
        }
        if (fieldReordering.allFieldsFilled)
        {
            outputBuffer.clear;
            writeln(fieldReordering.outputFields.joinAppend(outputBuffer, ('\t')));
        }
        else
        {
            writeln("Error: Insufficient number of field on the line.");
        }
    }
    return 0;
}