Joka

A nogc utility library for the D programming language. Joka provides data structures and functions that can work without garbage collection, offering precise memory control. It is designed for C-style memory management and as a lightweight complement to the D standard library.

/// Arrays, printing, and string interpolation.
import joka;

void main() {
    auto numbers = List!int(4, 6, 8);
    scope (exit) numbers.free();
    foreach (i, number; numbers) {
        println(i"[$(i)]: $(number)");
    }
}

Why Joka

• Minimalistic: Avoids many abstractions
• Modular: Has minimal cross-module dependencies
• Simple: Uses basic structs for everything
• BetterC: Fully compatible via -betterC -i
• WebAssembly: Lightweight enough to run on a single 64KB page

Performance Benchmark

Here's a comparison of Joka's dynamic array versus other popular libraries when appending and removing 50,000,000 integers on a Ryzen 3 2200G running Ubuntu, with 16 GB of memory:

Append 50000000 items with `int[]`: 1116 ms
Remove 50000000 items with `int[]`: 130 ms
Append 50000000 items with `Array!int`: 264 ms
Remove 50000000 items with `Array!int`: 0 ms
Append 50000000 items with `Appender!int`: 244 ms
Remove 50000000 items with `Appender!int`: 0 ms
Append 50000000 items with `nulib`: 777 ms
Remove 50000000 items with `nulib`: 212 ms
Append 50000000 items with `emsi`: 352 ms
Remove 50000000 items with `emsi`: 123 ms
Append 50000000 items with `memutils`: 168 ms
Remove 50000000 items with `memutils`: 0 ms
Append 50000000 items with `automem`: 333 ms
Remove 50000000 items with `automem`: 0 ms
Append 50000000 items with `joka`: 167 ms
Remove 50000000 items with `joka`: 0 ms

Below are also some high-level cross-language results using a similar workload. These are not direct benchmarks and are intended only as a point of reference:

Testing: ./app_d
real 0.09
user 0.02
sys 0.06
Testing: ./app_rs
real 0.09
user 0.03
sys 0.06
Testing: ./app_zig
real 0.09
user 0.03
sys 0.06
Testing: ./app_odin
real 0.14
user 0.05
sys 0.08

Note

The project is still early in development. If something is missing, it will probably be added when someone (usually the main developer) needs it. NuMem or NuLib are good alternatives.

Quick Start

This guide shows how to install Joka using DUB. Create a new folder and run inside the following commands:

dub init -n
dub add joka

That's it. Copy-paste one of the examples to make sure everything is working correctly.

Documentation

Start with the examples folder for a quick overview.

Modules

• joka.io: Input and output procedures
• joka.math: Mathematics
• joka.memory: Memory utilities and containers
• joka.ranges: Range utilities
• joka.types: Common type definitions and ASCII strings
• joka.stdc: C standard library functions
• joka.wasip1: WebAssembly System Interface Preview 1 helpers
• joka.wasm4: WASM-4 fantasy console interface
• joka.wit: Wasm Interface Type helpers

Versions

• JokaCustomMemory: Allows the declaration of custom allocation functions.
• JokaGcMemory: Like JokaCustomMemory, but preconfigured to use the D garbage collector.
• JokaSmallFootprint: Uses less memory for some static buffers in Joka.
• JokaNoTypes: Removes the dependency on types.d from some modules and uses internal stubs instead.
• JokaTypesStubs: Removes the string.h dependency from the types.d module and uses internal stubs instead. Useful when working with WebAssembly.
• JokaMemoryStubs: Removes the stdlib.h dependency from the memory.d module and uses internal stubs instead. Useful when working with WebAssembly.
• JokaMathStubs: Removes the math.h dependency from the math.d module and uses internal stubs instead. Useful when working with WebAssembly.
• JokaRuntimeSymbols: Allows defining some required runtime symbols when they are missing. Currently only useful when working with Windows + -betterC. Might get removed in new Joka releases.

Types

• ForeignSlice: Non-D slice
• Fault: Error code
• Maybe: Optional value + error code
• Option: Optional value (generally prefer Maybe)
• Result: Success or error value (generally prefer Maybe)
• Union: Tagged union
• List: Dynamic array
• BufferList: Fixed-buffer array
• FixedList: Stack-based array
• SparseList: Sparse array
• GenList: Generational array
• BitList: Dynamic bit array
• BitSet: Static bit set
• Grid: 2D array
• Arena: Fixed linear allocator
• GrowingArena: Growing linear allocator
• MemoryContext: Allocator API
• NumericRange: Number sequence
• SliceRange: Slice view
• EnumeratedRange: Indexed sequence
• TransformedRange: Map or filter result
• ArgTokenRange: Command-line arguments

WebAssembly

Joka supports WebAssembly via the -betterC -i flags and integrates seamlessly with Emscripten and Wasmtime. For environments without those runtimes, you can still use some of Joka's modules by enabling "stubs." Check the versions section for more information about them.

Below is a WASI hello-world example:

import joka.wasip1;
import joka.types;

extern(C)
void _start() {
    auto value = 40 + 29;
    auto text = "Value is: {}\n".fmt(value).toCiovec();
    fdWrite(stdout, &text, 1, null);
}

Compile and run with Wasmtime:

ldc2 -betterC -i --mtriple=wasm32 --checkaction=halt app.d
wasmtime app.wasm

Additionally, the wasm32-wasi target can be used to enable some versions like JokaMemoryStubs by default and get WASI specific implementations:

import joka.io;

extern(C)
void _start() {
    println("Value is: ", 40 + 29);
}

ldc2 -betterC -i --mtriple=wasm32-wasi app.d
wasmtime app.wasm

A basic WASM-4 template is also available in the scripts folder. To get started, copy the contents of that folder into your project folder. A WASM-4 project should follow this structure:

my_wasm4_project/
├── app.d
├── joka/
└── run

Memory Tracking

Joka includes a lightweight memory tracking system that can detect leaks or invalid frees in debug builds. By default, the helper function memoryTrackingInfo produces output like this:

Memory Leaks: 4 (total 699 bytes, 5 ignored)
  1 leak, 20 bytes, source/app.d:24
  1 leak, 53 bytes, source/app.d:31
  2 leak, 32 bytes, source/app.d:123

The leak summary above can be filtered, showing only leaks with paths containing the filter string. For example, memoryTrackingInfo("app.d") shows only leaks with "app.d" in the path. Specific allocations can be ignored with ignoreLeak like this:

// struct Game { int hp; int mp; }
// Game* game;
game = jokaMake!Game().ignoreLeak();

Allocations can also be grouped to make it easier to understand what each allocation is used for with AllocationGroup like this:

// This can also be done with the `beginAllocationGroup` and `endAllocationGroup` functions.
with (AllocationGroup("World")) {
    allocateMonsters();
    allocateActors();
    with (AllocationGroup("Contents")) {
        allocateItems();
        allocateEvents();
    }
}
allocateText(); // Not part of any group.

You can check whether memory tracking is active with static if (isTrackingMemory), and if it is, you can inspect the current tracking state via _memoryTrackingState. _memoryTrackingState is thread-local, so each thread has its own separate tracking state.

Standalone memory.d

It's possible to just use the memory allocation module without a full dependency on Joka. To do this, copy memory.d and types.d into a project.

Standalone math.d

It's also possible to just use the math module without a full dependency on Joka. Copy math.d and types.d (optional for this module with JokaNoTypes) into a project.

Note

Using JokaNoTypes will change how some functions work. For example, the toStr functions for vectors will return empty strings.

Frequently Asked Questions

Does Joka have an allocator API?

Yes. Look at MemoryContext in memory.d. Joka by default is designed to feel like the C standard library, but many data structures do accept an optional allocator. More about the API will be explained in the next section.

Does Joka have a global context like Jai?

Yes, and it has an intentionally ugly name (__memoryContext) to discourage people from using it. The reason for this is that a global context tends to make low-level APIs fragile. In Joka, it is encouraged to be used only for exceptional cases.

Compared to Jai, Joka's version is only about memory management. Below is some information about it:

struct MemoryContext {
    void* allocatorState;
    AllocatorReallocFunc reallocFunc;

    void* malloc(Sz alignment, Sz size, IStr file, Sz line);
    void* realloc(Sz alignment, void* oldPtr, Sz oldSize, Sz newSize, IStr file, Sz line);
    void  free(Sz alignment, void* oldPtr, Sz oldSize, IStr file, Sz line);
}

alias AllocatorReallocFunc = void* function(void* allocatorState, Sz alignment, void* oldPtr, Sz oldSize, Sz newSize, IStr file, Sz line);

struct ScopedMemoryContext {
    MemoryContext _previousMemoryContext;

    this(MemoryContext newContext);
    this(ref Arena arena);
    this(ref GrowingArena arena);
}

ScopedMemoryContext ScopedDefaultMemoryContext();

void jokaRestoreDefaultAllocatorSetup(ref MemoryContext context);
void jokaRestoreNullAllocatorSetup(ref MemoryContext context);
void jokaEnsureCapture(ref MemoryContext capture);

MemoryContext __memoryContext;

Some types like List keep track of the allocator they are using. The member that has the allocator is usually called a capture. It is recommended to call jokaEnsureCapture on a capture before using it.

The context can also be ignored with the jokaSystem* functions. For example, the GrowingArena type is using jokaSystemMalloc and jokaSystemFree.

Intercepting third-party code

One cited reason for such a system is the ability to intercept third-party code. That is a nice feature, but it's easily abusable. For example, the community around the Odin language relies on context changes for almost everything, even within their own APIs. Calling this "interception" is misleading when it is actually the intended way to use the API.

My recommendation is to avoid this kind of thing if you don't like spaghetti. Of course, this isn't a huge problem if you have full control over your dependencies.

Why aren't some functions @nogc?

Because the D garbage collector can be used to allocate memory with the JokaGcMemory version. The @nogc attribute is just a hint to the compiler, telling it to check that called functions also carry that hint. It can be helpful but not essential for writing GC-free code.

For example, consider this function:

char[] temporaryString() {
    static char[64][32] buffers = void;
    static currentBuffer = 0;

    currentBuffer = (currentBuffer + 1) % buffers.length;
    return buffers[currentBuffer][];
}

This function uses a static buffer to create a temporary string at runtime. It never allocates with the GC, so it is a nogc function in practice, but it is not a @nogc function. If you try to call it from a @nogc function, the compiler will reject it simply because the attribute is missing. What this shows is that attributes in D are not a memory management tool.

For what it's worth, I generally avoid attributes in my own projects unless I'm writing a library. You can also track GC usage using the -vgc flag.

Why are you supporting the D garbage collector?

It's another tool for memory management. Joka normally uses a tracking allocator in debug builds to help identify mistakes, but the JokaGcMemory version exists for people who prioritize safety. This approach is similar to the one used in Fil-C.

Is it hard to mix Joka with other D libraries?

No. Joka doesn't impose arbitrary restrictions on code, so it works smoothly with Phobos or other libraries. Some libraries choose to be @safe, @nogc, or nothrow only, but those are their constraints, not Joka's.

I avoid the "attribute-oriented" style of structuring a project entirely.

Why isn't there a jokaFreeSlice function?

Because slices are meant to be used like arrays, not pointers. They also show up everywhere in D code, meaning it would be far too easy to free the wrong one by accident. Using jokaFree(slice.ptr) avoids that. It makes the unsafe part obvious and helps prevent mistakes. Another benefit is that it's easier to reason about. Joka has only one function that frees memory.

For context, the Odin language has three functions for freeing memory:

• free
• free_all
• delete

It might be hard to tell what each one does just from the name if you are new to Odin. The one that frees memory using slices is delete. It looks like this when used:

main :: proc() {
    buffer: [256]u8    // Create a buffer on the stack.
    slice := buffer[:] // Take a slice from the buffer.
    // ...
    delete(slice)      // Try to free the memory.
}

To sum up, Joka is trying to be simple and safe about this.

What are common -betterC errors?

Using -betterC as a global @nogc attribute. This flag does more than just remove the garbage collector and adds extra checks that can sometimes be overly restrictive. If writing GC-free code is important and compiler assistance is really needed, then add @nogc: at the top of every file.

Another is using -betterC without the -i flag. The combination -betterC -i works in most cases.

Then there are the TypeInfo errors. Search for new in the source code and remove it.

The clasic is using struct[N]. Some parts of the D runtime (_memsetn, ...) are needed when using types like this and they can be missing due to how -betterC works. The solution for static arrays is to implement the missing functions or use a custom static array type (StaticArray in joka.types).

A basic one is string errors. It's common to want to use functions to create strings at compile time, but this gets harder because of some extra checks added by the -betterC flag. Below is a function that creates a string the "normal" way, followed by an alternative that works with the flag:

// Doesn't work with `-betterC`.
// The parameter can come from runtime or compile time.
string createString(string value) {
    return value ~ "_OwO";
}

// Works with `-betterC`.
// The parameter must be known at compile time.
string createString(string value)() {
    return value ~ "_UwU";
}

What is Joka used for?

It's primarily used for Parin, a game engine.

Why use D without the GC?

Because manual memory management is fun!

The end?

Yes.