AllocArrays

struct AllocArray{T,N,A<:AbstractArray{T,N}} <: AbstractArray{T,N}
    arr::A
end

AllocArray(arr::AbstractArray)

Wrapper type which forwards most array methods to the inner array arr, but dispatches similar to special allocation methods.

Use the constructor AllocArray(arr) to construct an AllocArray.

Typically this constructor is only used at the entrypoint of a larger set of code which is expected to use similar based on this input for further allocations. When inside a with_allocator block, similar can be dispatched to a (dynamically-scoped) bump allocator.

CheckedAllocArray(arr::AbstractArray)

"Slow but safe" version of AllocArray.

Keeps track of whether or not its memory is valid. All accesses to the array first check if the memory is still valid, and throw an InvalidMemoryException if not.

If the array has memory allocated via a BumperAllocator, when the BumperAllocator is reset via reset!, its memory will be marked invalid.

Uses locks to ensure concurrency safety to avoid races between deallocating memory on one task (with reset!) while accessing it on another task (e.g. getindex). However, this array is as unsafe as any other to write and read its contents simultaneously with multiple tasks. (i.e. locks are used only to ensure validity of the memory backing the array when the memory is accessed, not to remove data races when using the array as usual).

See also: AllocArray.

BumperAllocator(b)

Use with with_allocator to dispatch similar calls for AllocArrays and CheckedAllocArrays to allocate using the buffer b, an AllocBuffer provided by Bumper.jl.

Uses a lock to serialize allocations to the buffer b, which should allow safe concurrent usage.

Used with reset! to deallocate. Note it is not safe to deallocate while another task may be allocating, except with CheckedAllocArrays which will error appropriately.

See also: UncheckedBumperAllocator.

Example

using AllocArrays, Bumper

b = BumperAllocator(AllocBuffer(2^24)) # 16 MiB
input = AllocArray([1,2,3])
c = Channel(Inf)
with_allocator(b) do
    # ...code with may be multithreaded but which must not escape or return newly-allocated AllocArrays...
    @sync for i = 1:10
        Threads.@spawn put!(c, sum(input .+ i))
    end
    reset!(b) # called outside of threaded region
    close(c)
end
sum(collect(c))

# output
225

with_allocator(f, allocator)

Run f within a dynamic scope such that similar calls to AllocArrays and CheckedAllocArrays dispatch to allocator allocator.

Used with allocators DefaultAllocator, BumperAllocator, and UncheckedBumperAllocator.

reset!(B::UncheckedBumperAllocator)

Resets the UncheckedBumperAllocator, deallocating all of the arrays created by it.

This must only be used if those arrays will not be accessed again.

It is not safe to deallocate on one task while using the allocator to allocate on another task. Therefore this should only be called outside of threaded regions of code.

reset!(b::BumperAllocator)

Resets the BumperAllocator, deallocating all of the arrays created by it.

This must only be used if those arrays will not be accessed again. However, CheckedAllocArrays allocated by this allocator will be marked invalid, causing future accesses to them to error, as a safety feature. AllocArrays have no such safety feature, and access to them after reset! is unsafe.

It is also not safe to deallocate on one task while using the allocator to allocate on another task. Therefore this should only be called outside of threaded regions of code.

UncheckedBumperAllocator(b)

Use with with_allocator to dispatch similar calls for AllocArrays to allocate using the buffer b, an AllocBuffer provided by Bumper.jl.

Does not support CheckedAllocArray.

This provides a naive & direct interface to allocating on the buffer with no safety checks or locks.

This is unsafe to use if multiple tasks may be allocating simultaneously, and using BumperAllocator is recommended in general.

Used with reset! to deallocate.

See also: BumperAllocator.

Example

using AllocArrays, Bumper

input = AllocArray([1,2,3])
b = UncheckedBumperAllocator(AllocBuffer(2^24)) # 16 MiB
with_allocator(b) do
    # ...code with must not allocate AllocArrays on multiple tasks via `similar` nor escape or return newly-allocated AllocArrays...
    ret = sum(input .* 2)
    reset!(b)
    return ret
end

# output
12

DefaultAllocator()

Represents the default Julia allocator.

Used to dispatch similar calls for AllocArrays and CheckedAllocArrays to allocate using the the default Julia allocator.

This allocator is used by default if another one is not used via with_allocator.

AutoscalingAllocBuffer(initial_buffer_size::Int=134217728;
                       max_history_size=5)

Construct a AutoscalingAllocBuffer. This constructs an AllocBuffer of size initial_buffer_size. If an allocation (of some size allocation_size) is requested and there is not enough space, it will allocate a new buffer (whose size is determined by internal heuristics).

When reset_buffer! is called, if there are additional buffers, a new larger main buffer will be created so that if similar sized allocations occur on the next run no additional buffers will be needed. It will remember the amount of allocations used in the past max_history_size runs to inform the size of the new main buffer, to ensure there is at least enough room for each of those past runs.

This means:

• AutoscalingAllocBuffer does not run out of memory (unlike AllocBuffer), since new memory will allocated on-demand when necessary
• for repeated runs of the same size, a single contiguous buffer will be used, which should approximately match the performance of a tuned AllocBuffer
• AutoscalingAllocBuffer can reuse allocated memory between runs like AllocBuffer, but with safety from overrunning a fixed buffer size and OOMing like SlabBuffer. Additionally, AutoscalingAllocBuffer separately tracks the memory used by the main buffer vs the additional buffers allocated dynamically, so small unexpected additional allocations don't double the memory consumption (unlike a second slab being allocated).