json/doc/qbk/02_07_allocators.qbk

[/
    Copyright (c) 2019 Vinnie Falco (vinnie.falco@gmail.com)
    Copyright (c) 2020 Krystian Stasiowski (sdkrystian@gmail.com)

    Distributed under the Boost Software License, Version 1.0. (See accompanying
    file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

    Official repository: https://github.com/cppalliance/json
]

[/-----------------------------------------------------------------------------]

[section Allocators]

Here we discuss the various allocator models used in the
C++ standard, followed by an explanation of the model used in
this library and its benefits. Finally we discuss how the library
interoperates with existing code that uses polymorphic allocators.

[note
    In the sections which follow, the aliases
    __memory_resource__ and __polymorphic_allocator__
    refer to either Boost types, or `std` types when
    `BOOST_JSON_STANDALONE` is defined.
]

[/-----------------------------------------------------------------------------]

[section Background]

The first version of allocators in C++ defined the named requirement
__Allocator__, and made each standard container a class template
parameterized on the allocator type. For example, here is the
declaration for __std_vector__:

[snippet_background_1]

The standard allocator is __DefaultConstructible__. To support stateful
allocators, containers provide additional constructor overloads taking
an allocator instance parameter.

[snippet_background_2]

While the system works, it has some usability problems:

* The container must be a class template.
* Parameterizing the allocator on the element type is clumsy.
* The system of allocator traits, especially POCCA and POCMA,
  is complicated and error-prone.

Allocator-based programs which use multiple allocator types
incur a greater number of function template instantiations and
are generally slower to compile because class template function
definitions must be visible at all call sites.

[heading Polymorphic Allocators]

C++17 improves the allocator model by representing the low-level
allocation operation with an abstract interface called __memory_resource__,
which is not parameterized on the element type, and has no traits:

[snippet_background_3]

The class template __polymorphic_allocator__ wraps a __memory_resource__
pointer and meets the requirements of __Allocator__, allowing it to be
used where an allocator is expected. The standard provides type aliases
using the polymorphic allocator for standard containers:

[snippet_background_4]

A polymorphic allocator constructs with a pointer to a memory resource:

[snippet_background_5]

The memory resource is passed by pointer; ownership is not transferred.
The caller is responsible for extending the lifetime of the memory
resource until the last container which is using it goes out of scope,
otherwise the behavior is undefined. Sometimes this is the correct model,
such as in this example which uses a monotonic resource constructed from
a local stack buffer:

[snippet_background_6]

However, sometimes shared ownership is needed. Specifically, that the
lifetime extension of the memory resource should be automatic. For
example, if a library wants to return a container which owns an
instance of the library's custom memory resource as shown below:

[snippet_background_7]

This can be worked around by declaring the container to use a custom
allocator (perhaps using a `std::shared_ptr< memory_resource >` as a
data member). This hinders library composition; every library now
exports unique, incompatbile container types. A raw memory resource
pointer is easy to misuse:

[snippet_background_8]

Workarounds for this problem are worse than the problem itself. The library
could return a pair with the vector and `unique_ptr<memory_resource>`
which the caller must manage. Or the library could change its function
signatures to accept a `memory_resource*` provided by the caller, where
the library also makes public the desired memory resources
(`my_resource` above).

[heading Solution]

It is desired to create a single type `T` with the following properties:

* `T` is not a class template
* `T` references a __memory_resource__
* `T` supports both shared ownership, and non-ownership
* `T` interoperates with code already using `std::pmr`

The __storage_ptr__ used in Boost.JSON builds and improves
upon C++17's memory allocation interfaces, accomplishing the
goals above. As a result, libraries which use this type compose
more easily and enjoy faster compilation, as container function
definitions can be out-of-line.

[endsect]

[/-----------------------------------------------------------------------------]

[section:storage_ptr The __storage_ptr__]

Variable-length containers in this library all use dynamically allocated
memory to store their contents. Callers can gain control over the strategy
used for allocation by specifying a __storage_ptr__ in select constructors
and function parameter lists. A __storage_ptr__ has these properties:

* A storage pointer always points to a valid,
  type-erased __memory_resource__.

* Default-constructed storage pointers reference the
  ['default resource], an implementation-defined instance
  which always uses the equivalent of global operator new
  and delete.

* Storage pointers constructed from a
  [link json.ref.boost__json__memory_resource `memory_resource*`]
  or __polymorphic_allocator__ do not acquire ownership; the
  caller is responsible for ensuring that the lifetime of
  the resource extends until it is no longer referenced.

* A storage pointer obtained from __make_counted_resource__
  acquires shared ownership of the memory resource; the
  lifetime of the resource is extended until all copies
  of the storage pointer are destroyed.

* The storage pointer remembers the value of
  __is_deallocate_trivial__ before type-erasing the resource,
  allowing the value to be queried at run-time.

This lists all of the allocation-related types and functions
available when using the library:

[table Functions and Types
[ [Name] [Description] ]
[
    [__is_deallocate_trivial__]
    [
        A customization point allowing a memory resource type
        to indicate that calls to deallocate are trivial.
    ]
][
    [__make_counted_resource__]
    [
        A function returning a smart pointer with shared
        ownership of a newly allocated memory resource.
    ]
][
    [__memory_resource__]
    [
        The abstract base class representing an allocator.
    ]
][
    [__monotonic_resource__]
    [
        A memory resource which allocates large blocks of memory and
        has a trivial deallocate function. Allocated memory is not
        freed until the resource is destroyed, making it fast for
        parsing but not suited for performing modifications.
    ]
][
    [__null_resource__]
    [
        A memory resource always throws an exception upon allocation.
        This is used to to achieve the invariant that no parsing
        or container operation will dynamically allocate memory.
    ]
][
    [__polymorphic_allocator__]
    [
        An __Allocator__ which uses a reference to a
        __memory_resource__ to perform allocations.
    ]
][
    [__static_resource__]
    [
        A memory resource that uses a single caller provided
        buffer. No dynamic allocations are used. This is fast for
        parsing but not suited for performing modifications.
    ]
][
    [__storage_ptr__]
    [
        A smart pointer through which a __memory_resource__
        is managed and accessed.
    ]
]]

[heading Default Resource]

The library provides a ['default memory resource] object which wraps calls to the
global allocation and deallocation functions (`operator new` and `operator delete`).
This memory resource is not reference counted, and requires calls to deallocate to
free storeage. Default constructed instances of __storage_ptr__, as well as 
__storage_ptr__ which have been moved from will refer to the default memory resource
until they are either destroyed or reassigned.
Likewise, library types such as __value__, __object__, and __array__ will use the
default memory resource if one is not specified when they are constructed: 

[snippet_allocators_1]

The default memory resource is suited for general purpose operations.
It allocates only what is needed, and frees memory upon reallocation 
or destruction. It is a good choice for modifying a __value__
containing a JSON document.

[heading Monotonic Resources]

The library provides another memory resource called __monotonic_resource__,
optimized for parsing without subsequent modification. This implementation
acquires large blocks of memory and then allocates from within these blocks to satisfy
allocation requests, only ever deallocating when the memory resource is destroyed.
Every block allocated by __monotonic_resource__ will be twice as large as the last,
and an initial buffer may be optionally provide for the resource to use before
it makes any dynamic allocations. Once the initial buffer is exhausted, the 
default memory resource will be used to allocate new blocks:

[snippet_allocators_2]

The following example shows how a string can be parsed into a __value__
using a monotonic resource:

[snippet_allocators_3]

In the above sample, ownership of the resource is shared by the returned
value and its nested elements. The monotonic resource is destroyed only when
the last __value__ referencing it is destroyed.

Monotonic resources are faster for parsing and insertion, but consume more memory
than the default memory resource as adding and removing elements over time
will continuously allocate more memory to satisfy requests.

[heading Resource Lifetime]

The __value__, __object__, __array__, and __string__ classes
use __storage_ptr__ to manage and access memory resources:

[snippet_allocators_4]

A __storage_ptr__ can function as either a reference wrapper like
__polymorphic_allocator__, or as a smart pointer that
shares ownership of a __memory_resource__ through reference counting:

[snippet_allocators_5]

A storage pointer can refer to one of three kinds of memory resources:

* the default resource,

* a ['counted resource], or

* an ['uncounted resource].

A default constructed __storage_ptr__ will refer to the default resource.
Storage pointers that refer to the default memory resource do not have
ownership of the resource, and are not counted.

[snippet_allocators_6]

A storage pointer that refers to a counted resource is obtained by calling
`make_counted_resource`:

[snippet_allocators_7]

The resulting `storage_ptr` will have shared ownership of a dynamically allocated
memory resource of the specified type. This storage pointer functions similar to
__shared_ptr__, allowing for ownership to be shared through initialization and assignment:

[snippet_allocators_8]

The reference counting for __storage_ptr__ is atomic, meaning the
sharing of ownership is thread safe:

[snippet_allocators_ref_thread_safe]

[caution
    While the sharing of ownership is thread safe, the use of the
    managed resource may not be.
]

Allowing the ownership of a resource to be shared can greatly alleviate
lifetime concerns and promotes ease of use, as the lifetime of the
underlying resource is managed automatically by the storage pointers.

A __storage_ptr__ constructed from a pointer to a __memory_resource__
will refer to an uncounted resource. Such storage pointers function as
reference wrappers and do not take ownership of the resource.

[snippet_allocators_10]

[caution
    Care must be taken to ensure that the resource managed by
    a __storage_ptr__ is not accessed after that resource is destroyed.
]

Since atomic operations are relatively expensive compared to their
non-atomic counterparts, it is sometimes desirable to avoid these
operations when the lifetime of the JSON object can be bounded.
For example, consider a network server which receives a JSON
representing an RPC command. It parses the JSON, dispatches the
command, and then destroys the value. Because the
lifetime of the value is bounded by the function in which it
appears, we can use an uncounted resource to avoid the overhead
of atomic operations:

[snippet_allocators_11]

The `deallocate_is_null` function can be used to determine if the
memory resource referred to by a  __storage_ptr__ will perform no action
when `deallocate` is called. Skipping calls to `deallocate` when it
performs no action can have significant performance benefits,
as destructor calls for library types may also be elided
if the memory resource is non-counted.

[heading Custom Memory Resources]

Users who need precise control over allocation can implement
their own memory resource. Custom memory resources shall be derived from
__memory_resource__ and must implement the functions `do_allocate`,
`do_deallocate`, and `do_is_equal`. The `allocate`, `deallocate`, and
`is_equal` functions are not virtual, and generally should not be
redeclared in the custom memory resource.

The `do_allocate`, `do_deallocate` and `do_is_equal` functions are not
called directly by the user. These functions are called by
`memory_resource::allocate`, `memory_resource::deallocate`, and `memory_resource::is_equal`
with the provided arguments to fufill requests for allocation and deallocation, or
to check equality, respectively:

[snippet_allocators_12]

The function `do_is_equal` indicates whether memory allocated by a
memory resource can be deallocated by another memory resource:

[snippet_allocators_13]

The class template __is_deallocate_trivial__ can be specialized
to indicate whether calling `do_deallocate` on a memory resource
will have no effect:

[snippet_allocators_14]

The following is an example implementation of a custom memory resource
that will log allocations and deallocations, and then forward them to 
`::operator new` and `::operator delete`:

[snippet_allocators_15]

[endsect]

[/-----------------------------------------------------------------------------]

[section:uses_allocator Uses-allocator construction]

To support code bases which are already using polymorphic allocators,
the containers in this library support __std_uses_allocator__ construction.
For __array__, __object__, __string__, and __value__:

* The nested type `allocator_type` is an alias for a __polymorphic_allocator__

* All eligible constructors which accept __storage_ptr__ will also accept
    an instance of __polymorphic_allocator__ in the same argument position.

* The member function `get_allocator` returns an instance of
    __polymorphic_allocator__ constructed from the __memory_resource__
    used by the container. Ownership of this memory resource is not
    transferred.

[note
    Since __polymorphic_allocator__ does not own the memory resource,
    calls to library containers' `get_allocator` member will throw an
    exception if the associated memory resource is retained by shared
    ownership.
]

Practically, this means that when a library container type is used in a
standard container that uses a polymorphic allocator, the allocator will
propagate to the JSON type. For example:

[snippet_uses_allocator_1]

Library containers can be constructed from polymorphic allocators:

[snippet_uses_allocator_2]

The polymorphic allocator is propagated recursively.
Child elements of child elements will use the same memory
resource as the parent.

[endsect]

[/-----------------------------------------------------------------------------]

[endsect]