RecyclableMemoryStream class
December 6, 2023 · View on GitHub
MemoryStream implementation that deals with pooling and managing memory streams which use potentially large buffers.
public sealed class RecyclableMemoryStream : MemoryStream, IBufferWriter<byte>
Public Members
| name | description |
|---|---|
| RecyclableMemoryStream(…) | Initializes a new instance of the RecyclableMemoryStream class. (6 constructors) |
| override CanRead { get; } | Whether the stream can currently read. |
| override CanSeek { get; } | Whether the stream can currently seek. |
| override CanTimeout { get; } | Always false. |
| override CanWrite { get; } | Whether the stream can currently write. |
| override Capacity { get; set; } | Gets or sets the capacity. |
| Capacity64 { get; set; } | Returns a 64-bit version of capacity, for streams larger than int.MaxValue in length. |
| override Length { get; } | Gets the number of bytes written to this stream. |
| override Position { get; set; } | Gets the current position in the stream. |
| Advance(…) | Notifies the stream that count bytes were written to the buffer returned by GetMemory or GetSpan. Seeks forward by count bytes. |
| override Close() | Equivalent to Dispose. |
| override CopyTo(…) | |
| override CopyToAsync(…) | Asynchronously reads all the bytes from the current position in this stream and writes them to another stream. |
| override GetBuffer() | Returns a single buffer containing the contents of the stream. The buffer may be longer than the stream length. |
| GetMemory(…) | |
| GetReadOnlySequence() | Returns a sequence containing the contents of the stream. |
| GetSpan(…) | |
| override Read(…) | Reads from the current position into the provided buffer. (2 methods) |
| override ReadByte() | Reads a single byte from the current position in the stream. |
| SafeRead(…) | Reads from the specified position into the provided buffer. (2 methods) |
| SafeReadByte(…) | Reads a single byte from the specified position in the stream. |
| override Seek(…) | Sets the position to the offset from the seek location. |
| override SetLength(…) | Sets the length of the stream. |
| override ToArray() | Returns a new array with a copy of the buffer's contents. You should almost certainly be using GetBuffer combined with the Length to access the bytes in this stream. Calling ToArray will destroy the benefits of pooled buffers, but it is included for the sake of completeness. |
| override ToString() | Returns a useful string for debugging. This should not normally be called in actual production code. |
| override TryGetBuffer(…) | Returns an ArraySegment that wraps a single buffer containing the contents of the stream. |
| override Write(…) | Writes the buffer to the stream. (2 methods) |
| override WriteByte(…) | Writes a single byte to the current position in the stream. |
| override WriteTo(…) | Synchronously writes this stream's bytes to the argument stream. |
| WriteTo(…) | Synchronously writes this stream's bytes, starting at offset, for count bytes, to the argument stream. (4 methods) |
Protected Members
| name | description |
|---|---|
| override Dispose(…) | Returns the memory used by this stream back to the pool. |
| override Finalize() | The finalizer will be called when a stream is not disposed properly. |
Remarks
This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream-derived objects to callers, while avoiding these specific problems:
- LOH allocations – Since all large buffers are pooled, they will never incur a Gen2 GC
- Memory waste – A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
- Memory copying – Each time a
MemoryStreamgrows, all the bytes are copied into new buffers. This implementation only copies the bytes whenGetBufferis called. - Memory fragmentation – By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.
The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.
The biggest wrinkle in this implementation is when GetBuffer is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).
Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).
A further wrinkle is what happens when the stream is longer than the maximum allowable array length under .NET. This is allowed when only blocks are in use, and only the Read/Write APIs are used. Once a stream grows to this size, any attempt to convert it to a single buffer will result in an exception. Similarly, if a stream is already converted to use a single larger buffer, then it cannot grow beyond the limits of the maximum allowable array size.
Any method that modifies the stream has the potential to throw an OutOfMemoryException, either because the stream is beyond the limits set in RecyclableStreamManager, or it would result in a buffer larger than the maximum array size supported by .NET.
See Also
- namespace Microsoft.IO