Rebol/Zstd

Zstandard compression extension for Rebol3 (version 3.20.5 or newer)

Basic usage

Use Zstandard as a codec for the standard compress and decompress functions:

import zstd
bin: compress "some data" 'zstd
txt: to string! decompress bin 'zstd

Streaming API

The streaming API lets you process data in chunks, which is useful for large inputs, pipes, or network streams.

zstd: import zstd        ;; Import the module and assign it to a variable
enc: zstd/make-encoder   ;; Initialize the Zstandard encoder state handle
zstd/write :enc "Hello"  ;; Process some input data
zstd/write :enc " "
zstd/write :enc "Zstandard"
;; When there is enough data to compress,
;; use `read` to finish the current data block and get the encoded chunk
bin1: zstd/read :enc
;; Continue with other data and use `/finish` to encode all remaining input
;; and mark the stream as complete.
bin2: zstd/write/finish :enc " from Rebol!"

;; Decompress both compressed blocks again (using extension's command this time):
text: to string! zstd/decompress join bin1 bin2
;== "Hello Zstandard from Rebol!"

;; Or using streaming API:
dec: zstd/make-decoder   ;; Initialize the Zstandard decoder state handle
zstd/write :dec bin1     ;; Feed some compressed chunks
zstd/write :dec bin2
text: to string! zstd/read :dec ;; Resolve decompressed data
;== "Hello Zstandard from Rebol!"

Example: file helpers

These (basic) examples show how to build file-level utilities on top of the streaming API.

compress-file: function[file][
    src: open/read file                 ;; input file
    out: open/new/write join file %.zst ;; output file
    enc: zstd/make-encoder/level 6      ;; initialize Zstandard encoder
    chunk-size: 65536
    while [not finish][
        chunk: copy/part src chunk-size
        ;; If length of the chunk is less than chunk-size,
        ;; it must be the last chunk and we can finish the stream.
        finish: chunk-size > length? chunk
        ;; Flush output after each chunk.
        write out zstd/write/flush/:finish :enc :chunk
    ]
    close src
    close out
]
decompress-file: function[file][
    src: open/read file                 ;; input file
    dec: zstd/make-decoder              ;; initialize Zstandard decoder
    chunk-size: 65536
    while [not empty? chunk: copy/part src chunk-size][
        zstd/write :dec :chunk
    ]
    close src
    zstd/read :dec
]

Extension commands:

version

Native Zstd version

compress :data

Compress data using Zstandard

• data [binary! any-string!] Input data to compress.
• /part Limit the input data to a given length.
• length [integer!] Length of input data.
• /level
• quality [integer!] Compression level from 1 to 22.

decompress :data

Decompress data using Zstandard

• data [binary! any-string!] Input data to decompress.
• /part Limit the input data to a given length.
• length [integer!] Length of input data.
• /size Limit the output size.
• bytes [integer!] Maximum number of uncompressed bytes.

make-encoder

Create a new Zstandard encoder handle.

• /level
• quality [integer!] Compression level from 0 to 11.

make-decoder

Create a new Zstandard decoder handle.

write :codec :data

Feed data into a Zstandard streaming codec.

• codec [handle!] Zstandard encoder or decoder handle.
• data [binary! any-string! none!] Data to compress or decompress, or NONE to finish the stream.
• /flush Finish the current data block and return the encoded chunk.
• /finish Encode all remaining input and mark the stream as complete.

read :codec

Retrieve pending encoded or decoded data from the stream.

• codec [handle!] Zstandard encoder or decoder handle.