Future work

April 25, 2025 ยท View on GitHub

Aside from obvious things like EQUAL? semantics on map and set keys.

Eliminate ALEXANDRIA and SERAPEUM package dependencies

The size of the these packages is not worth the use they get in CLJ-COLL, so at some point they may be removed as dependencies. This should not affect any CLJ-COLL users except for poorly formed ASDF system definitions. If your system relies on either of those packages, you should declare them as dependencies of your system.

Unification of Clojure-in-CL libs

I hope that with the advent of CLJ-COLL and my other libraries for Clojure (arrow macros, concurrency, and regexp processing), that there might someday be "one library to bind them all" into a predefined CLJ-USER package.

COMPARE logic

Defining comparison behaviors for CLJ-COLL entities for use in sorting contexts such as sorted sets.

Clojure str package

I briefly considered addressing str and the necessary tostring() type of behavior needed for it with CLJ-COLL, but in the end I decided to leave it for some other work, it really has little to do with collections and seqs and as with most of this stuff the devil is in the details (for example, how it interacts with PRINT-OBJECT or other mechanisms).

Clojure syntax for defn, fn, and other destructuring contexts

The vector and map syntax supported by CLJ-COLL might be sufficient to implement Clojure syntax for defn, fn, and general destructuring such as a clj-coll:let emulating clojure's. I'm not sure if it would work, or if the quoted syntax limitation precludes it, I haven't thought about it. It also isn't on my short list, because other than destructuring CLJ-COLL is most of what I needed to do my day to day hacking, though I did make a defn-ish internal macro to better deal with all the multi-arity APIs, which really clutter up the code if you have to declare every multi-arity function like this:

(defun foo (pred init-or-coll &optional (coll nil coll-suppied-p)))

CL lambda lists work fine if you're thinking in CL, but not so well for Clojure's arity-focused API.

defn-ish is another thing which might benefit from some tuning and compiler macros, we could probably avoid the runtime arity check entirely.

More testing

Most of our stateful transducers with completion steps could probably use more tests that actually ensure it's working. (sequence (comp (xf) (xf) (xf)) [input seq]) is sometimes a good way to do that.

Streams as seqs?

[flexi-]stream types that might be treated as collections/seqs for the CLJ-COLL APIs? Of course I could make a StreamSeq type perhaps, it would be nice if that could be user extensible.

Performance thoughts

These may be incoherent, some are dated/stupid/impossible. Move along.

Unbundling iterator state from iterator functions (internal)

Performance: For TAKE-LAST, and perhaps other seq-rewinding uses, would be nice to have an iterator capability that let's use explicitly manage/save state so that we could save points in iteration and to back to them. Of course seqs will do this, they just cons a lot, where an iterator state is an O(1) thing without O(n) allocations following when it's traversed. Current functional clj-coll iterators close over state and can't do this without adding an undesirable argument to the iterator function.

Simple-vector collectors

Allow make-collector to create non-adjusteble vectors, and maintain them with svref. The CLJ-COLL apis that enlarge vectors know how to deal with non-adjustable vectors now (they didn't for most of the project).

Faster FSet seqs

Use FSet iterators for set & map seqs, and CLJ-COLL iterators? We could use them and it would make the creation of the seqs be space O(1) instead of O(n) (because we currently copy keys). However we'd have to implement a thread-safe locking iterator like Clojure's IteratorSeq.java to make sure an iterator is used only once for each value.

Seq/collection-specific traversals where needed

Using the seq/next/first pattern for all collection types is the last refuge internally. ArraySeqs and such cons way too much. I think I've done the most important APIs to be efficient with type-specific iteration, but there may be more that coulid benefit from it. Note that the internal colliter.lisp iterators are very fast and non-consing except for the make-iterator call. Don't hesitate to use them.

Consider sealing CLJ-COLL generic functions?

See https://github.com/marcoheisig/fast-generic-functions.

Multiple values for TAKE/DROP?

If ya got 'em, smoke 'em (re: multiple values).

Some nontrivial code was already written to avoid seq re-traversal instead of simpler but less efficient take and drop use in functions like partition.

Another path might have been to have various seq-traversing functions return a second value. For example take can return the collection, but also the seq pointing to the next element.

Then callers like 'partition' don't need to re-traverse what was taken to drop the same elements doubling up on the number of passes on the input.

This might have been a nice CL-ism that could augment clojure APIs without altering them in their primary-value semantics, but at this point perhaps most of the harder "avoid-retraversal" work has been done.

Compiler macros for some arglist transforms

Can we use compiler macros to eliminate the CASE on the length of a &rest list, and call directly to some function with the correct arity for defn-ish type situations would probably be good. Not only does it eliminate the &rest and count/case overhead, but we can also then define the function return values of distinct arities. E.g. map returns a collection with on arity, and a function with the transucer arity. There is a downside to this, tracing a defn-ish function would require chasing each of the correct arity function namesakes. I.e. (defn foo ([a]) ([a b])) might translate to functions foo%1, foo%2, and so on.